Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server crashes on bulk load #3881

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

Server crashes on bulk load #3881

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2015-12-07 16:30:59 +0100
From: Daniel Sali <<daniel.sali>>
To: SQL devs <>
Version: 11.21.11 (Jul2015-SP1)
CC: @njnes

Last updated: 2016-03-25 09:59:13 +0100

Comment 21642

Date: 2015-12-07 16:30:59 +0100
From: Daniel Sali <<daniel.sali>>

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36
Build Identifier:

We are trying to load a large amount of data using Pentaho Kettle a Java ETL suite, which creates a COPY INTO table FROM fifo file statement and uses the driver. The Java process makes a COPY INTO call every 100,000 records, but after about 32 million records loaded, the server process dies with a segmentation fault.

Reproducible: Always

Steps to Reproduce:

  1. Load 100000 records 33-34 times with COPY INTO table

Actual Results:

The server process died with a segmentation fault.

Expected Results:

Records should load successfully.

We have attached gdb to the mserver process after repeatedly experiencing the issue, and the following stack trace was captured:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f7f54bef700 (LWP 21904)]
0x00007f7f5a9460b2 in ebat_copy () from /usr/lib64/monetdb5/lib_sql.so
(gdb) where
0 0x00007f7f5a9460b2 in ebat_copy () from /usr/lib64/monetdb5/lib_sql.so
1 0x00007f7f5a9437bc in dup_bat () from /usr/lib64/monetdb5/lib_sql.so
2 0x00007f7f5a9447eb in append_col () from /usr/lib64/monetdb5/lib_sql.so
3 0x00007f7f5a833ecf in mvc_append_wrap () from /usr/lib64/monetdb5/lib_sql.so
4 0x00007f7f679c41c8 in runMALsequence () from /lib64/libmonetdb5.so.19
5 0x00007f7f679c514a in callMAL () from /lib64/libmonetdb5.so.19
6 0x00007f7f5a845acf in SQLexecutePrepared () from /usr/lib64/monetdb5/lib_sql.so
7 0x00007f7f5a845fca in SQLengineIntern () from /usr/lib64/monetdb5/lib_sql.so
8 0x00007f7f679df1c0 in runScenarioBody () from /lib64/libmonetdb5.so.19
9 0x00007f7f679dfcad in runScenario () from /lib64/libmonetdb5.so.19
10 0x00007f7f679e00f0 in MSserveClient () from /lib64/libmonetdb5.so.19
11 0x00007f7f679e06fb in MSscheduleClient () from /lib64/libmonetdb5.so.19
12 0x00007f7f67a8564c in doChallenge () from /lib64/libmonetdb5.so.19
13 0x00007f7f674b5d1f in thread_starter () from /lib64/libbat.so.12
14 0x00007f7f64ff7df5 in start_thread (arg=0x7f7f54bef700) at pthread_create.c:308
15 0x00007f7f64d1e1ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

The output of the 'list' gdb command:
(gdb) list
76 else
77
78 /* This is a "normal" system call stub: if there is an error,
79 it returns -1 and sets errno. */
80
81 T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
82 ret
83 T_PSEUDO_END (SYSCALL_SYMBOL)
84
85 endif

Thread dump will be added as an attachment.

Thanks,
Daniel Sali

Comment 21643

Date: 2015-12-07 16:32:21 +0100
From: Daniel Sali <<daniel.sali>>

Created attachment 376
Output of the 'thr app all bt' gdb command

Attached file: monetdb_thread_dump.txt (text/plain, 8686 bytes)
Description: Output of the 'thr app all bt' gdb command

Comment 21649

Date: 2015-12-08 11:25:29 +0100
From: Daniel Sali <<daniel.sali>>

Please note that though the symptoms are similar to Bug #3848, the stack trace is different.

Comment 21714

Date: 2016-01-06 13:30:21 +0100
From: @njnes

I'm tried to reproduce this problem (and failed). Could you give some more details about the data types your loading. A schema /create table statement could help a lot. On what system are you running, do you have enough diskspace available for the db?

Comment 21894

Date: 2016-03-14 08:30:36 +0100
From: @njnes

the ebat_copy code has now more protection against out of memory situations.

Comment 21957

Date: 2016-03-25 09:59:13 +0100
From: @sjoerdmullender

Jul2015-SP3 has been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant