Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mserver5 crash: infinite recursive happens at rel_bin.c:489 #6664

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

mserver5 crash: infinite recursive happens at rel_bin.c:489 #6664

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2018-11-29 17:11:12 +0100
From: Trung Le <>
To: SQL devs <>
Version: 11.31.11 (Aug2018-SP1)
CC: @PedroTadim

Last updated: 2019-01-14 17:29:14 +0100

Comment 26702

Date: 2018-11-29 17:11:12 +0100
From: Trung Le <>

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
Build Identifier:

When I execute a query with my 200GB Monetdb instance, mserver5 crashes. The stack trace is provided below.

Hardware: Dell R740. 32 CPU core, 768 GB Ram.
OS: Centos 7.5, running Oracle UEK R5 kernel 4.14.35-1818.4.7.el7uek.x86_64

dmesg output:
[1050368.792302] mserver5[248164]: segfault at 7fc59a1f1fd0 ip 00007fc677869a6f sp 00007fc59a1f1fc0 error 6 in lib_sql.so[7fc677827000+181000]
[1050375.992003] mserver5[248247]: segfault at 7fd6e55ebfd0 ip 00007fd7a6964a6f sp 00007fd6e55ebfc0 error 6 in lib_sql.so[7fd7a6922000+181000]

Stack trace:
0 0x00007f46a3f05a6f in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e74c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:355
1 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e78c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
2 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e7cc0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
3 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e80c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
4 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e84c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
5 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e88c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
6 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e8cc0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
7 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dc5e90c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
....
9312 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42df77c1c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9313 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42dffead20, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9314 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d0666360, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9315 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d0b279c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9316 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d16bcdf0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9317 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d1c2f020, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9318 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d286bb60, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9319 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d2f011c0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9320 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=0x7f42d33d95e0, left=left@entry=0x7f42cc9e7230, right=right@entry=0x7f42cc9e7d60,
grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9321 0x00007f46a3f060a0 in exp_bin (be=be@entry=0x7f45cc005d10, e=e@entry=0x7f42d397c640, left=left@entry=0x7f42cc9e7230,
right=right@entry=0x7f42cc9e7d60, grp=grp@entry=0x0, ext=ext@entry=0x0, cnt=cnt@entry=0x0, sel=sel@entry=0x0) at rel_bin.c:489
9322 0x00007f46a3f0b30f in rel2bin_project (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f42dc39a610, refs=refs@entry=0x7f42cafae660,
topn=topn@entry=0x0) at rel_bin.c:2574
9323 0x00007f46a3f02f31 in subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42dc39a610, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5163
9324 0x00007f46a3f03d2d in rel2bin_select (refs=0x7f42cafae660, rel=0x7f42caf819e0, be=0x7f45cc005d10) at rel_bin.c:2698
9325 subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42caf819e0, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5167
9326 0x00007f46a3f0b241 in rel2bin_project (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f42caf84d40, refs=refs@entry=0x7f42cafae660,
topn=topn@entry=0x0) at rel_bin.c:2562
9327 0x00007f46a3f02f31 in subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42caf84d40, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5163
9328 0x00007f46a3f0b9ee in rel2bin_groupby (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f42caf883c0, refs=refs@entry=0x7f42cafae660)
at rel_bin.c:2779
9329 0x00007f46a3f03ffe in subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42caf883c0, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5171
9330 0x00007f46a3f02dd8 in rel2bin_union (refs=0x7f42cafae660, rel=0x7f42caf8ae40, be=0x7f45cc005d10) at rel_bin.c:2251
---Type to continue, or q to quit---
9331 subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42caf8ae40, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5151
9332 0x00007f46a3f0b9ee in rel2bin_groupby (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f42dc3606a0, refs=refs@entry=0x7f42cafae660)
at rel_bin.c:2779
9333 0x00007f46a3f03ffe in subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42dc3606a0, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5171
9334 0x00007f46a3f02dd8 in rel2bin_union (refs=0x7f42cafae660, rel=0x7f42dc363150, be=0x7f45cc005d10) at rel_bin.c:2251
9335 subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f42dc363150, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5151
9336 0x00007f46a3f0b9ee in rel2bin_groupby (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f4313626e30, refs=refs@entry=0x7f42cafae660)
at rel_bin.c:2779
9337 0x00007f46a3f03ffe in subrel_bin (be=be@entry=0x7f45cc005d10, rel=0x7f4313626e30, refs=refs@entry=0x7f42cafae660) at rel_bin.c:5171
9338 0x00007f46a3f0b241 in rel2bin_project (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f4313627760, refs=refs@entry=0x7f42cafae660,
topn=topn@entry=0x0) at rel_bin.c:2562
9339 0x00007f46a3f02f31 in subrel_bin (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f4313627760, refs=0x7f42cafae660) at rel_bin.c:5163
9340 0x00007f46a3f059da in _subrel_bin (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f4313627760, refs=) at rel_bin.c:5226
9341 0x00007f46a3f0c795 in output_rel_bin (be=be@entry=0x7f45cc005d10, rel=rel@entry=0x7f4313627760) at rel_bin.c:5249
9342 0x00007f46a3f1a904 in sql_relation2stmt (r=0x7f4313627760, be=0x7f45cc005d10) at sql_gencode.c:606
9343 backend_dumpstmt (be=be@entry=0x7f45cc005d10, mb=0x7f45cc09d010, r=r@entry=0x7f4313627760, top=top@entry=1, add_end=add_end@entry=0,
query=query@entry=0x7f42cafb5fa0 "select salecode,productname,suppliercode,supplier,careercode,career,groupcode,"group",typecode,type,\n sum(tongtondauky) as tongtondauky,\n sum(trigiatondauky) as trigiatondauky,\n sum(tongtoncuoiky) as "...) at sql_gencode.c:646
9344 0x00007f46a3ef60cc in SQLparser (c=0x7f46a44986b0) at sql_scenario.c:1226
9345 0x00007f46aa5d5438 in runPhase (phase=1, c=0x7f46a44986b0) at mal_scenario.c:510
9346 runScenarioBody (c=c@entry=0x7f46a44986b0, once=once@entry=0) at mal_scenario.c:532
9347 0x00007f46aa5d5c7d in runScenario (c=c@entry=0x7f46a44986b0, once=once@entry=0) at mal_scenario.c:569
9348 0x00007f46aa5d613f in MSserveClient (dummy=dummy@entry=0x7f46a44986b0) at mal_session.c:519
9349 0x00007f46aa5d672a in MSscheduleClient (command=command@entry=0x7f45cc086490 "(", challenge=challenge@entry=0x7f45b42f1d70 "OoQitpvNg",

 fin=0x7f45cc005c50, fout=fout@entry=0x7f45940120a0, protocol=protocol@entry=PROTOCOL_9, blocksize=blocksize@entry=8190, 
 compute_column_widths=compute_column_widths@entry=0) at mal_session.c:397

9350 0x00007f46aa642d5e in doChallenge (data=) at mal_mapi.c:279
9351 0x00007f46aa2d3eae in thread_starter (arg=) at gdk_system.c:480
9352 0x00007f46a7ceee25 in start_thread () from /lib64/libpthread.so.0
9353 0x00007f46a7a18bad in clone () from /lib64/libc.so.6

Reproducible: Always

Steps to Reproduce:

This happens everytime I run this specific query. To reproduce it, I may have to provide the DB but its size is huge (200GB). Please try taking a look at the stack trace first. Should it is not helpful enough, I will provide the DB.

Actual Results:

mserver5 crashes

Expected Results:

mserver5 should not crash

Comment 26703

Date: 2018-11-29 17:13:01 +0100
From: Trung Le <>

Full dmesg:
[1050035.113791] mserver5[246888]: segfault at 7f45b40f2fd0 ip 00007f46a3f05a6f sp 00007f45b40f2fc0 error 6 in lib_sql.so[7f46a3ec3000+181000]
[1050103.074217] mserver5[247009]: segfault at 7f26221f1fd0 ip 00007f26fb553a6f sp 00007f26221f1fc0 error 6 in lib_sql.so[7f26fb511000+181000]
[1050116.174902] mserver5[247093]: segfault at 7f4bb57ecfd0 ip 00007f4c7abd2a6f sp 00007f4bb57ecfc0 error 6 in lib_sql.so[7f4c7ab90000+181000]
[1050123.348258] mserver5[247167]: segfault at 7f399a1f1fd0 ip 00007f3a7360ba6f sp 00007f399a1f1fc0 error 6 in lib_sql.so[7f3a735c9000+181000]
[1050130.365147] mserver5[247248]: segfault at 7ffb443e2fd0 ip 00007ffbe1720a6f sp 00007ffb443e2fc0 error 6 in lib_sql.so[7ffbe16de000+181000]
[1050137.756239] mserver5[247338]: segfault at 7fc89abf6fd0 ip 00007fc987f95a6f sp 00007fc89abf6fc0 error 6 in lib_sql.so[7fc987f53000+181000]
[1050147.386239] mserver5[247414]: segfault at 7fb4c3bfefd0 ip 00007fb5c914aa6f sp 00007fb4c3bfefc0 error 6 in lib_sql.so[7fb5c9108000+181000]
[1050171.903320] mserver5[247499]: segfault at 7f4f783e2fd0 ip 00007f50156b2a6f sp 00007f4f783e2fc0 error 6 in lib_sql.so[7f5015670000+181000]
[1050183.945027] mserver5[247598]: segfault at 7f99ccbe6fd0 ip 00007f9a79f6aa6f sp 00007f99ccbe6fc0 error 6 in lib_sql.so[7f9a79f28000+181000]
[1050196.931510] mserver5[247677]: segfault at 7f050cbe6fd0 ip 00007f05b9f4ea6f sp 00007f050cbe6fc0 error 6 in lib_sql.so[7f05b9f0c000+181000]
[1050207.543134] mserver5[247760]: segfault at 7f53fcfe8fd0 ip 00007f54b23e6a6f sp 00007f53fcfe8fc0 error 6 in lib_sql.so[7f54b23a4000+181000]
[1050220.163175] mserver5[247849]: segfault at 7f3765beefd0 ip 00007f3832f21a6f sp 00007f3765beefc0 error 6 in lib_sql.so[7f3832edf000+181000]
[1050276.853480] mserver5[247967]: segfault at 7f4c01ff0fd0 ip 00007f4cd738ba6f sp 00007f4c01ff0fc0 error 6 in lib_sql.so[7f4cd7349000+181000]
[1050368.792302] mserver5[248164]: segfault at 7fc59a1f1fd0 ip 00007fc677869a6f sp 00007fc59a1f1fc0 error 6 in lib_sql.so[7fc677827000+181000]
[1050375.992003] mserver5[248247]: segfault at 7fd6e55ebfd0 ip 00007fd7a6964a6f sp 00007fd6e55ebfc0 error 6 in lib_sql.so[7fd7a6922000+181000]

Comment 26704

Date: 2018-11-29 17:55:18 +0100
From: @PedroTadim

Hello Trung Le,

Can you tell the version of MonetDB you are using? Also can you send us the crashing query along with definition of the tables it uses?

Best regards,
Pedro

Comment 26706

Date: 2018-11-30 06:51:45 +0100
From: Trung Le <>

Hi Pedro, thanks for your support. I figured out that a long IN (...) list was the root cause. mserver5 would crash if I passed a 10000 item list. But if I reduce the list to 5000 items, mserver5 runs normally. I think the bug can be closed.

Comment 26727

Date: 2018-12-13 09:42:34 +0100
From: MonetDB Mercurial Repository <>

Changeset bdfc23a47d73 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=bdfc23a47d73

Changeset description:

Check for stack space.
This should mitigate bug #6664 by giving an error instead of a crash.

Comment 26728

Date: 2018-12-13 09:43:30 +0100
From: @sjoerdmullender

I added a check that should prevent the crash. Instead it should give an error when the recursion becomes too deep.

Can you test?

Comment 26776

Date: 2019-01-04 10:01:09 +0100
From: @sjoerdmullender

Assuming fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant