un-rolled-back savepoints cause crash upon scenario exit

[monetdb-team]

Date: 2010-11-03 11:47:39 +0100
From: @grobian
To: SQL devs <>
Version: 11.5.3 (Aug2011-SP1) [obsolete]
CC: @drstmane, taowen

Last updated: 2011-10-27 20:55:23 +0200

Comment 15167

Date: 2010-11-03 11:47:39 +0100
From: @grobian

As revealed by insert_values.SF-1578838 and savepoint-SF.1898533, the server crashes upon destroying of the sql scenario when a savepoint was used which is not rolled back to. Commenting out savepoints which are not rolled back to, resolves the crash.

Program terminated with signal 6, Aborted.
0 0x0000003b6d2327f5 in raise () from /lib64/libc.so.6
(gdb) bt
0 0x0000003b6d2327f5 in raise () from /lib64/libc.so.6
1 0x0000003b6d233fd5 in abort () from /lib64/libc.so.6
2 0x0000003b6d22b785 in __assert_fail () from /lib64/libc.so.6
3 0x00007f34c9575115 in decref (i=1326, logical=1, releaseShare=0, lock=1)
at /ufs/fabian/scratch/monetdb/Oct2010/common/src/gdk/gdk_bbp.mx:2004
4 0x00007f34c9576086 in BBPdecref (i=1326, logical=1)
at /ufs/fabian/scratch/monetdb/Oct2010/common/src/gdk/gdk_bbp.mx:2107
5 0x00007f34b00508c1 in temp_destroy (b=1326)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/bat/bat_utils.mx:137
6 0x00007f34b004d6b1 in destroy_delta (b=0x7f34a00190f8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/bat/bat_storage.mx:1009
7 0x00007f34b004d73e in destroy_bat (tr=0x0, b=0x7f34a00190f8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/bat/bat_storage.mx:1021
8 0x00007f34b004d783 in destroy_col (tr=0x0, c=0x7f34a0019058)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/bat/bat_storage.mx:1029
9 0x00007f34b0036713 in column_destroy (c=0x7f34a0019058)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/store.mx:157
10 0x00007f34b005d43b in node_destroy (l=0x7f34a0019178, n=0x7f34a00191b8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:75
11 0x00007f34b005d4a4 in list_destroy (l=0x7f34a0019178)
---Type to continue, or q to quit---
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:90
12 0x00007f34b005ebbf in cs_destroy (cs=0x7f34a00186e8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_changeset.mx:50
13 0x00007f34b00367ae in table_destroy (t=0x7f34a0018698)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/store.mx:170
14 0x00007f34b005d43b in node_destroy (l=0x7f34a00195f8, n=0x7f34a0019638)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:75
15 0x00007f34b005d4a4 in list_destroy (l=0x7f34a00195f8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:90
16 0x00007f34b005ebbf in cs_destroy (cs=0x7f34a00182f8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_changeset.mx:50
17 0x00007f34b003687a in schema_destroy (s=0x7f34a00182d8)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/store.mx:189
18 0x00007f34b005d43b in node_destroy (l=0x7f34a0048648, n=0x7f34a0048688)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:75
19 0x00007f34b005d4a4 in list_destroy (l=0x7f34a0048648)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_list.mx:90
20 0x00007f34b005ebbf in cs_destroy (cs=0x7f34a0018280)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/common/sql_changeset.mx:50
21 0x00007f34b00369b2 in sql_trans_destroy (t=0x7f34a0018258)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/storage/store.mx:224
22 0x00007f34affef198 in mvc_commit (m=0x7f34a0005468, chain=0, name=0x0)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/server/sql_mvc.mx:456
---Type to continue, or q to quit---
23 0x00007f34aff5f2e2 in SQLtransaction (cntxt=0x605898, mb=0x7f34a8114ba8,
stk=0x7f34a00828c8, pci=0x7f34a005c798)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/backends/monet5/sql.mx:1541
24 0x00007f34ca08ba13 in runMALsequence (cntxt=0x605898, mb=0x7f34a8114ba8,
startpc=1, stoppc=0, stk=0x7f34a00828c8, env=0x0, pcicaller=0x0)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_interpreter.mx:2105
25 0x00007f34ca079e86 in runMAL (cntxt=0x605898, mb=0x7f34a8114ba8,
startpc=1, mbcaller=0x0, env=0x0, pcicaller=0x0)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_interpreter.mx:339
26 0x00007f34affac2f8 in SQLengineIntern (c=0x605898, be=0x7f34a0053538)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/backends/monet5/sql_scenario.mx:1547
27 0x00007f34affac712 in SQLengine (c=0x605898)
at /ufs/fabian/scratch/monetdb/Oct2010/sql/src/backends/monet5/sql_scenario.mx:1628
28 0x00007f34ca0c1833 in runPhase (c=0x605898, phase=4)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_scenario.mx:603
29 0x00007f34ca0c1a0a in runScenarioBody (c=0x605898)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_scenario.mx:653
30 0x00007f34ca0c1c32 in runScenario (c=0x605898)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_scenario.mx:684
31 0x00007f34ca07816a in MSserveClient (dummy=0x605898)
at /ufs/fabian/scratch/monetdb/Oct2010/five/src/mal/mal_session.mx:484
---Type to continue, or q to quit---
32 0x0000003b6de068e0 in start_thread () from /lib64/libpthread.so.0
33 0x0000003b6d2e0c9d in clone () from /lib64/libc.so.6
34 0x0000000000000000 in ?? ()
(gdb)

The column that is about to be destroyed is schemas.id, which probably shouldn't be freed. Probably the left-over savepoint causes this to happen.

Comment 15169

Date: 2010-11-03 12:25:16 +0100
From: @drstmane

*** Bug #2711 has been marked as a duplicate of this bug. ***

Comment 15170

Date: 2010-11-03 12:26:12 +0100
From: @drstmane

For convenience, here's also the original posting of bug #2711:

SQL tests sql/src/test/BugTracker/Tests/insert_values.SF-1578838.sql &
sql/src/test/BugTracker/Tests/savepoint-SF.1898533.sql trigger an assertion
failure in BBPdecref:
"
MonetDB/src/gdk/gdk_bbp.mx:2198: decref: Assertion `0' failed.
"

cf., Current testing (always with assertion enabled)

http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests103/Int.64.32.d.1-Fedora10/src_test_BugTracker/insert_values.SF-1578838.err.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Current/sql/.mTests103/GNU.64.64.d.1-Fedora10/src_test_BugTracker/savepoint-SF.1898533.err.00.html

or Candidate testing on odd days (with assertion disabled)

http://monetdb.cwi.nl/testing/projects/monetdb/Candidate/sql/.mTests103/Int.64.32.d.1-Fedora10/src_test_BugTracker/insert_values.SF-1578838.err.00.html
http://monetdb.cwi.nl/testing/projects/monetdb/Candidate/sql/.mTests103/GNU.64.64.d.1-Fedora10/src_test_BugTracker/savepoint-SF.1898533.err.00.html

With assertions disabled (e.g., Candidate testing on even days), the first test
works fine

http://monetdb.cwi.nl/testing/projects/monetdb/Candidate/sql/.mTests103/Int.64.32.d.1-Fedora10/src_test_BugTracker/insert_values.SF-1578838.err.00.html

while the second test yields a recounting error:
"
QUERY = commit;
ERROR = !MALException:!ERROR: BBPdecref: tmp_2472 does not have logical
references.
"

http://monetdb.cwi.nl/testing/projects/monetdb/Candidate/sql/.mTests103/GNU.64.64.d.1-Fedora10/src_test_BugTracker/savepoint-SF.1898533.err.00.html

Comment 15187

Date: 2010-11-11 11:25:13 +0100
From: @sjoerdmullender

Unblock this bug.
This is a somewhat unusual situation, so it doesn't warrant blocking a release.
It should still be fixed, though.

Comment 15602

Date: 2011-03-18 10:25:10 +0100
From: @grobian

in Mar2011, this test causes decref errors on all non-Linux platforms

http://monetdb.cwi.nl/testing/projects/monetdb/testweb/web/testgrid.php?serial=39522:49d57d0ded3b&order=platform,compiler,arch&module=sql

e.g.
http://monetdb.cwi.nl/testing/projects/monetdb/testweb/tests/39522:49d57d0ded3b/GNU-Darwin-powerpc/sql/mTests/test/BugTracker/savepoint-SF.1898533.err.diff.html

Comment 15686

Date: 2011-03-28 17:36:32 +0200
From: @sjoerdmullender

The Mar2011 version has been released.

Comment 15938

Date: 2011-07-13 16:00:36 +0200
From: @sjoerdmullender

The two tests run fine for me, but I see that they fail on various systems (not all) during nightly testing. Can anybody explain this?

Comment 15944

Date: 2011-07-13 22:20:06 +0200
From: @drstmane

Works for me when I run the test in isolation.

Fails when I run all tests in sql/test/BugTracker/Tests/ .

Looks like some other test earlier in that directory leaves some remains that influence this test ...

Comment 15945

Date: 2011-07-13 22:37:37 +0200
From: @drstmane

running test authorization.SF-1430616 before test savepoint-SF.1898533 seems to trigger the failure of the latter:

.../MonetDB/sql/test/BugTracker/Tests $ Mtest.py -q . savepoint-SF.1898533
.
.../MonetDB/sql/test/BugTracker/Tests $ Mtest.py -q . authorization.SF-1430616 savepoint-SF.1898533
..

• (savepoint-SF.1898533.test.err.FILTERED) significantly

Comment 15978

Date: 2011-07-19 19:51:02 +0200
From: @grobian

I still have this on Apr2011-SP2_pre:

.......................................................
insert_values.SF-1578838 : Crashed!

• (insert_values.SF-1578838.test.err.FILTERED) significantly
  ...................................................................
  savepoint-SF.1898533 : Crashed!

Comment 16038

Date: 2011-07-29 11:00:05 +0200
From: @sjoerdmullender

Apr2011-SP2 has been released.

Comment 16275

Date: 2011-09-16 15:10:38 +0200
From: @sjoerdmullender

The Aug2011 version has been released.

Comment 16436

Date: 2011-10-17 10:33:50 +0200
From: @sjoerdmullender

This seems to be fixed by changeset a9fb3d2364d7

Comment 16490

Date: 2011-10-27 20:55:23 +0200
From: @sjoerdmullender

*** Bug #2913 has been marked as a duplicate of this bug. ***