mserver5 crashes with corruption, double free, invalid size or invalid pointer

[monetdb-team]

Date: 2019-09-20 17:34:07 +0200
From: @drstmane
To: SQL devs <>
Version: 11.33.11 (Apr2019-SP1)

Last updated: 2019-11-28 10:00:06 +0100

Comment 27308

Date: 2019-09-20 17:34:07 +0200
From: @drstmane

mserver5 crashes with one of the following errors:

corrupted double-linked list
corrupted size vs. prev_size
double free or corruption (out)
double free or corruption (!prev)
free(): invalid size
munmap_chunk(): invalid pointer

script to reproduce will follow.

Comment 27309

Date: 2019-09-20 17:37:17 +0200
From: @drstmane

Please find a script to reproduce the bug at
http://homepages.cwi.nl/~manegold/Bug-6764.tar

Comment 27312

Date: 2019-09-21 21:22:00 +0200
From: @drstmane

when running a debug build of mserver5,

some crashes still occur:

corrupted double-linked list
corrupted size vs. prev_size
free(): invalid size

while instead(?) of the other crashes (see initial comment), these assertions are triggered:

MonetDB/gdk/gdk_bat.c:1587: BATsetcount: Assertion b->batCapacity >= cnt' failed. MonetDB/gdk/gdk_utils.c:1789: GDKfree: Assertion (asize & 2) == 0' failed.
MonetDB/gdk/gdk_utils.c:1794: GDKfree: Assertion `((char *) s)[i] == '\xBD'' failed.

Comment 27313

Date: 2019-09-23 11:44:54 +0200
From: @drstmane

Created attachment 635
table schema

Attached file: Bug-6764-schema.ddl (text/plain, 20721 bytes)
Description: table schema

Comment 27314

Date: 2019-09-23 11:45:24 +0200
From: @drstmane

Created attachment 636
query that triggers assertion / crash

Attached file: Bug-6764-query.sql (application/sql, 83 bytes)
Description: query that triggers assertion / crash

Comment 27315

Date: 2019-09-23 11:47:07 +0200
From: @drstmane

For what it's worth,
a simpler way to reproduce the assertion / crash is by loading the data from
http://homepages.cwi.nl/~manegold/Bug-6764-data.csv.bz2
into the table defined by attached Bug-6764-schema.ddl
and then run attached Bug-6764-query.sql

Comment 27316

Date: 2019-09-23 12:34:53 +0200
From: @drstmane

ps: the copy into statement should look as follows:

COPY OFFSET 3 INTO "data-x" FROM '.../Bug-6764-data.csv.bz2' DELIMITERS ',','\n','' NULL AS '';

Comment 27317

Date: 2019-09-23 14:24:38 +0200
From: MonetDB Mercurial Repository <>

Changeset cc708f0d0b28 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see https//devmonetdborg/hg/MonetDB?cmd=changeset;node=cc708f0d0b28

Changeset description:

Make sure enough space is allocated for extents and histogram BATs.
This fixes bug #6764.

Comment 27319

Date: 2019-09-24 09:21:08 +0200
From: @drstmane

Changeset cc708f0d0b28 indeed appears to fix also the other incarnations of the bug (crash/assertion) reported here.
Thank you very much!