Query on large string table fails on HEAPextend

[monetdb-team]

Date: 2015-08-13 14:54:30 +0200
From: @hannesmuehleisen
To: GDK devs <>
Version: 11.21.1 (Jul2015)

Last updated: 2015-11-03 10:18:19 +0100

Comment 21139

Date: 2015-08-13 14:54:30 +0200
From: @hannesmuehleisen

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
Build Identifier:

Reproducible: Always

Steps to Reproduce:

1. Run mserver5 with gdk_nr_threads to largish number, e.g. 96 (like the default on diamonds1)
2. CREATE TABLE s(s STRING);
3. Insert significant amount of unique strings, e.g. 10M strings with 10 random characters, in our example this leads to a string heap of ~24 GB.
4. SELECT * FROM s;

Actual Results:

Query fails,
HEAPextend: failed to extend to 2303616614400 for 03/324.theap: h->storage == STORE_MEM && can_map && fd >= 0 && HEAPload_intern() != GDK_SUCCEED

Expected Results:

Query should run, even though its result is admittedly very large.

From debugging, we found that MATpackIncrement() calls HEAPextend()

Comment 21141

Date: 2015-08-13 16:18:11 +0200
From: MonetDB Mercurial Repository <>

Changeset 32935e5e09fb made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=32935e5e09fb

Changeset description:

Don't extend string heaps since they may get replaced due to sharing.
This fixes bug #3789.

Comment 21303

Date: 2015-09-29 14:28:23 +0200
From: @sjoerdmullender

I'm assuming this was indeed fixed.

Comment 21439

Date: 2015-11-03 10:18:19 +0100
From: @sjoerdmullender

Jul2015 SP1 has been released.