Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory leak (of InstrRecord) in opt_mergetable #3732

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

memory leak (of InstrRecord) in opt_mergetable #3732

monetdb-team opened this issue Nov 30, 2020 · 0 comments
Labels

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2015-05-29 15:08:02 +0200
From: Richard Hughes <<richard.monetdb>>
To: MonetDB5 devs <>
Version: 11.19.11 (Oct2014-SP3)
CC: @njnes

Last updated: 2015-08-28 13:42:00 +0200

Comment 20899

Date: 2015-05-29 15:08:02 +0200
From: Richard Hughes <<richard.monetdb>>

Created attachment 335
proposed fix

3,984 bytes in 18 blocks are definitely lost in loss record 1,879 of 2,145
at 0x4C28C20: malloc (vg_replace_malloc.c:296)
by 0x5478F28: GDKmalloc_prefixsize (gdk_utils.c:652)
by 0x5478F28: GDKmallocmax (gdk_utils.c:678)
by 0x5478F28: GDKmalloc (gdk_utils.c:701)
by 0x4E78530: pushArgument (mal_instruction.c:1603)
by 0x4F8DDA0: mat_topn (opt_mergetable.c:1367)
by 0x4F91497: OPTmergetableImplementation (opt_mergetable.c:1541)
by 0x4FA5AF3: OPTwrapper (opt_wrapper.c:176)
by 0x4F9FE5B: optimizeMALBlock (opt_support.c:292)
by 0xDBD2E95: addQueryToCache (sql_optimizer.c:283)
by 0xDBD2630: backend_dumpproc (sql_gencode.c:2703)
by 0xDBB6FB8: SQLparser (sql_scenario.c:2535)
by 0x4E96018: runPhase (mal_scenario.c:526)
by 0x4E96018: runScenarioBody (mal_scenario.c:561)
by 0x4E96CCC: runScenario (mal_scenario.c:590)
by 0x4E97157: MSserveClient (mal_session.c:450)
by 0x4E9774E: MSscheduleClient (mal_session.c:330)
by 0x4F3C545: doChallenge (mal_mapi.c:192)
by 0x5502F4E: thread_starter (gdk_system.c:535)
by 0x669F0A3: start_thread (pthread_create.c:309)
by 0x699A04C: clone (clone.S:111)

[line numbers are against Oct2014 head as of today (66be10fc2789), i.e. http://dev.monetdb.org/hg/MonetDB/file/9bfe2441d4a3/monetdb5/optimizer/opt_mergetable.c]

I'm not actually sure what query caused this - I noticed memory leaking so I just ran an entire day's worth of queries under valgrind to see what came out. If you need more information then let me know and I'll try to narrow it down a bit better.

The attached patch fixes what I reckon is causing the leak of 'pck', and also tweaks the handling of 'gpck' which looks wrong to me as well (although I don't know what query to run to cause that code path to execute). Please check my work carefully because I'm not very confident about the correctness of this patch.

Attached file: opt_mergetable-leak.diff (text/plain, 775 bytes)
Description: proposed fix

Comment 20909

Date: 2015-06-08 13:57:08 +0200
From: Richard Hughes <<richard.monetdb>>

Here's a repro for you:

create table foo (a int);
insert into foo (a) select * from generate_series(cast(0 as int),5000);
select a from foo order by a limit 10;

The select statement will cause:
232 bytes in 1 blocks are definitely lost in loss record 1,473 of 1,966
at 0x4C28C20: malloc (vg_replace_malloc.c:296)
by 0x5478F28: GDKmalloc_prefixsize (gdk_utils.c:652)
by 0x5478F28: GDKmallocmax (gdk_utils.c:678)
by 0x5478F28: GDKmalloc (gdk_utils.c:701)
by 0x4E78530: pushArgument (mal_instruction.c:1603)
by 0x4F8DDA0: mat_topn (opt_mergetable.c:1367)
by 0x4F91497: OPTmergetableImplementation (opt_mergetable.c:1541)
by 0x4FA5AF3: OPTwrapper (opt_wrapper.c:176)
by 0x4F9FE5B: optimizeMALBlock (opt_support.c:292)
by 0xDBD2E95: addQueryToCache (sql_optimizer.c:283)
by 0xDBD2630: backend_dumpproc (sql_gencode.c:2703)
by 0xDBB6FB8: SQLparser (sql_scenario.c:2535)
by 0x4E96018: runPhase (mal_scenario.c:526)
by 0x4E96018: runScenarioBody (mal_scenario.c:561)
by 0x4E96CCC: runScenario (mal_scenario.c:590)
by 0x4E97157: MSserveClient (mal_session.c:450)
by 0x4E9774E: MSscheduleClient (mal_session.c:330)
by 0x4F3C545: doChallenge (mal_mapi.c:192)
by 0x5502F4E: thread_starter (gdk_system.c:535)
by 0x669F0A3: start_thread (pthread_create.c:309)
by 0x699A04C: clone (clone.S:111)

Comment 20991

Date: 2015-07-10 21:20:58 +0200
From: @njnes

fixed in jul2015 branch, the generators optimizer leaked

Comment 20998

Date: 2015-07-14 11:46:23 +0200
From: Richard Hughes <<richard.monetdb>>

Still reproducible in Jul2015 (3f8fe587d596), I'm afraid.

My repro in Comment 1 is flawed. Change the insert to:

insert into foo (a) select * from generate_series(cast(0 as int),500000);

and you can reproduce without forcemito. It's definitely the select statement that's doing it because I'm restarting mserver5 between the insert and select.

184 bytes in 1 blocks are definitely lost in loss record 1,327 of 1,811
at 0x4C28C20: malloc (vg_replace_malloc.c:296)
by 0x56B6487: GDKmalloc_prefixsize (gdk_utils.c:641)
by 0x56B653E: GDKmallocmax (gdk_utils.c:667)
by 0x56B6646: GDKmalloc (gdk_utils.c:690)
by 0x4E8543B: pushArgument (mal_instruction.c:1582)
by 0x5045E7D: mat_topn (opt_mergetable.c:1371)
by 0x5046DD9: OPTmergetableImplementation (opt_mergetable.c:1548)
by 0x505D8F8: OPTwrapper (opt_wrapper.c:149)
by 0x5058787: optimizeMALBlock (opt_support.c:262)
by 0xF488011: addQueryToCache (sql_optimizer.c:294)
by 0xF485BA5: backend_dumpproc (sql_gencode.c:2804)
by 0xF451D9B: SQLparser (sql_scenario.c:1178)

Comment 21068

Date: 2015-07-29 17:30:28 +0200
From: MonetDB Mercurial Repository <>

Changeset 3ded79fe4907 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=3ded79fe4907

Changeset description:

Don't claim instruction was pushed when it wasn't.
ml->v[piv] refers to the instruction "pck" that was added by the
mat_add_var call.  That instruction is never added to the MAL plan.

This should fix bug #3732.

Comment 21071

Date: 2015-07-30 13:09:10 +0200
From: @sjoerdmullender

I'll go out on a limb and say this bug has now been fixed.

Comment 21072

Date: 2015-07-31 11:16:10 +0200
From: Richard Hughes <<richard.monetdb>>

Confirmed. Thanks Sjoerd.

Comment 21197

Date: 2015-08-28 13:42:00 +0200
From: @sjoerdmullender

Jul2015 has been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant