Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memleak (in querycache?) while querying #3136

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

Memleak (in querycache?) while querying #3136

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2012-08-02 13:44:32 +0200
From: info
To: SQL devs <>
Version: 11.11.7 (Jul2012-SP1)
CC: @njnes, @yzchang

Last updated: 2013-01-22 09:29:03 +0100

Comment 17600

Date: 2012-08-02 13:44:32 +0200
From: info

User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.107 Safari/535.1
Build Identifier:

After disconnecting the clientconnection the querycache doesn't seem to be emptied, as this leak doesn't show when using "SET cache=false".

We expected that the leak would only increase when using different queries, since using a query that has already been executed would've already
been cached. But even when using the same queries the leak continues to increase with each query.

There also seems to be a leak when only creating a clientconnection.

Reproducible: Always

Steps to Reproduce:

Test1: Queries=1
Terminal1: valgrind --leak-check=full mserver5 --set gdk_alloc_map=no --dbname=leaktest >& output_mserver5_1.txt
Terminal2: mclient -dleaktest
Terminal2: SELECT * from types LIMIT 1;
Terminal2: \q
Terminal1: \q

Test2: Queries=2
Terminal1: valgrind --leak-check=full mserver5 --set gdk_alloc_map=no --dbname=leaktest >& output_mserver5_1.txt
Terminal2: mclient -dleaktest
Terminal2: SELECT * from types LIMIT 1;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: \q
Terminal1: \q

Test3: Queries=3
Terminal1: valgrind --leak-check=full mserver5 --set gdk_alloc_map=no --dbname=leaktest >& output_mserver5_1.txt
Terminal2: mclient -dleaktest
Terminal2: SELECT * from types LIMIT 1;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: \q
Terminal1: \q

Test4: Queries=3 + "SET cache=false"
Terminal1: valgrind --leak-check=full mserver5 --set gdk_alloc_map=no --dbname=leaktest >& output_mserver5_1.txt
Terminal2: mclient -dleaktest
Terminal2: SET cache=false;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: SELECT * from types LIMIT 1;
Terminal2: \q
Terminal1: \q

Test5: Client connection only
Terminal1: valgrind --leak-check=full mserver5 --set gdk_alloc_map=no --dbname=leaktest >& output_mserver5_1.txt
Terminal2: mclient -dleaktest
Terminal2: \q
Terminal1: \q

Actual Results:

Test1: Queries=1
==24266== 40 bytes in 1 blocks are definitely lost in loss record 733 of 2,042
==24266== at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24266== by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24266== by 0x58AF3E9: GDKmalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24266== by 0x58AF475: GDKstrdup (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24266== by 0xB1C1827: SQLinitClient (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24266== by 0x4E9FFFF: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266== by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266== by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266== by 0x6EE69C9: start_thread (pthread_create.c:300)
==24266== by 0xC1376FF: ???

==24266== 4,272 bytes in 2 blocks are definitely lost in loss record 1,831 of 2,042
==24266==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24266==    by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24266==    by 0x58AF359: GDKzalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24266==    by 0x4E7D8B8: setLifespan (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266==    by 0x543BD22: OPTgroups (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266==    by 0x542719E: optimizeMALBlock (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266==    by 0xB1C92C1: addQueryToCache (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24266==    by 0xB1C8B4E: backend_dumpproc (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24266==    by 0xB1C0393: SQLparser (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24266==    by 0x4EA0069: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266==    by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24266==    by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)

==24266== LEAK SUMMARY:
==24266==    definitely lost: 4,312 bytes in 3 blocks
==24266==    indirectly lost: 0 bytes in 0 blocks
==24266==      possibly lost: 16,105,263 bytes in 114,360 blocks
==24266==    still reachable: 136,067 bytes in 226 blocks
==24266==         suppressed: 0 bytes in 0 blocks
==24266== Reachable blocks (those to which a pointer was found) are not shown.
==24266== To see them, rerun with: --leak-check=full --show-reachable=yes

Test2: Queries=2
==24292== 40 bytes in 1 blocks are definitely lost in loss record 729 of 2,040
==24292== at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24292== by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24292== by 0x58AF3E9: GDKmalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24292== by 0x58AF475: GDKstrdup (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24292== by 0xB1C1827: SQLinitClient (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24292== by 0x4E9FFFF: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292== by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292== by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292== by 0x6EE69C9: start_thread (pthread_create.c:300)
==24292== by 0xC1376FF: ???

==24292== 7,368 bytes in 3 blocks are definitely lost in loss record 1,849 of 2,040
==24292==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24292==    by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24292==    by 0x58AF359: GDKzalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24292==    by 0x4E7D8B8: setLifespan (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292==    by 0x543BD22: OPTgroups (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292==    by 0x542719E: optimizeMALBlock (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292==    by 0xB1C92C1: addQueryToCache (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24292==    by 0xB1C8B4E: backend_dumpproc (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24292==    by 0xB1C0393: SQLparser (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24292==    by 0x4EA0069: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292==    by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24292==    by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)

==24292== LEAK SUMMARY:
==24292==    definitely lost: 7,408 bytes in 4 blocks
==24292==    indirectly lost: 0 bytes in 0 blocks
==24292==      possibly lost: 16,105,295 bytes in 114,361 blocks
==24292==    still reachable: 136,067 bytes in 226 blocks
==24292==         suppressed: 0 bytes in 0 blocks
==24292== Reachable blocks (those to which a pointer was found) are not shown.
==24292== To see them, rerun with: --leak-check=full --show-reachable=yes

Test3: Queries=3
==24355== 40 bytes in 1 blocks are definitely lost in loss record 733 of 2,042
==24355== at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24355== by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24355== by 0x58AF3E9: GDKmalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24355== by 0x58AF475: GDKstrdup (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24355== by 0xB1C1827: SQLinitClient (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24355== by 0x4E9FFFF: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355== by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355== by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355== by 0x6EE69C9: start_thread (pthread_create.c:300)
==24355== by 0xC1376FF: ???

==24355== 10,464 bytes in 4 blocks are definitely lost in loss record 1,941 of 2,042
==24355==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24355==    by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24355==    by 0x58AF359: GDKzalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24355==    by 0x4E7D8B8: setLifespan (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355==    by 0x543BD22: OPTgroups (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355==    by 0x542719E: optimizeMALBlock (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355==    by 0xB1C92C1: addQueryToCache (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24355==    by 0xB1C8B4E: backend_dumpproc (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24355==    by 0xB1C0393: SQLparser (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24355==    by 0x4EA0069: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355==    by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24355==    by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)

==24355== LEAK SUMMARY:
==24355==    definitely lost: 10,504 bytes in 5 blocks
==24355==    indirectly lost: 0 bytes in 0 blocks
==24355==      possibly lost: 16,105,327 bytes in 114,362 blocks
==24355==    still reachable: 136,067 bytes in 226 blocks
==24355==         suppressed: 0 bytes in 0 blocks
==24355== Reachable blocks (those to which a pointer was found) are not shown.
==24355== To see them, rerun with: --leak-check=full --show-reachable=yes

Test4: Queries=3 + "SET cache=false"
==24374== 40 bytes in 1 blocks are definitely lost in loss record 730 of 2,036
==24374== at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24374== by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24374== by 0x58AF3E9: GDKmalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24374== by 0x58AF475: GDKstrdup (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24374== by 0xB1C1827: SQLinitClient (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24374== by 0x4E9FFFF: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374== by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374== by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374== by 0x6EE69C9: start_thread (pthread_create.c:300)
==24374== by 0xC1376FF: ???

==24374== 1,176 bytes in 1 blocks are definitely lost in loss record 1,720 of 2,036
==24374==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24374==    by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24374==    by 0x58AF359: GDKzalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24374==    by 0x4E7D8B8: setLifespan (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374==    by 0x543BD22: OPTgroups (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374==    by 0x542719E: optimizeMALBlock (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374==    by 0xB1C92C1: addQueryToCache (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24374==    by 0xB1C8B4E: backend_dumpproc (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24374==    by 0xB1C0393: SQLparser (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24374==    by 0x4EA0069: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374==    by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24374==    by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)

==24374== LEAK SUMMARY:
==24374==    definitely lost: 1,216 bytes in 2 blocks
==24374==    indirectly lost: 0 bytes in 0 blocks
==24374==      possibly lost: 16,105,232 bytes in 114,359 blocks
==24374==    still reachable: 136,067 bytes in 226 blocks
==24374==         suppressed: 0 bytes in 0 blocks
==24374== Reachable blocks (those to which a pointer was found) are not shown.
==24374== To see them, rerun with: --leak-check=full --show-reachable=yes

Test5: Client connection only
==24421== 40 bytes in 1 blocks are definitely lost in loss record 729 of 2,032
==24421== at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24421== by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24421== by 0x58AF3E9: GDKmalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24421== by 0x58AF475: GDKstrdup (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24421== by 0xB1C1827: SQLinitClient (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24421== by 0x4E9FFFF: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421== by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421== by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421== by 0x6EE69C9: start_thread (pthread_create.c:300)
==24421== by 0xC1376FF: ???

==24421== 1,176 bytes in 1 blocks are definitely lost in loss record 1,716 of 2,032
==24421==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==24421==    by 0x58AEF65: GDKmallocmax (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24421==    by 0x58AF359: GDKzalloc (in /data/programs/MonetDB/lib/libbat.so.6.0.0)
==24421==    by 0x4E7D8B8: setLifespan (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421==    by 0x543BD22: OPTgroups (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421==    by 0x542719E: optimizeMALBlock (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421==    by 0xB1C92C1: addQueryToCache (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24421==    by 0xB1C8B4E: backend_dumpproc (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24421==    by 0xB1C0393: SQLparser (in /data/programs/MonetDB/lib/monetdb5/lib_sql.so)
==24421==    by 0x4EA0069: ??? (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421==    by 0x4EA023E: runScenario (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)
==24421==    by 0x4EA118C: MSserveClient (in /data/programs/MonetDB/lib/libmonetdb5.so.12.0.0)

==24421== LEAK SUMMARY:
==24421==    definitely lost: 1,216 bytes in 2 blocks
==24421==    indirectly lost: 0 bytes in 0 blocks
==24421==      possibly lost: 16,104,160 bytes in 114,341 blocks
==24421==    still reachable: 136,067 bytes in 226 blocks
==24421==         suppressed: 0 bytes in 0 blocks
==24421== Reachable blocks (those to which a pointer was found) are not shown.
==24421== To see them, rerun with: --leak-check=full --show-reachable=yes

Expected Results:

No memory leaks

Comment 17601

Date: 2012-08-02 13:47:32 +0200
From: info

Created attachment 137
Logfiles valgrind

Attached file: queryleak.tar.bz2 (application/x-bzip, 117461 bytes)
Description: Logfiles valgrind

Comment 17635

Date: 2012-08-22 11:26:00 +0200
From: info

Update:

monetdb --version
MonetDB Database Server Toolkit v1.0 (Jul2012-SP1)

We did a new test on empty databases through the funnel,
constantly firing the same point-query.

When we DON'T use "set cache=false":
- The queries seem to get slower over time.
- The resident memory of the mservers increases very fast to 2GB then
the increase seems to slow down (possibly because the queries are a
factor 4 slower by then)

When we use set "set cache=false":
- The queries don't get slower
- The resident memory doesn't increase.
- Switching with the funnel results in the same behavior as when NOT
using "set cache=false" as "set cache=false" works per client
connection which gets disconnected at each switch.

Comment 17636

Date: 2012-08-22 11:41:02 +0200
From: @grobian

thanks for the analysis, it looks like a clear case now

Comment 17667

Date: 2012-08-24 14:55:08 +0200
From: @sjoerdmullender

Jul2012-SP1 has been released.

Comment 17700

Date: 2012-08-24 15:25:09 +0200
From: @sjoerdmullender

After my fixes for bug #3144 I also don't see any "definitely lost" bytes anymore when replaying the tests given here. Setting or not setting cache=false doesn't matter.
I din't check slow down or growth of resident set size when doing many of these queries.

Comment 17716

Date: 2012-08-31 14:45:34 +0200
From: @grobian

(In reply to comment 5)

After my fixes for bug #3144 I also don't see any "definitely lost" bytes
anymore when replaying the tests given here. Setting or not setting
cache=false doesn't matter.
I din't check slow down or growth of resident set size when doing many of these
queries.

@op: is the problem still visible to you after these fixes?

Comment 17717

Date: 2012-08-31 14:53:41 +0200
From: info

(In reply to comment 6)

(In reply to comment 5)

After my fixes for bug #3144 I also don't see any "definitely lost" bytes
anymore when replaying the tests given here. Setting or not setting
cache=false doesn't matter.
I din't check slow down or growth of resident set size when doing many of these
queries.

@op: is the problem still visible to you after these fixes?

Yes, the slowdown and growth are still visible. For now we have made the following change to sql_mvc.c:

Function mvc_create:
m->cache = 1;
to
m->cache = 0;

Function mvc_create:
if (m->cache != 1)
stack_set_number(m, "cache", 1);
m->cache = 1;
to
if (m->cache != 0)
stack_set_number(m, "cache", 0);
m->cache = 0;

This was done to have 'set cache=false' as the default. Not sure if these changes are completely correct, but it does stop the growth and slowdown for us.

Comment 18045

Date: 2012-11-27 12:53:29 +0100
From: @njnes

maybe we should make the size of the cache tunable.

Comment 18106

Date: 2012-11-27 15:14:05 +0100
From: @yzchang

Not possible to add test for this bug

Comment 18218

Date: 2012-11-28 21:02:38 +0100
From: @njnes

leaks are fixed and where unrelated to the query cache.
The sql env variable 'cache' can now be set to the maximum allowed cached queries.
Default is 20000.

Comment 18358

Date: 2013-01-22 09:29:03 +0100
From: @sjoerdmullender

Oct2012-SP3 has been released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant