Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in fbclient [CORE6067] #6317

Closed
firebird-issue-importer opened this issue May 24, 2019 · 12 comments
Closed

Memory leak in fbclient [CORE6067] #6317

firebird-issue-importer opened this issue May 24, 2019 · 12 comments

Comments

@firebird-issue-importer

Submitted by: Tobias Zipfel (tobiasz)

Relate to DNET882

Attachments:
Program.cs
MemoryLeakFbClient.dll.mht

When executing read queries against 3.0.4.33054 as embedded server we experienced constantly growing values for private bytes.

We originally realized the growing memory consumption in a larger application when running actions resulting in many database reads.

For easier testing purposes we used a simple C#⁠ program (source in attachment) with the .Net driver (https://firebirdsql.org/en/net-provider/) version 6.6 to test.

- The read queries are run against the example EMPLOYEE database from the Firebird installation folder.
- We used the default firebird.conf and only change the server mode.
- connection, command and reader are used in "using" blocks to trigger disposing.

We tested the following conditions all having the same problem:

- latest nightly build (Firebird-3.0.5.33126-0_x64)
- .Net driver 6.5, 5.12
- SuperClassic, SuperServer
- reusing the same connection and commands objects
- calling FBconnection.ClearAllPools after each read

The memory leak disappeared for the following conditions:

- Run Firebird 3.0.4 as dedicated server. Both applications (test program and the Firebird server) have a constant amount of private bytes.
- Run Firebird 2.5 with driver version 6.6 as embedded server.

We used Microsofts Debug Diagnostics tool to create reports of the memory consumption (see attachment).
The report contains analysis of three memory dumps (.Net and native) taken at different points in time.

Here you can see that in the last snapshot the allocations in fblclient have grown a lot.
Maybe the call stack information in the last memory dump report can help somehow...
From these reports and results we got with pure .Net memory analyzers you can see that the .net heap is constant over time.

Do we miss something here?
Is this behavior configuration dependent? Or is there something else wrong about how we use the driver?

Is this maybe rather a driver problem?

We are glad to help if more information/testing is required.

Commits: 9766094 f8eb917

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 24, 2019

Modified by: Tobias Zipfel (tobiasz)

Attachment: Program.cs [ 13350 ]

Attachment: MemoryLeakFbClient.dll.mht [ 13351 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 24, 2019

Commented by: Sean Leyne (seanleyne)

Your test program is using an invalid programming approach, it is not using prepared statements.

So, it is creating a *new SQL statement* with each program loop, as such new memory would be allocated for the new statement.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 24, 2019

Commented by: @cincuranet

Sean, your statement is not entirely correct. The "op_prepare" happens implicitly in this case, because it was not called explicitly. Also at the end of the `using` block, the statement is released on server by sending "op_free".

I have to try to run the program myself to see what's what, but from the code POV, nothing looks wrong.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 25, 2019

Commented by: @hvlad

The problem is because provider first commits transaction and then closes cursors opened within it.
fbclient should be ready for such case but massive refactoring in v3 introduced memory leak of cursor object (YResultSet) in this scenario.
Looking for the fix in fbclient.
Suggest to fix provider code too.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 25, 2019

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 25, 2019

Commented by: @cincuranet

Vlad, by closing cursor you mean op_free_statement (if I'd talk network protocol)?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 25, 2019

Commented by: @hvlad

Jiri,

in this *embedded* case i speak about isc_dsql_free_statement.
It is called after isc_commit_transaction.

If speak about network protocol - yes, you correct: op_free_statement passed by client to server to close cursor and\or drop statement at server side.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 29, 2019

Commented by: @hvlad

Description changed to better reflect nature of issue.

Note, this memory leak happens when both conditions are true:
- application works with ISC API
- cursors (result set) is closed after transaction is committed (rolled back)

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 29, 2019

Modified by: @hvlad

Version: 4.0 Beta 1 [ 10750 ]

Version: 3.0.3 [ 10810 ]

Version: 4.0 Alpha 1 [ 10731 ]

Version: 3.0.2 [ 10785 ]

Version: 3.0.1 [ 10730 ]

Version: 3.0.0 [ 10740 ]

Version: 4.0 Initial [ 10621 ]

Fix Version: 3.0.5 [ 10885 ]

Fix Version: 4.0 Beta 2 [ 10888 ]

Component: API / Client Library [ 10040 ]

summary: Memory leak in embedded server => Memory leak in fbclient

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 29, 2019

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 30, 2019

Modified by: @cincuranet

Link: This issue relate to DNET882 [ DNET882 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jun 10, 2019

Commented by: Tobias Zipfel (tobiasz)

Tested the nightly build, no more memory leaks.

Thanks for the quick response and fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment