Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firebird Handle Leak Windows [CORE5702] #5968

Closed
firebird-issue-importer opened this issue Jan 8, 2018 · 78 comments
Closed

Firebird Handle Leak Windows [CORE5702] #5968

firebird-issue-importer opened this issue Jan 8, 2018 · 78 comments

Comments

@firebird-issue-importer
Copy link

firebird-issue-importer commented Jan 8, 2018

Submitted by: Mayko Sartor (maykosartor)

Attachments:
handle_leak_events.png
handle_leak.png
2018_04_02__FB3_states__32912_and_32939.png
FB_2018_04_29__08_27.png
FB_2018_05_03__07_42.png
plot_handles.png
FB_2018_05_08__08_05.png
plot_handles2.png
FB_2018_05_13__22_29.png
FB_2018_05_16__07_31.png
plot_handles3.png
FB_2018_05_17__07_00.png
plot_handles4.png

The firebird running on windows has a leak of handles. Even closing all connections to the database the process firebird.exe does not release the allocated handles. Verified by the proccess explorer it is possible to see that the accumulated handles are of type Event, as shown in the image.

Commits: e5ba540 d5fa522 ad0580e 0cc3d04 0e96bbf 64b6de0 07a9515 4fa1bf4

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 8, 2018

Modified by: Mayko Sartor (maykosartor)

Attachment: handle_leak_events.png [ 13191 ]

Attachment: handle_leak.png [ 13192 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 8, 2018

Commented by: @hvlad

Could you provide reproducible example ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 8, 2018

Commented by: Sean Leyne (seanleyne)

Are you running SuperServer or SuperClassic mode?

How are the connections being closed?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: Mayko Sartor (maykosartor)

I'm running Superserver. My software use the UniDac componentes to connect to firebird with connection pooling enabled. We set the component to release the connections every 4 minutes. So every open connection execute many querys for 4 minutes and then component closes it.
I tried to reproduce the error through a sample application but I could not. The error does not follow a pattern, simply the handles will accumulate after the days until reaching the limit. I can provide access to a virtual machine on windows azure where the process is running with many accumulated handles.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: Mayko Sartor (maykosartor)

I forgot to say that in superclassic mode the error also occurs.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: @hvlad

Could you create full memory dump of process (firebird.exe) and make it available for download ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: Mayko Sartor (maykosartor)

Link dump:

https://drive.google.com/open?id=1NSOguJyl66bl3Ienlxa7RBDxSzJSd9Au

Generated with Sysinternals ProcDump

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: @hvlad

This dump contains no useful information.
Is it really *full memory* dump ? It is too small, just 2MB.
Does you set -ma parameter for procdump ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 9, 2018

Commented by: Mayko Sartor (maykosartor)

Link with -ma option:

https://drive.google.com/open?id=1bFDohuu8oVRJ7Zs9wEUqXAfz4l3Urnnh

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 10, 2018

Commented by: @hvlad

Thanks, last dump is good for analyze.
Anyway, i see no reason for events leakage there.
Afraid, without test case it is close to impossible to find a reason for it.

Or, if you familiar with WinDbg - it could show stack trace when event(s) was created, see

https://blogs.technet.microsoft.com/yongrhee/2011/12/19/how-to-troubleshoot-a-handle-leak/

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Jan 28, 2018

Commented by: @hvlad

Is there any news on this issue ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 7, 2018

Commented by: Christian Geldmann (geldo79)

No news?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 7, 2018

Commented by: @hvlad

I have no news

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 8, 2018

Commented by: Christian Geldmann (geldo79)

Do you know possible reasons for increasing (event) handles in firebird.exe? I had the same problem when I upgraded from FB1.5.6 to FB3.0.1 (without changing anything in my application). Everything works fine except the increasing handles.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 8, 2018

Commented by: @hvlad

Show me the issue and i will investigate and explain it or fix it.
So far there is no sign of a real problem.

Do you use SuperServer ?
What is page cache size ?
How many handles you see ?
How long Firebird instance is running ?
Does handles count increased all the time or stop at some number ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 8, 2018

Commented by: Christian Geldmann (geldo79)

Do you use SuperServer ? => Yes, but also tried SuperClassic

What is page cache size ? => Page Size 4096 / Page Buffers 2048

How many handles you see ? => depending on how long the system was running. Max. was > 80.000

How long Firebird instance is running ? => ca. 4 Month

Does handles count increased all the time or stop at some number ? => increasing all the time

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 13, 2018

Commented by: @hvlad

Please, check next snapshot build of v3.0.4

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 13, 2018

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 16, 2018

Commented by: Christian Geldmann (geldo79)

From first observations the handles still increase.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 16, 2018

Commented by: @hvlad

Number of handles - much more than number of pages in page cache ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 16, 2018

Commented by: Christian Geldmann (geldo79)

I was too quick with my answer......it seems to be stable now!

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 17, 2018

Commented by: @hvlad

Sounds good :) Lets wait one-two days more to finally confirm fix.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: Christian Geldmann (geldo79)

Well ok, after three days, i see that the handles did increase, but much slower than it did before. Reduced by ca. factor 10.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: Sean Leyne (seanleyne)

Are you saying that the number of handles is still increasing but at a slower rate? Or, the number is stable?

What is the current number?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: Christian Geldmann (geldo79)

Still increasing at slower rate. Startet last friday with 400 handles, now 900.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: @hvlad

Every page cache buffer could consume event handle.
So, so far all is under control, i'd say.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: Sean Leyne (seanleyne)

I could understand a handle for the page cache as a whole, but each page having it's own handle seems odd.

Is there a formula that can be used to determine if the number of handles is too large? (i.e. expected handles <= Cache page + active DB connections + ???)

Otherwise, how would anyone know if they had a problem or not?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 19, 2018

Commented by: @hvlad

> I could understand a handle for the page cache as a whole, but each page having it's own handle seems odd.

Handle *could* be allocated when there is concurrency at this page buffer.

> Is there a formula that can be used to determine if the number of handles is too large? (i.e. expected handles <= Cache page + active DB connections + ???)

No

> Otherwise, how would anyone know if they had a problem or not?

80000 handles for one database with 2048 pages - definitely not good.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 20, 2018

Commented by: Christian Geldmann (geldo79)

By the way, is there a possibility to find out if the cache is completely used (e.g. using a page buffer of 2048, and 2048 pages are currently in cache) ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Mar 20, 2018

Commented by: @hvlad

There is no such possibility, sorry

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 15, 2018

Commented by: @hvlad

Next snapshot after 4.0.0.978, i.e. tomorrow

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 16, 2018

Commented by: @ibprovider

Load test of 3.0.4.32972 (SS, x64, Win10) at system with 128GB

After first iteration - 229 descritions (expected 220)
After second iteration - 230 descriptions

Just for info.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 16, 2018

Modified by: @ibprovider

Attachment: FB_2018_05_13__22_29.png [ 13249 ]

Attachment: FB_2018_05_16__07_31.png [ 13250 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: Christian Geldmann (geldo79)

See next plot. Much better now, but there still seem to be an increase of handles.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Modified by: Christian Geldmann (geldo79)

Attachment: plot_handles3.png [ 13251 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: @hvlad

Is there a moment when all attachments gone at the last graph ?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: Christian Geldmann (geldo79)

I don't know if I understand......what do you mean with "attachments gone"?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: @hvlad

I mean - no attachments.
I.e. moment, when there is no attachments and Firebird able to release resources such as page cache etc

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: Christian Geldmann (geldo79)

Hmm...no, I don't think so.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: @ibprovider

231 descriptors (+1 to previous point)

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Modified by: @ibprovider

Attachment: FB_2018_05_17__07_00.png [ 13252 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: @hvlad

Christian,

do you expect that Firebird could allocate all required handles\memory at start and never allocate something else ?
I already wrote here that every page cache buffer could consume event handle in some condition.
With page cache of 2048 pages we could expect at least 2300 handles. Could be more but not too much.
Most of them should be released on full disconnect.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 17, 2018

Commented by: Christian Geldmann (geldo79)

Yes I know. I'll continue the test. If it stops at < 3000 handles, everything's ok.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 18, 2018

Commented by: @ibprovider

FB 3.0.4.32672, SS, x64, Win10.

I see leak of 1 (one) descriptor after each execution of load test.

Happy debugging :)

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 23, 2018

Commented by: Christian Geldmann (geldo79)

For me, at the moment, it seems to be ok (see new plot).
@vlad: Newest Snapshots of Firebird 3 also contain that bugfix?
And will there be an official release soon containing the bugfix?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 23, 2018

Modified by: Christian Geldmann (geldo79)

Attachment: plot_handles4.png [ 13255 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 23, 2018

Commented by: @sim1984

Christian Geldmann,
Try last snapshot Firebird-3.0.4.32977
To release an official release, you need to test it.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 23, 2018

Commented by: @hvlad

Christian,

thanks for testing and for confirmation.
All current and newest snapshots contains this fix, sure.
Can't say when 3.0.4 will be released, sorry.
Usually it happens 2 times a year (for actively supported version).

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented May 23, 2018

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 Beta 1 [ 10750 ]

Fix Version: 3.0.4 [ 10863 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 18, 2018

Commented by: Christian Geldmann (geldo79)

Hi, are there any news regarding the new firebird version (3.0.4) release date?

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Sep 18, 2018

Commented by: @dyemanov

It's being prepared these days, so you may expect the release quite soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants