Cyclic attempt to open new connection with Oracle.ManagedDataAccess.Core creates new threads

[WagnerPhillip]

Hello,

we have a code section, where we try to open a connection to oracle database cyclical. Once a connection was established, we use this one until the main application is closed an when it failes, we will try again to open a connection, until the database can be reached.

With Oracle.ManagedDataAccess.Core version 3.21.70 or lower this was not a problem.
But since version 3.21.80 everytime our code is trying to open a connection, there are new threads being created in the background that stay alive as long as the main application is running.
In one case we had it running for so long, that our application had over 1000 threads running!

Why does this happen? Are we doing something wrong or is this a bug in the newest versions?
Because as mentioned above it only happens in 3.21.80 or 3.21.90 version of the Oracle.ManagedDataAccess.Core package, the lower versions are only creating threads once when the connection cannot be established, but not every single time.

Attached you will find a simplified sample project, where hopefully you will be able to reproduce it.
Any information or help will be much appreciated :)

OracleDbConnectionThreadLeakSample.zip

[alexkeh]

@WagnerPhillip I tried ran your sample app with ODP.NET Core 21.9 and 21.7 and saw similar behavior.

Using Process Monitor, I saw about 23 thread creations and one thread exit before the thread exits started increasing in frequency. After that, the thread activity was fairly stable. What behavior are you seeing with your sample app?

In ODP.NET 21.8, there were some changes made to the connection pool to improve performance under high load conditions. One was an elimination of a race condition under high load. The second was using dedicated threads to get pools to Min Pool Size level faster. It's possible that one or both of these changes could lead to more threads created in a smaller time frame.

For the test case you provided though, it's somewhat doubtful either change would have a material effect. The test app has Min Pool Size of 1 and has a sleep that lasts half a second between each connection attempt.

[WagnerPhillip]

When i have this sample app running and the database is just not reachable all the time it keeps on creating threads frequently.
So here i have some screenshots, where my sample app is not able to connect to the database and you can see the threads at the beginning, after 5 minutes and after 10 minutes of failing to open a connection.
This was with ODP.Net Core 21.9 and as mentioned, the database i want to connect to is not reachable.

I have checked these numbers in Windows Resource Monitor.

[WagnerPhillip]

This screenshot is after about 1 hour of failing to open a connection to the oracle database:

[alexkeh]

I see. The test case needs to be run when DB connections cannot be opened. I tried running it again.

For the first 10 minutes, the threads hovered around 17 when the app could connect.

I shutdown the DB. After ten minutes, the thread count went to 27. At the 25 minute mark post-shutdown, the thread count was about the same at 26.

Can you turn on tracing and share that file? It will provide details on the thread creation and cleanup activity.

[WagnerPhillip]

Interesting, in my case it never stops to produce threads.
I tried it with wrong username, wrong password or a host, that doesn't exist. In every of this cases it was the same result, maybe it's just with these cases and not with yours, where you have shutdown the DB.

Here you find two trace files that my colleague and i created with JetBrains dotTrace. I hope you can open them on your side.
OracleThreadsTest - TraceFiles.zip

[alexkeh]

Sorry, I should have been clearer that I was asking for an ODP.NET trace. I don't have dotTrace myself and I'm not sure if it will have the trace info we need to debug. You can add the following code segment before you open your first ODP.NET connection to generate a trace. Just sent your TraceFileLocation to the directory where you want the trace written.

OracleConfiguration.TraceFileLocation = @"C:\traces";
OracleConfiguration.TraceLevel = 7;

[WagnerPhillip]

I'm sorry that was my mistake.
Thank you for the clarification and the lines of code you provided.

Now i have made a ODP.NET trace file, where my application was running for about 2 minutes, i hope that is enough. Otherwise i will be happy to make a new trace file.
ORACLETHREADSTEST.EXE_PID_21244_DATE_2023_02_17_TIME_07_46_56_058085.zip

[alexkeh]

Thanks @WagnerPhillip! The Oracle team will review.

[alexkeh]

I've filed a bug (35100428) to track this issue.

[daniel-liuzzi]

I've filed a bug (35100428) to track this issue.

3.21.100 release notes state bug 35100428 is fixed

https://nuget.info/packages/Oracle.ManagedDataAccess.Core/3.21.100

[WagnerPhillip]

@daniel-liuzzi thank you for drawing my attention to the release notes for 3.21.100

@alexkeh thank you for your help. When we find time for it, we will test our case again with the newest version to see if it works as expected.

[alexkeh]

@WagnerPhillip No problem!
@daniel-liuzzi Thanks for the reminder that we included this bug fix in 21.10.