Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deadlock in multi-threaded environment [JDBC435] #476

Closed
firebird-issue-importer opened this issue May 13, 2016 · 18 comments
Closed

Deadlock in multi-threaded environment [JDBC435] #476

firebird-issue-importer opened this issue May 13, 2016 · 18 comments

Comments

@firebird-issue-importer

Submitted by: @reevespaul

Is duplicated by JDBC434

Attachments:
deadlock.jpeg

When closing multiple threaded connections the driver hangs. The problem seems to be related to the test for mc == null in isClosed()

I tried a while ago with 2.2.3 and most recently with the code in the v3.0.0 snapshot. Neither work.

The last version that works for me is 2.1.6.

Commits: 78d882c a085ae0 7cd423e 4d7f118

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @reevespaul

Hopefully this screen shot from netbeans will give a better idea of what is happening. If necessary I can provide the app and the database.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Modified by: @reevespaul

Attachment: deadlock.jpeg [ 12963 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Modified by: @mrotteveel

Link: This issue is duplicated by JDBC434 [ JDBC434 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @mrotteveel

I will investigate this. Having access to the app and database would be very helpful.

As a side note, although JDBC asks that drivers are threadsafe, in general you shouldn't use the same connection from multiple threads simultaneously.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @mrotteveel

The problem is not with the null check in isClosed. It is with the fact that the order that executeBatch first acquires a lock on the statement, and later on on the connection, while connection close first acquires a lock on the connection, and then (when closing statements in response to the connection close) on the statement.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @reevespaul

Just to comment on your side note - no, in the application each thread creates its own connection. With 2.16 I've had up to 100 concurrent threads working without problem.

But good to hear that you may have identified the problem. I can test the fix easily enough when it is ready.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @mrotteveel

The screenshot seems to indicate that the connection and statements are shared: Thread 4 and Thread 6 in the screenshot are deadlocked on the monitors of the same connection and statement.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @reevespaul

But where does that monitor come from?
It is certainly not something in the application code, as far as I am aware. I grepped 'monitor' and found nothing.

Is it something in the JVM? Jaybird?

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @mrotteveel

Monitor is the technical term for what you lock when code does synchronized(object) { .... } (or use a synchronized method) It is an integral part of the object being locked on (so it is a fundamental part of Java).

In this case, it seems to be one and the same connection used by more than one thread.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 13, 2016

Commented by: @reevespaul

Which poses an interesting question. Is the synchronization problem in my code or is it something in Jaybird?

I just checked with Jaybird 2.1.6 and there is no deadlock. On the other hand I can reproduce this consistently with Jaybird 3.0.0 snapshot running in a debug session in netbeans.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented May 25, 2016

Commented by: @mrotteveel

Paul, could you send me the database + the code (or a download link to it)?

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Modified by: @mrotteveel

Fix Version: Jaybird 3.0 [ 10440 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Commented by: @mrotteveel

I decided to make the locking a little bit more coarse, with a single lock object for all related objects (eg the statement lock object is the same as the connection lock object, etc). I'm first doing this in the http://org.firebirdsql.gds.ng.* objects, and will check if I will also need to do something similar with the org.firebirdsql.jdbc.* objects. As connections etc should not really be used from multiple threads at the same time, having these coarser locks should not impact performance for most use cases. It might have an impact on use case where a single connections is shared by multiple threads.

It would still be very helpful to have the code to test with.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Commented by: @mrotteveel

In Jaybird 3, I applied a coarser synchronization with - where possible - a single lock object per connection.

This should prevent the potential of deadlock as described above, which may occur because in some situations the locks are obtained in order connection->statement, and in others statement->connection (etc).

I've not yet decided if I'm going to backport this to Jaybird 2.2 or not.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Modified by: @mrotteveel

Fix Version: Jaybird 2.2.11 [ 10751 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Commented by: @mrotteveel

Similar change made for 2.2.11.

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Jun 19, 2016

Modified by: @mrotteveel

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-issue-importer
Copy link
Author

@firebird-issue-importer firebird-issue-importer commented Aug 20, 2016

Modified by: @mrotteveel

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants