-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Data Race During --race Test #94
Comments
It seems this is something that should be fixed in upstream gocql/gocql as it seems that upstream is affected too. I'll be happy to merge the fix upstream. |
I've checked the code, but it seems gocql does not spawn goroutines using the policy before calling |
The issue is the assignment of the logger on line 401 during Init() can race with a use of the logger in the updateReplicas where it passes to getStrategy. The updateReplicas is getting called concurrently with the Init() because the schema event listener debouncer starts further up in the sequence, and if you do per-test schema operations, its possible to get into a state where one arrives during the race, because it's seeing stuff like the previous tests keyspace getting torn down.
In short, if a schema update arrives during startup, you can get this. I believe without the race detector, it'll probably nil-pointer as the logger wasn't assigned. |
But what leads to that? It seems that Init is called in NewSession: Line 161 in 6aca262
There are no goroutines that gocql spawns until this point. First connection is made in Session.init: Line 181 in 6aca262
The only explanation I have right now is that the policy instance is used with multiple sessions. Is that the case? Or why do we reach handleEvent? Could you please provide example code that reproduces the issue? |
Multiple sessions, but each only created after the previous has been .Closed()'d. However, I can't share the entire body of the application - but it seems like the thrashing during my unit tests (creating and deleting lots of keyspaces, as each test uses its own) makes this exponentially more likely. |
Reusing host selection policies between multiple sessions (even after closing the old session) is not supported as the old session might still be using the policy. Adding a lock to |
Sounds like it’s a more along the lines of a defective design pattern - and the strategies should be passed as a factory so that each session the driver summons can get its own.
However that boat has probably sailed in terms keeping things compatible with todays world. It’d be a fairly breaking change to fix.
Get Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Martin Sucha ***@***.***>
Sent: Wednesday, January 26, 2022 4:30:56 PM
To: scylladb/gocql ***@***.***>
Cc: Steve Gray ***@***.***>; Author ***@***.***>
Subject: Re: [scylladb/gocql] Data Race During --race Test (Issue #94)
Reusing host selection policies between multiple sessions (even after closing the old session) is not supported as the old session might still be using the policy. Adding a lock to Init won't fix the fact that the old session is still using the policy. I suggest you use a new instance of the policy every time you create a session. I guess we need to document that reusing policies is not supported and panic in tokenAwareHostPolicy.Init if the policy was already initialized.
—
Reply to this email directly, view it on GitHub<#94 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADPDTP46JG5ZHUB75FZGHHDUX6IKBANCNFSM5MOREHVQ>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Yes, I agree. Unfortunately the policy interface is like this for a long time and there could be external implementations, so we can't easily change it. |
Sharing a policy between sessions is not supported because the policy receives state updates from the session. Let's update the documentation and add a panic in TokenAwareHostPolicy constructor. It is better to panic early than to have races and undefined behavior later. See also discussion in scylladb#94
Sharing a policy between sessions is not supported because the policy receives state updates from the session. Let's update the documentation and add a panic in TokenAwareHostPolicy constructor. It is better to panic early than to have races and undefined behavior later. See also discussion in scylladb#94
Sharing a policy between sessions is not supported because the policy receives state updates from the session. Let's update the documentation and add a panic in TokenAwareHostPolicy constructor. It is better to panic early than to have races and undefined behavior later. See also discussion in scylladb#94
Sharing a policy between sessions is not supported because the policy receives state updates from the session. Let's update the documentation and add a panic in TokenAwareHostPolicy constructor. It is better to panic early than to have races and undefined behavior later. See also discussion in scylladb#94
Probably needs proper mutexing - this one periodically causes flaking in CI for me. The items hitting it are:
github.com/gocql/gocql.(*tokenAwareHostPolicy).Init() vs github.com/gocql/gocql. (*tokenAwareHostPolicy).updateReplicas()
The text was updated successfully, but these errors were encountered: