Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework pool lock logic to handle new connections context switches #109

Merged
merged 3 commits into from Sep 12, 2019

Conversation

bcardiff
Copy link
Member

Fixes #77

I was able to make a spec to repro the issue in crystal-lang/crystal-mysql@325493b#diff-f41a8330fda8a5ae668e7e17638aef40R40-R57

A couple of things I considered during this PR:

  • go's sql package added a SetConnMaxLifetime that would be good to have in order to limit the timespan of the whole connection. It's a good policy.
  • When releasing a connection and if the max idle connection was reached, we could choose where to drop one in the idle pool rather than the one that is been released in order to keep the younger one.
  • The checkout_some might be redundant. Maybe is just enough to reuse the statement and allow any connection to be used. If we do this, the idle connections could be kept in a Channel, as the message that is sent. But this will also require for the buffered channel to have a (theoretical) unlimited max capacity.

For now, this PR should fix the issue but there are some ideas to improve the logic.

@bcardiff bcardiff changed the title Rework pool lock logic to handle new connections context switched Rework pool lock logic to handle new connections context switches Aug 20, 2019
@Xosmond
Copy link

Xosmond commented Aug 20, 2019

@bcardiff Hey, great to know you worked on this PR, I will try it tomorrow to see it under pressure

src/db/pool.cr Outdated Show resolved Hide resolved
@bcardiff
Copy link
Member Author

@Xosmond were you able to check if this PR works for you?

@bcardiff bcardiff merged commit ff5c326 into master Sep 12, 2019
@straight-shoota straight-shoota deleted the pool-lock branch September 12, 2019 18:13
src/db/pool.cr Outdated Show resolved Hide resolved
@Xosmond
Copy link

Xosmond commented Oct 12, 2021

@bcardiff I'm back 2 years after that 😀 . I apologize. I remember I tested and I was still having the same problem on high concurrent environment (Unknown why it was happening, I would need to debug). Btw I will give it a shot again. I have to test it with mysql because postgres shard on high concurrent scenario fails for some reason (SSL error IRCC).

Its kinda tricky to test, you have to configure everything to work on a high concurrent scenario and get a group of servers to hit each other 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pool issues
5 participants