Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FR] connection pooling #50

Closed
lantos1618 opened this issue Nov 13, 2019 · 5 comments · Fixed by #177
Closed

[FR] connection pooling #50

lantos1618 opened this issue Nov 13, 2019 · 5 comments · Fixed by #177
Labels
enhancement New feature or request imporant

Comments

@lantos1618
Copy link

I saw that there is this project that has conn pooling
https://nimble.directory/pkg/pg
also that JohnAD has plans to pull mongodb and conn pooling.
Have we got plans for conn pooling for postgress/sqlite?
@moigagoo
Copy link
Owner

No plans so far. Feel free to contribute.

@PhilippMDoerner
Copy link
Collaborator

Just saw this issue.
I wrote myself a connection pooling package for exactly this:
https://github.com/PhilippMDoerner/TinyPool

Could it make sense to use the package in norm?
Or possibly just copy paste some of the code as starting point for norm?

@moigagoo
Copy link
Owner

It would be awesome to support it in Norm. I see the package is compatible with Nim stdlib's DbConn type, so it shouldn't be too hard to use.

@moigagoo moigagoo added enhancement New feature or request imporant labels Aug 15, 2022
@PhilippMDoerner
Copy link
Collaborator

PhilippMDoerner commented Aug 15, 2022 •
edited
Loading

In that case my first approach would likely be copy pasting the code into norm, modifying it to better integrate with norm's logging and swapping out the std/db_* libs with ndb/sqlite and ndb/postgres.
Sound good?

PhilippMDoerner added a commit to PhilippMDoerner/norm that referenced this issue Sep 15, 2022
@PhilippMDoerner
moigagoo#50 ADD first draft for connection pooling code
6ac34f1 
The general code and logic is implemented in genericPool.
It provides instructions how to handle a pool of connections.
The pool itself is just a glorified seq of pre-determined size with a lock for safe multithreaded access.
It fills with connections upon initialization and closes them upon destruction.

Should connections run out during runtime, it goes into "burst-mode" for a limited time.
Burst mode refills the pool and allows it to have essentially infinite size.
Only after its timer runs out does it go back to "normal" mode, after which every connection that would make the pool larger than its intended size gets closed.
The timer gets extended every time it is noticed that more connections are still needed.
@PhilippMDoerner PhilippMDoerner mentioned this issue Sep 15, 2022
Closed
PhilippMDoerner added a commit to PhilippMDoerner/norm that referenced this issue Sep 16, 2022
@PhilippMDoerner
moigagoo#50 adjusted log and exception messages from tinypool to norm
569ec6f
PhilippMDoerner added a commit to PhilippMDoerner/norm that referenced this issue Sep 18, 2022
@PhilippMDoerner
moigagoo#50 REFACTOR dropDb to be in postgres/sqlite modules
de7a570 
The proc is about database manipulation, not connection pooling and thus does not belong in the module.
@moigagoo moigagoo linked a pull request Oct 15, 2022 that will close this issue
Connection pool #177
Merged
@lantos1618
Copy link
Author

nice one moigagoo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request imporant
Projects
Norm
Status: In progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Connection pool moigagoo/norm
3 participants
@moigagoo @lantos1618 @PhilippMDoerner