Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could this lib I made be added to the Awesome Libraries page? #2963

Open
obnoxiousish opened this issue Feb 26, 2024 · 3 comments
Open

Could this lib I made be added to the Awesome Libraries page? #2963

obnoxiousish opened this issue Feb 26, 2024 · 3 comments

Comments

@obnoxiousish
Copy link

https://pypi.org/project/trio-mongodb/
github.com/obnoxiousish/trio-mongodb

It uses multiprocess+trio.to_sync to process pymongo queries
Or just trio.to_sync in the main process alternatively

Couldn't find any other nosql dbms option using trio, there is only *SQL or redis options, would be better if it was true async but would take 5ever to implement mongodb from scratch

@CoolCat467
Copy link
Contributor

Pypi Project homepage link points to 404

@arthur-tacca
Copy link
Contributor

By to_sync I guess you mean to_thread.run_sync()

Using threads only can potentially result in a massive slow down for your main process

Above is from your readme. Have you tested this? It could well be true but, then again, it might not be: Python releases the GIL during I/O so I would normally expect something I/O heavy like database client to scale very well across threads. Then again, if pymongo does a lot of processing to the data in pure Python (i.e., if it is poorly written) then it could be true. I'm just saying I wouldn't make a statement like that without some testing first.

A couple of other suggestions:

  • Make the client an async context manager so that client will definitely be closed.
  • For the threaded client, allow passing in a trio capacitylimiter, so that users can choose how many threads to use and also when some client connections share threads (and could block each other) and when not (by passing different capacity limiter objects).

@CoolCat467
Copy link
Contributor

CoolCat467 commented Mar 8, 2024

I had similar thoughts with adding support for async context managers when I took a look at the project the other day, and I was also questioning if just moving everything to another thread would be helpful. Having some sort of comparison data would be very helpful. If there were a small script for generating comparison data, I would be happy to assist with your data collection!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants