Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use asyncio.TaskGroup for downloading in parallel #128

Open
exitflynn opened this issue Jan 23, 2023 · 4 comments
Open

Use asyncio.TaskGroup for downloading in parallel #128

exitflynn opened this issue Jan 23, 2023 · 4 comments
Labels
enhancement A new feature or other enhancement to the library

Comments

@exitflynn
Copy link

Task Groups are a new feature introduced in python 3.11 which make it more convenient to create/work with parallel tasks.
This change would more or less just be a nice-to-have in the sense that it'd reduce some complexity when looking at the code and make it more easier to maintain/contribute.
However there's a requirement of a python version >=3.11, which I'm guessing won't be on most machines right now, so I'm not sure if it's something we can implement in the near-future (i'm unfamiliar with how these things are dealt with usually in projects). I figured I'd raise an issue addressing it.

@Cadair
Copy link
Owner

Cadair commented Jan 23, 2023

Oh that looks super useful, but it's going to be a very long time till we can be 3.11+. I wonder of there is a backport on pypi we could use for <3.11 ?

@exitflynn
Copy link
Author

there are! just gave it a look and there's aiotools and taskgroup.

@Cadair
Copy link
Owner

Cadair commented Jan 24, 2023

You got me going down a rabbit hole, this looks interesting as well: https://aiojobs.readthedocs.io/en/stable/intro.html What I can't decide at the moment is at what layer in the stack it would live.

All these things are great ideas for a complete internal rewrite I would like to do to simplify the internal logic, but I can't see that happening any time soon!

@exitflynn
Copy link
Author

i seee! i'll love to see that and help out in any way i can!

@Cadair Cadair added the enhancement A new feature or other enhancement to the library label Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or other enhancement to the library
Projects
None yet
Development

No branches or pull requests

2 participants