Add AutoContext to automatically insert tasks in batches#105
Conversation
Intended to automatically insert asyncs in batches (asynchronously) as they are added to the context.
Move most of handle_tasks() into _handle_tasks() to allow AutoContext to insert some tasks before fully exiting. Also add a parameter to allow different batch_sizes.
Also ensure they are added correctly in batches.
When batch_size is None, AutoContext will act like Context, where all tasks are inserted at once when the context is exited. Ensujre that when no tasks are added, there are no errors.
|
@robertkluin-wf @beaulyddon-wf @tannermiller-wf @johnlockwood-wf @tylertreat-wf @mattheweverett-wf |
|
I think it looks good. It'd be nice to add a kwarg to the |
|
I don't wonder why we don't just build this into the |
|
The auto_add sounds good. |
|
I do think it might make sense to add the functionality to the core context. It shouldn't add that much code. What do you think @ericolson-wf? Also, I can't merge this. Looks like there might be a merge conflict? |
Conflicts: furious/context/context.py
Add AutoContext to automatically insert tasks in batches
ghost
merged commit
|
I merged this. But, the more I've thought about it, the more I think this might be just as well being in plain old Context. |
|
+1 |
4 participants
Automatically insert asyncs in batches as they are added to the context instead of waiting until the context is closed.
Also allows custom batch size.
Here is a PR that uses add_async() with AutoContext: ericolson-wf#4