Add Multiprocess Maybe #10
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
Wason1797
commented
Oct 23, 2022
Wason1797
commented
- QueueAutomator now includes a new more abstracted interface.
* I decided to start using mypy to typecheck the library
* Move the contents of main.py to automator.py so the naming was clearer * Also move the example outside the src folder of the library as it is not library code * Add support to insert data to a queue other than the initial one * Fix bug in the validate_non_empty_args helper * add a reset() api to clear the QueueAutomator state
* This multiprocessing helper uses the Maybe pattern as inspiration to chain several function calls using multiprocessing * It wraps a QueueAutomator instance and provides a different style API which could be cleaner than having a bunch of decorators and manually defining queue names. * This is a work in progress, and it is improving with every iteration
* Showcase the MultiprocessingMaybe capabilities in an example -> insert to intermediate queues -> replace None or values matching a predicate with the default value -> Prevent crashes for None values not executing the function when the input matches a predicate
* Include the first round of tests for the queue automator, this will ensure that the main functionality does not break if changed
* This is to exclude build and dist directories from typechecking
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.