Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
python: misc fixes, refactors, and additions #2218
Main changes are fixing streaming RPCs, fixing
Minor changes include adding some methods to the Flux handle class, improve error messages, and extending the check-format/format scripts to be usable as git commit hooks.
If no arguments are provided, it will traverse the entire source tree looking for python files to format. If arguments are provided, only those files are considered for checking. The script will filter out any non-python files from the arguments list before passing them to `black`. Useful for git pre-commit scripts, which can produce a list of files that changed in a given commit. This results in a much lighterweight operation than traversing the entire source tree and opening every file.
Generally, we want to keep user's terminals free of stacktraces from Python (unless they have verbose logging on). By moving the common top-level exception handling to a decorator, hopefully all our CLI tools will avoid dumping stacktraces unnecessarily, have consistent output, and avoid duplicate code.
adds event_subscribe to improve auto-completion suggestions adds respond method to provide automatic conversion of the message and payload arguments as well as improve auto-completion suggestions
This all looks good to me, though I'm not confident in reviewing a lot of the python code.
I do think 3995f32 could use a commit body which describes the problem with streaming RPC then callbacks and how it is resolved by the commit (perhaps some comments in the implementation as well, though that could be due to my python inexperience)
It was a good idea to abstract and standardize some of the front-end command setup into a utility module. I wonder if that should eventually be wrapped up into an exported module for use in other flux framework projects? (not needed for now though)
If the same future has its callback executed (e.g., in the case of a streaming RPC), an error occurs. After the first execution of the callback, the bindings remove the python-side reference to the data handle. So by the time that the second call happens, the data handle has already been garbage collected. Add a reference count dictionary to keep futures with a pending `then` callback alive, even if there are no remaining references to the future in the user's program scope. When a callback is first set on the future, add an entry to the dictionary with the future itself as the key and a value of 1. Whenever a future's callback is run, decrement the count in the dictionary for that future, and whenever reset is called on a future, increment the counter. If the counter hits 0, delete the reference to the future from the dictionary.
Wrap the return of `submit_async` in a python Future object rather than return an opaque, c future pointer so that users can leverage all python Future methods. Add a `wait_for` to ensure that the future is fulfilled before getting the id. Otherwise, an error occurs if the submission future is not fulfilled when `submit_get_id` is called.
Yeah. That's a good point. I think we have that functionality now. All of the python bindings are exported/installed, so
@@ Coverage Diff @@ ## master #2218 +/- ## ========================================= + Coverage 80.69% 80.7% +<.01% ========================================= Files 202 202 Lines 32259 32259 ========================================= + Hits 26032 26034 +2 + Misses 6227 6225 -2