-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API that expose the list of active Run UIDs to the client #110
Conversation
81bee9b
to
8e9f24e
Compare
9850dcc
to
0b37f66
Compare
Nice work! I tested this and it is working as expected. |
I am in the process of reviewing it now, please don't merge - I have a few suggestions. |
…presentation of results in case of failure
I think my next step will be to deal with randomly failing HTTP server tests, because it is getting really annoying. |
BTW, after the modifications I was able to successfully run the unit tests on Mac OS. So in one of the future PRs I will probably add OSX to CI, may be allowing it to fail. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did it! That's a very important and useful functionality, already relied on the webclient side. Here are my suggestions.
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
Co-authored-by: Maksim Rakitin <mrakitin@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for addressing almost all comments/suggestions, @dmgav! I spotted a couple of more places which can reuse the _generate_run_list_uid
helper function. For a few of them I made my suggestions, but some were in the code before. Want to address them here too?
$ git grep 'str(u'
bluesky_queueserver/manager/comms.py: msg["id"] = str(uuid.uuid4())
bluesky_queueserver/manager/manager.py: return str(uuid.uuid4())
bluesky_queueserver/manager/plan_queue_ops.py: return str(uuid.uuid4())
bluesky_queueserver/manager/tests/test_plan_monitoring.py: uids = [str(uuid.uuid4()) for _ in range(3)]
bluesky_queueserver/manager/tests/test_plan_monitoring.py: uid = str(uuid.uuid4())
bluesky_queueserver/tests/common.py: msg["id"] = str(uuid.uuid4())
Thank you, @dmgav. |
The API that expose the list of active Run UIDs were implemented. The list of active runs include all the runs that were opened by the currently running plan. The contents of the list may change as the plan is executed. The runs that are closed by the plan are marked as closed and remain in the list. API allow obtain the full list of active runs or subsets of currently open or already closed runs. The list is cleared once the plan execution is completed. The full list of Run UIDs for the completed plans could be found in the plan history.
In case of simple single-run plans, the list of Run UIDs will contain a single item. The client will probably never see this run (or the last run of a multi-run plan) reported as closed, since the list will be cleared once the plan execution is completed.
The items of the list are dictionaries with the following keys:
uid
- Run UID,is_open
- boolean value that indicates if the run is currently open (set toFalse
when the 'stop' document is received),exit_status
- run exit status extracted from the 'stop' document.To keep track of active Run UIDs, Run Engine is subscribed to a custom callback, which extracts UIDs from each emitted 'start' document and creates new items of the list. The callback also captures the emitted 'stop' documents, extracts 'run_start' UID and 'exit_status' values and modifies the respective list entry. Each time the list is modified, it is propagated from RE Worker to RE Manager process and becomes available to client via
re_runs
ZMQ API. To make tracking of the list more convenient, each state of the list is assigned a uid (same format as the other uids). The uid is passed asrun_list_uid
field of RE Manager status (status
request). A new list uid is generated each time the list is updated. The client application will keep track of the list uid and download the updated list whenever therun_list_uid
changes. Downloading the list only when it is changed is expected to minimize the number of requests to the server. Clearing the list at the end of the plan execution is also considered an update and a new value ofrun_list_uid
will be generated.New features:
re_state
status field ('status' request) - current state of the Run Engine. The state of Run Engine is related, but different from the state of RE Manager. The field may be used for monitoring of Run Engine and detection of transition states (e.g.pausing
orresuming
) or states that are not handled by RE Manager (such assuspended
).run_list_uid
status field (status
request) - UID generated for the current state of the list of active runs. New UID is generated each time the list is changed. Monitoring this field allows the client to know when the list was updated at the server and should be loaded.re_runs
ZMQ API - load the list of active runs. Optional parameteroption
may have valuesactive
(default),open
,closed
that allow to choose between the full list, or subsets of open or closed runs./re/runs/active
,/re/runs/open
,/re/runs/closed
HTTP API (GET requests).CLI support:
permissions_reload
ZMQ API - reloads the list of available plans and devices and permssions for user groups from the files on disk. Calling this function makes sense if the files were changed on disk, otherwise reloading will have no effect. The API was necessary to write the unit tests for this PR, but it may be also useful for debugging of profile collection. Support for HTTP and CLI will be implemented as a separate PR.The PR also includes additional minor tweaks to the code:
configuration of logging so that it works on OSX;
PipeJsonRpcReceive
class is now removing all messages from the pipe before starting the processing thread. Clearing the pipe from outdated messages makes code more stable. The change was made to get rid of error messages on OSX.Addresses the issue #111