Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

job-list: allow list-id to wait for job state #5213

Merged
merged 8 commits into from Jun 6, 2023
Merged

job-list: allow list-id to wait for job state #5213

merged 8 commits into from Jun 6, 2023

Commits on Jun 6, 2023

  1. testsuite: add missing python kvs test 

    Problem: In python/t0005-kvs.py an illegally stored json object
was stored as binary and tested.  A legally stored json object
stored as binary was not tested.

Add a legally stored json object in binary for coverage.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    66db8b7 View commit details
    Browse the repository at this point in the history

  2. testsuite: use common functions 

    Problem: In t2800-jobs-cmd.t some convenience functions are implemented
that are identical to some helper functions in helper files.

Solution: Use the helper functions provided in job-list-helper.sh and
remove the unnecessary function declarations.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    6336369 View commit details
    Browse the repository at this point in the history

  3. job-list: cleanup correct memory on error path 

    Problem: In the idsync_add_waiter() function, the cleanup path cleaned
up the wrong memory.

Solution: Clean up the zlistx_t that was possibly created within the function,
not the data structure passed in as a parameter.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    01f18eb View commit details
    Browse the repository at this point in the history

  4. job-list: store jobid in idsync list struct 

    Problem: In the idsync API a list is stored in a hash and
an externally stored jobid in a list entry is used as the key.
This currently is fine because the entire list is read back and
destroyed in idsync_check_waiting_id().

However, if we ever want to read/delete some list entries but not
others, the externally stored jobid makes the hash lookup unsafe,
as the storage of the key can no longer be guaranteed.

Solution: Store the list in a new struct that also stores the
jobid.  Also, adjust the name of a cleanup function as a result.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    c123d54 View commit details
    Browse the repository at this point in the history

  5. job-list: allow list-id to wait for job state 

    Problem: It'd be convenient if the job-list.list-id service not only
waited for a job id to be legal but to also waited for the job to
pass a certain job state.

Solution: Support an optional job state in the job-list.list-id service.
Return when a job passes this job state or reaches the inactive state.

Fixes flux-framework#2864
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    526a2d3 View commit details
    Browse the repository at this point in the history

  6. flux-job: add --wait-state to list-ids subcommand 

    Problem: The job-list.list-ids RPC now supports the ability to return
after a specific job state has passed.  However, the flux job list-ids
command does not support any way to use it.

Solution: Add a --wait-state option which allows the caller to tell
list-ids to only return after a specific job state has been hit.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    0d912ab View commit details
    Browse the repository at this point in the history

  7. testsuite: cover job-list.list-ids state option 

    Problem: There is no coverage of job-list.list-ids and its
ability to return only after a specific job state has been reached.

Add coverage in t2260-job-list.t.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    b658aa9 View commit details
    Browse the repository at this point in the history

  8. testsuite: update job-list consistency checks 

    Problem: The job-list module has eventual consistency for the jobs
that are submitted to the job-manager queue.  A number of helper
functions have been written to check for this consistency before
running tests.  These functions are no longer necessary due to the
new `flux job list-ids --wait-state` option.

Use the flux job list-ids --wait-state option when appropriate.  Remove
no longer necessary code.
    @chu11
    chu11 committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    75a64be View commit details
    Browse the repository at this point in the history