Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shell: live output #2366

Merged
merged 8 commits into from Sep 21, 2019
Merged

shell: live output #2366

merged 8 commits into from Sep 21, 2019

Commits on Sep 20, 2019

  1. shell: Fix tabbing

    chu11 committed Sep 20, 2019
    Copy the full SHA
    0a5064b View commit details
    Browse the repository at this point in the history
  2. shell: Retype flux_shell_task_io_f

    For consistency to other shell code, make type flux_shell_task_io_f
    a pointer to a function, instead of a typedef to a function.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    4ebb578 View commit details
    Browse the repository at this point in the history
  3. t/t2500-job-attach: Add bad executable test

    Add missing test, in which flux-srun is based an invalid executable.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    8076c01 View commit details
    Browse the repository at this point in the history
  4. shell: Output data as it arrives

    Instead of saving all output data and writing it out when a job completes,
    write output to the KVS (or stdout/stderr in standalone mode) as output
    data arrives to the leader shell.
    
    In this change, every output that is received by the leader shell is
    appended to an output array before it is immediately written to the KVS (or
    stdout/sterr in standalone mode).  This output array is not explicitly
    needed but has been left in tact for future use in aggregating output.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    73eaabb View commit details
    Browse the repository at this point in the history
  5. shell: Emit output-ready event to exec.eventlog

    Alongside the output header, write the output-ready event to the
    exec.eventlog directory.  This event will indicate that the
    guest.output directory has been created.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    382fb95 View commit details
    Browse the repository at this point in the history
  6. shell: Add references to avoid racy completion

    Under racy scenarios, the shell could exit before several futures
    completed within the shell output.  Add completion references to
    ensure those futures complete within the shell output before
    allowing the shell to exit.
    
    As a side effect, the completion references ensure that output commits
    will complete before the shell exits.  Therefore, the pending_commits
    list is no longer needed.
    
    For consistency, rename the old completion reference "io-leader" to
    "output.write".
    
    Fixes flux-framework#2363
    chu11 committed Sep 20, 2019
    Copy the full SHA
    be3e009 View commit details
    Browse the repository at this point in the history
  7. flux-job: Update attach for live output

    Adjust attach for fact that output is now live and occurs as the
    job is happening.  Watch exec.eventlog to determine when the guest.output
    directory is ready for reading.  Once it is ready, watch the guest.output
    and output as data is available, and not after the job completes.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    6482c72 View commit details
    Browse the repository at this point in the history
  8. t/t2500-job-attach: Add output tests

    Add several new tests to ensure output behavior of job attach
    works.  Most notably that output can occur as job is running and
    won't occur only after a job finishes.
    chu11 committed Sep 20, 2019
    Copy the full SHA
    2b3a8cd View commit details
    Browse the repository at this point in the history