Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature.skip mode #6039

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Feature.skip mode #6039

wants to merge 6 commits into from

Conversation

wxtim
Copy link
Member

@wxtim wxtim commented Mar 27, 2024

Closes #5641

Skip Mode Proposal Doc

This branch includes (marked against skip mode proposal):

  1. A new task run mode called "skip" to sit alongside the existing "live" and "simulation" and "dummy" modes.
  2. Skip mode settings in `[runtime][<namespace>][skip].
  3. Task-level control of the run mode - allowing run mode of tasks to override that of the workflow using [runtime][<namespace>]run mode.
    a. Broadcast can change run mode for future task job submissions.
    b. Cylc Validate and lint will warn about the setting not being live.
  4. Not Done
  5. Tests to ensure that run mode = skip respects is_held flag.
  6. Tests to ensure that force triggering a task will not override the run mode.
  7. Run Mode is available as an task attribute in the UI

There are two extensions, which I haven't dealt with yet, because I want to ensure that the basic functionality works, and move to the substantial documentation PR which need follow this.

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • Feature: PR against master

@wxtim wxtim marked this pull request as draft March 27, 2024 12:57
@wxtim wxtim force-pushed the feature.skip_mode branch 2 times, most recently from b43d296 to d26c315 Compare March 28, 2024 09:11
@oliver-sanders oliver-sanders added this to the cylc-8.x milestone Apr 2, 2024
@wxtim wxtim self-assigned this Apr 22, 2024
@wxtim wxtim requested review from oliver-sanders, markgrahamdawson and MetRonnie and removed request for markgrahamdawson April 23, 2024 10:00
@wxtim wxtim marked this pull request as ready for review April 23, 2024 14:26
@MetRonnie MetRonnie modified the milestones: 8.x, 8.4.0 Apr 23, 2024
cylc/flow/cfgspec/workflow.py Outdated Show resolved Hide resolved
cylc/flow/cfgspec/workflow.py Show resolved Hide resolved
cylc/flow/cfgspec/workflow.py Outdated Show resolved Hide resolved
cylc/flow/cfgspec/workflow.py Outdated Show resolved Hide resolved
@MetRonnie MetRonnie added the config change Involves a change to global or workflow config label Apr 23, 2024
@wxtim wxtim requested a review from MetRonnie April 24, 2024 15:10
@wxtim wxtim force-pushed the feature.skip_mode branch 4 times, most recently from 3d002ee to dd0a9fc Compare April 25, 2024 12:56
cylc/flow/prerequisite.py Outdated Show resolved Hide resolved
cylc/flow/scheduler_cli.py Outdated Show resolved Hide resolved
cylc/flow/task_state.py Outdated Show resolved Hide resolved
cylc/flow/task_state.py Outdated Show resolved Hide resolved
cylc/flow/task_state.py Outdated Show resolved Hide resolved
cylc/flow/scheduler.py Outdated Show resolved Hide resolved
cylc/flow/scheduler.py Outdated Show resolved Hide resolved
cylc/flow/task_pool.py Outdated Show resolved Hide resolved
@wxtim wxtim force-pushed the feature.skip_mode branch 3 times, most recently from 0d42eab to c37ef72 Compare May 8, 2024 12:43
@wxtim wxtim requested a review from MetRonnie May 8, 2024 13:47
@wxtim wxtim force-pushed the feature.skip_mode branch 2 times, most recently from 3839a8a to d7ee653 Compare May 9, 2024 08:50
cylc/flow/network/schema.py Outdated Show resolved Hide resolved
cylc/flow/task_state.py Outdated Show resolved Hide resolved
Comment on lines +354 to +355
# blacklist Run Modes:
not_equals(*RunMode.MODES),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this just be RunMode.SKIP?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not sure that it's a good idea to have any of the RunMode strings as custom outputs. But you are right, it might concievably break someone's existing practice.

cylc/flow/task_proxy.py Outdated Show resolved Hide resolved
cylc/flow/task_pool.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/skip.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/skip.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/nonlive.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/nonlive.py Outdated Show resolved Hide resolved
cylc/flow/cfgspec/workflow.py Show resolved Hide resolved
@wxtim wxtim requested a review from MetRonnie May 23, 2024 15:28
cylc/flow/network/schema.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/nonlive.py Outdated Show resolved Hide resolved
cylc/flow/run_modes/nonlive.py Outdated Show resolved Hide resolved
cylc/flow/data_messages.proto Outdated Show resolved Hide resolved
cylc/flow/run_modes/nonlive.py Outdated Show resolved Hide resolved
wxtim added 2 commits May 29, 2024 16:17
* Add `[runtime][<namespace>]run mode` and `[runtime][<namespace>][skip]`.
* Spin run mode functionality into separate modules.
* Run sim mode check with every main loop - we don't know if any tasks are
  in sim mode from the scheduler, but it doesn't cost much to check
  if none are.
* Implemented separate job "submission" pathway switching.
* Implemented skip mode, including output control logic.
* Add a linter and a validation check for tasks in nonlive modes,
  and for combinations of outputs
* Enabled setting outputs as if task ran in skip mode using
  `cylc set --out skip`.
* Testing for the above.
remove ghost mode terminology

added run_mode as a field to be used in the gui

Response to review: Remove unwanted args to skip mode
wxtim and others added 3 commits May 30, 2024 08:20
Co-authored-by: Ronnie Dutta <61982285+MetRonnie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
config change Involves a change to global or workflow config
Projects
None yet
Development

Successfully merging this pull request may close these issues.

skip mode
3 participants