Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data store: add absolute graph edges #6103

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

oliver-sanders
Copy link
Member

Closes #5845

Include absolute graph edges into the data store.

At present, absolute graph parents apply only when the parent is in the same cycle as the child.

E.G. Taking this example:

R1 = build
P1 = build[^] => run

The build => run graph edge would only exist in the R1 cycle (the one in which the build task was scheduled to run) rather than all subsequent cycles to which it applied.

The cause of the bug was a bit of code that was purposefully filtering out absolute graph edges which point to tasks in cycles other than the one that the child task is in. The code contained this comment which suggests doing this resolved some other issue:

# If 'foo[^] => bar' only spawn off of '^'.

Taking this code out fixes #5845, but I'm worried that this code served a purpose and that by deleting it I'm solving one problem by introducing another.

The comment first appeared here:

416240f#diff-e67a65e5cc6c5da815c97f19bf1afc269d56a7f032351aa72364a414ae726048R84

I can't track down why it was added by scanning through #3811. Long shot, @dwsutherland you don't have any memory of this (3 years ago!)?

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@oliver-sanders oliver-sanders added this to the 8.x milestone May 14, 2024
@oliver-sanders oliver-sanders changed the title data store: add aboluste graph edges data store: add absolute graph edges May 14, 2024
@oliver-sanders oliver-sanders self-assigned this May 14, 2024
@dwsutherland
Copy link
Member

I might have shifted some code about also.. I guess this is separate to the issue of spawning off ^ ? (where you'd spawn out to the RH limit if this dependency was in more than just R1)

@dwsutherland
Copy link
Member

dwsutherland commented May 21, 2024

Actually yeah, so I've only included it in the same cycle (like you said)... it makes sense WRT limiting the edges to ^ cycle, but it does feel artificial ..

Copy link
Member

@dwsutherland dwsutherland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine, tests passing.. Probably desirable for NIWA site also..

I suspect it may have tripped up the old graph walk in some way? hard to tell now.

@oliver-sanders oliver-sanders modified the milestones: 8.x, 8.4.0 May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tdef: graph parents - absolute dependencies ignored
2 participants