Some doc updates for 8.3.0.dev changes.

[hjoliver]

A quick blast through the docs (mostly the glossary) whilst trying not to overlap too much with the optional-outputs-extension, which should be documented separately (mostly by pasting from the proposal).

(I did add a glossary item on output completion, needed for cross-referencing).

note on "incomplete"

We agreed not to define an "incomplete task" as a finished task with incomplete outputs (because that made "incomplete" different to "not complete" ).

Now, "completion" is tied entirely to "output completion". That means it is legit to say that a task is incomplete if its outputs are incomplete, whether or not it has finished. That's convenient and I've used it a couple of times here. E.g. it means we can say a stall occurs if there is nothing left to run but *there are any incomplete tasks in the n=0 window" - that covers both finished-with-incomplete-outputs and waiting-with-partially-satisfied-prerequisites. Both of those are "active tasks" (i.e., in n=0) and have incomplete outputs.

Requirements check-list

• I have read CONTRIBUTING.md and added my name as a Code Contributor.

[oliver-sanders]

task is incomplete if its outputs are incomplete, whether or not it has finished

As per cylc/cylc-admin#191 (and related discussion), there are a couple of issues with using "finished" in this way:

• It's not technically correct as "finished" infers that the task has started which incorrectly excludes "submit-failed" and "expired".
• It conflicts with the ":finished" qualifier which is correctly defined as succeeded or failed.

Suggest continuing with "task with final status" or simply "final task" for this purpose.

---

a stall occurs if there is nothing left to run but there are any incomplete tasks in the n=0 window

Unfortunately, there problems with this statement:

• Now, "completion" is tied entirely to "output completion", the phrase "incomplete tasks" includes active tasks. So this definition infers that a workflow can stall whilst tasks are still queued, preparing, submitted or running which is incorrect.
• Tasks with unsatisfied prerequites may actually have complete outputs (it is possible for a task not to have any required outputs at all) so "incomplete task" does not cover "task with partially satisfied prerequisites".
• Tasks waiting on xtriggers or extriggers may have incomplete outputs, but should not stall a workflow.

So "incomplete tasks" are not sufficient to define workflow stall or completion without also involving final task status.

(There are also a couple of small omissions, due to held tasks and the paused workflow status).

---

Now, "completion" is entirely to "output completion". That means it is legit to say that a task is incomplete if its outputs are incomplete

We are ok with brining back the phrase "incomplete task" in this manner (there's no issue with the term or definition), however, the term isn't especially useful as we usually simply refer to them as "waiting tasks". I'm not sure we would find uses for this new term other than in combination with "final task status".

The much more important term is "incomplete task with a final status" for which you've used, but not defined "finished incomplete task" (which has the issues outlined above). We still prefer the original definition of "incomplete task" (including status), but failing that (ground already conceded), could we have "final incomplete task" as a defined term? We need to refer to this a lot, we also need a task event for it (cylc/cylc-flow#4957), so could really do with a defined name or phrase.

[hjoliver]

task is incomplete if its outputs are incomplete, whether or not it has finished

there are a couple of issues with using "finished" in this way: ...
... Suggest continuing with "task with final status" or simply "final task" for this purpose.

I meant "finished" in the sense that the task has finished its automatic evolution within the workflow. Nothing else is gonna happen, short of manual intervention. That covers submit-failed and expired too.

However, I agree the semantic conflict with the finished pseudo-output is a problem.

"final task" sounds like the last task in the graph, so that won't fly.

So I guess we'll have to go with "final status".

[hjoliver]

a stall occurs if there is nothing left to run but there are any incomplete tasks in the n=0 window

Unfortunately, there problems with this statement:

Yeah, sorry, I didn't really intend that statement in the description, as opposed to the docs, to exhaust all the possibilities. It was implicit (in my mind at least :-) that nothing is currently running either.

However good point on tasks with no required outputs.

[hjoliver]

We are ok with brining back the phrase "incomplete task" in this manner (there's no issue with the term or definition), however, the term isn't especially useful as we usually simply refer to them as "waiting tasks".

No, it's not just waiting tasks - also submitted and running ones that haven't completed their outputs yet.

[hjoliver]

We still prefer the original definition of "incomplete task" (including status), but failing that (ground already conceded), could we have "final incomplete task" as a defined term?

We've covered that ground, as you say. I do think that "incomplete task" as we originally used it (final status with incomplete outputs) was nice in isolation, but in the wider context it didn't make any sense that a task wasn't incomplete until it achieved final status - so what on earth was its "completion status" prior to becoming incomplete? (I'm not even going to say "in my opinion" on that one, because it is a self-evident truth!).

could we have "final incomplete task" as a defined term? We need to refer to this a lot, we also need a task event for it (cylc/cylc-flow#4957), so could really do with a defined name or phrase.

Yes, it would be good to have a concise term for the concept "incomplete task with final status". It's a pity that we can't use "finished" as I meant it above, because of the pseudo output, as "finished incomplete" is pretty intuitive and concise.

I don't like "final incomplete task" much because it sounds like "the last incomplete task".

Perhaps we could invent a new term, divorced from all the baggage that comes with the words "final" and "complete"?

(I'll fix up the text in this PR to use the full definition for now).

[hjoliver]

@oliver-sanders - I've amended the text to fix up "finished" and "final status" as discussed above. The build fail is a link-check.

[oliver-sanders]

[from the OP]

E.g. it means we can say a stall occurs if there is nothing left to run but *there are any incomplete tasks in the n=0 window"

As noted, unfortunately, this is not true. Suggest not re-adding this term (not especially useful).

It might make it easier to define "final incomplete" though as this is a top-level concept that users need to be aware of.

[hjoliver]

Hopefully good to go now @oliver-sanders

Builds with cylc/cylc-flow#5809

[hjoliver]

The final small commit tweaks the docs for workflow-state changes cylc/cylc-flow#5809

[MetRonnie]

This is causing

Warning, treated as error:
~/cylc-doc/src/user-guide/writing-workflows/external-triggers.rst:349:py:mod reference target not found: cylc.flow.xtriggers.xrandom

[hjoliver]

huh, I added that to get rid of duplicate warnings ... removed again.

[hjoliver]

which has indeed returned:

/home/oliverh/cylc/cylc-doc/venv/lib/python3.8/site-packages/cylc/flow/xtriggers/xrandom.py:docstring
     of cylc.flow.xtriggers.xrandom:1: 
WARNING: duplicate object description of cylc.flow.xtriggers.xrandom, 
    other instance in plugins/xtriggers/built-in/cylc.flow.xtriggers.xrandom, use :noindex: for one of them

[MetRonnie]

I don't get that when building

[oliver-sanders]

Different Sphinx versions?

[hjoliver]

Probably. So let's just ignore the warning in my env for now.
(Note if I add :noindex: my warning goes away, but then @MetRonnie gets the one above that I don't get!)

[MetRonnie]

I can't see any duplicate autodoc of cylc.flow.xtriggers.xrandom

[oliver-sanders]

A duplicate index will cause a Sphinx error (which is what the no-index fiag is for).

[oliver-sanders]

(just minor stuff)

[hjoliver]

The one build problem is fixed on cylc/cylc-flow#5809

 <unknown>:1556: WARNING: term not in glossary: 'incomplete'

[oliver-sanders]

Just the one (expected) build error: https://github.com/cylc/cylc-doc/actions/runs/9497881655/job/26175443458?pr=727#step:10:320