Cylc commands, packages and versions

[oliver-sanders]

We will soon have a Conda "meta" package for installing Cylc, this presents some new challenges we will need to address pre-Cylc8 release:

1. We want to be able to install only certain parts of Cylc Flow on job hosts:
   • Message functionality (for tcp messaging).
   • Flow sub-commands which interact with running workflows (e.g. for broadcast).
   • Some other stuff I've not thought of.
2. We have multiple Python packages which provide Cylc commands:
   • Cylc Flow (e.g. cylc run).
   • The minimal Flow installation for job hosts (e.g. cylc message).
   • Cylc UIS (e.g. cylc hub)
   • Future Cylc Plugins (e.g. cylc report-timings)
3. cylc --version is incompatible with the "meta" package:
   • The cylc command belongs to Cylc Flow so cylc --version is really cylc flow --version
   • This becomes a problem with the "meta" package, where cylc --version should be the "meta" package version.
4. Sites will likely need multiple Cylc installations in multiple environments:
   • Users need an easy way to work out which version they are using.
   • In Cylc7 with the site script:
     • Users change versions like this export CYLC_VERSION=7.8.4...
     • They can check the version by doing this cylc version.

How do we resolve this issues? What solutions are available?

[hjoliver]

On point 1.

We want to be able to install only certain parts of Cylc Flow on job hosts

Having got rid of the UI, the docs, the test battery, and other miscellaneous cruft, this is not the same problem that it used to be.

Further, all workflow-interaction commands have to be available on job hosts. What's left after that (given the previous sentence) may not be worth worrying about too much?

[oliver-sanders]

After API-on-the-fly we might get to the point where all workflow-interaction commands are served via the same script which would make this easier to package if full Cylc installation were an issue.

[oliver-sanders]

With the release of 8.0b3 we can now close this 🎉. Breakdown of changes:

We want to be able to install only certain parts of Cylc Flow on job hosts:

• We shelved the idea of the message-only cylc-flow client for the moment.
• We separated the project into cylc-flow, cylc-uiserver and cylc-rose to allow more fine-grained installations.
• We added outputs for even more fine grained installation in meta packages #130.

We have multiple Python packages which provide Cylc commands:

Now unified behind a single command provided by cylc-flow.

If there is a requirement to have the Cylc CLI installed but not cylc-flow itself (see use cases above) then we can created a cylc.common package or something like that.

cylc --version is incompatible with the "meta" package:

Resolved by #130.

• The meta package is dead, we now use the cylc-flow version.
• Cylc components are pinned to the minor version of the matching cylc-flow.
• cylc version --long reports all installed Cylc components.

Sites will likely need multiple Cylc installations in multiple environments:

Resolved by cylc/cylc-flow#4367