cli: new setup-environment and run-example commands

[tiborsimko]

• Adds two new convenient helper commands to set up the shell environment and to
  run selected REANA examples with selected workflow engines. Useful for local
  integration testing purposes.

Signed-off-by: Tibor Simko tibor.simko@cern.ch

[tiborsimko]

Note: depends on #72.

[diegodelemos]

WORKFLOW_ENGINES_LIST_ALL? Or WORKFLOW_SPECS_LIST_ALL?

[diegodelemos]

select_workflow_engines?

[diegodelemos]

Ignoring unknown workflow engine?

[diegodelemos]

As in the other places, I would expect the workflow variable to be workflow_engine or workflow_engine_name so the workflow name is composed of ci.{workflow_name|example_name}.{workflow_engine_name}

[tiborsimko]

I chose workflow for the sake of CLI simplicity:

$ reana run-example --workflow cwl

and kept the same name consistently in variable names everywhere.

It is not the most precise indeed, but I am not too sure about the alternatives either:

• workflow_spec is not correct, because the argument value does not mean workflow "specification" per se;
• workflow_engine looks better, but seems perhaps too long for a CLI option;
• ditto for other composite names such as workflow_system or workflow_type;

Not easy to find a suitable single-word option for CLI name... pipeline would come to mind 😉

Two options:

1. Keep shorter workflow and have nice CLI and live with imprecision.
2. Keep longer workflow_engine and have nice precision and live with uglier CLI.

Quick preference poll? @diegodelemos @dinosk

P.S. We may just as well use the same terminology here as in the rest of the REANA systems (thinking of models, schemas, etc.). Note here that later on we may perhaps have both cwltool and rabix as two different supported implementations of the CWL standard. If we happen to offer both, then we may need to differentiate between "specification/type" and an "implementation/engine" in various places...

[dinosk]

+1 for workflow_type / --workflow-type

[tiborsimko]

If we go for type, then we may need to add an engine option later on:

--workflow-type cwl --workflow-engine rabix
--workflow-type cwl --workflow-engine cwltool

Why not to go simply for one option only?

--workflow-engine cwl

which is compatible with r-w-e-cwl names and which would allow to keep a single option later:

--workflow-engine rabix
--workflow-engine cwltool

[diegodelemos]

I think that, as long as the relationship between type and engine is 1 to 1, I would use only type because it could also happen that without adding a new implementation we replace a current one, let say cwltool for rabix, and in such a case we would have to change engine from cwltool to rabix, whereas in the case of type the value would still be cwl.

Once we have several implementations, then it makes sense to specify the concrete one by using engine.

[dinosk]

We could use workflow_engine, it's just that neither type nor engine are completely precise for options [cwl, rabix, cwltool, yadage, serial].
Workflow engines are terms close to reana, clearly referring to options [cwl, yadage, serial] whereas rabix / cwltool I see are referred to as tools.
If it's not extensively long I would go for --workflow-engine cwl --tool rabix

[diegodelemos]

I think the naming is tight to what we tell in the docs, in my eyes:

• type is closer to specification, but for end user we don't use specification, type is more user friendly. It can have many implementations.
• engine is a concrete "REANified" workflow implementation for a given specification.

Thats why I mean that meanwhile we have only one implementation per type we are good by using type only.

P.S. Change the names type for spec or something else, but I think the concept remains the same.

[tiborsimko]

We don't use "type" terminology virtually anywhere in the code base yet... so I think it's simpler not to introduce it now but rather stick to "engine", which is both (1) REANA system-speak friendly (=so that developers will see direct connection to r-w-e-foo components) and (2) typical user-speak friendly (=as in "which workflow engine do you use?" which is more common than "workflow type" that would be better said as "workflow system type" actually). Anyway we seem to prefer longer names, so I'll make the change...

[tiborsimko]

OK, modified.

Note that I have also changed ci.some-example.cwl workflow run names to some-example.cwl only, since it seems better to start the namespace with the given example first... simulates better what people will use. If we want .ci we can rather add it later in the namespace chain. (but it seems even simpler to remove it, which is what I did)