0.2.0

Release notes for simrun 0.2.0

Workflows without a cluster: --local runs a workflow on this machine through a local simrun service, with the same planner, harness, records and web app as on the cluster, at the speed of running the code directly. The cluster gets faster split tasks and a client which needs no mount of the store.

Features

• The local mode. simrun submit fit.yaml --source model=. --local runs on this machine, the same command without --local on the cluster. A local backend runs the tasks of a run as processes (a split task on the cores of the machine, several runs sharing them, a command with its args directly or in its image with --docker), and the service of a project lives in .simrun/ next to the workflow: started on demand by any --local command, on 127.0.0.1 only, stopped after 30 idle minutes or with simrun serve --local --stop, and restarted when the python environment, the simrun version or the task types change. Sources are recorded (branch, commit, uncommitted files) without copying; --snapshot installs them into an environment of the run instead. A run of three tasks takes 4.8 s for 4.6 s of work. See Local mode.
• The web app locally. The service of a project serves the web app at /ui/, Executions and Editor (simrun edit --local); the wheel ships the built app.
• submit --wait and simrun outputs. submit --wait follows a run with a live table per task and exits like wait, the default with --local; simrun outputs RUN TASK [--to DIR] lists or downloads the outputs of a task through the API.
• Snapshots through the API. Without SIMRUN_STORE the client uploads the files of a working tree to the service, only those the store lacks, so a laptop on the VPN needs a URL and a token only. The snapshots and their hashes are those of 0.1.0.
• The coordinator merges. A split task has no merge pod any more: its coordinator merges once its ledger is complete and checks the ledger every 0.5 s instead of 5 s, which saves several seconds per split task; a coordinator which succeeded without merging (an image with the harness of 0.1.0) fails its task after a grace of two minutes.
• The web app. Why a task failed (the exit of its last pod, also in the message of the run), the phase instead of a dash (starting, merging), a worker whose unit or command failed shown as failed, one short name per server, the lines of the Argo executor hidden unless asked for and errors in red; in the Editor the planned memory per worker, a + handle to add an input of a command, the inputs of a command in its inspector, older closed sessions behind Show older. The documentation shows the views.

Fixes

• A run with the simrun repository as a source failed in every pod: the harness now restarts after installing the sources.
• A failure which is not a stop is no longer retried a hundred times: only a pod which was stopped (no exit code, 137, 143) is retried, and the message of the run names the failed pod and its exit.
• The args of a command get {input}, {out} and {settings} as a plugin command does; a command cannot stage its inputs.
• The image of an environment is named also by the hash of the simrun wheel, so a new wheel of the same version builds a new image.
• Argo requeues a workflow after 2 s instead of 10 s, so a run shows running soon after its pods start.

Upgrading

• Add lookupcache=positive to the mountOptions of the existing PersistentVolume of the store and restart the pods (deploy/cluster/store.yaml has it for new installs; see Cluster deployment).
• Runs recorded by 0.2.0 carry coordinator_merges and merge_pending, which a service of 0.1.0 cannot read: the service cannot be rolled back to 0.1.0 once it recorded runs.
• Runs of 0.1.0 still in flight keep their merge pod.