Refactor/figures - #202
Conversation
# Conflicts: # workflow/rules/publication.smk # workflow/scripts/publication_scoremaps.py
jonasbhend
left a comment
jonasbhend
left a comment
There was a problem hiding this comment.
Looking really good. Thanks for seeing this through. One question. The current workflow still overwrites results based on stations and analysis, as both the manifest and the output paths are truth-hash-free (or experiment-hash-free). I think we should change that, such that we can run station and analysis-based workflows, without overwriting?
|
@jonasbhend Good point! Do you mean the snakemake workflow only or also when runing the standalone plotting? And generally, is it enough to use the truth-hash, or do we need to hash the config? |
Well, we've done this before and it didn't work. But I would probably just use the truth label to denote the manifests. That way it is easy to operate in standalone mode and 'reasonably' fail-safe. If you trigger a full rerun, this will anyway be recomputed if the config changes and we probably don't want to keep old manifests lying around. |
|
@jonasbhend like this maybe? See last commit :) |
jonasbhend
left a comment
jonasbhend
left a comment
There was a problem hiding this comment.
Hi @clairemerker.
Das sieht super aus, vielen Dank. Ich habe es noch nicht ausprobiert, und komme heute erst nach dem weekly dazu. Von mir aus kannst du aber auch gerne schon in den paper-figures branch mergen, wenn es für dich funktioniert.
Reconcile the two publication config designs by adopting paper-figures' per-task `enabled` switches and `scoremaps.steps` (over a single `publication.enabled` and `scoremaps.leadtimes`). Rewire the config coherence validator, publication CLI, Snakemake rules, tests and docs to the new schema; regenerate config.schema.json.
2 participants
AI assisted implementation!
This PR is an attempt to improve the plotting workflow for publication to be able to generate them from the snakemake workflow as well as interactively.
A documentation is currently available in
docs/publication_figures.mdI only tested creating figures using both workflows, never triggered the whole pipeline incl. results creation... So not sure if the whole workflow is still ok, but I think it should...