Split run identity from configuration

[frazane]

Summary

Separates environment identity from run configuration to allow inference environments to be reused across configuration changes, eliminating unnecessary rebuilds of venv and squashfs images. Closes #111

Changes

• Add ENV_FIELDS and HASH_EXCLUDE ClassVars to RunConfig documenting the identity contract
• Split hashing logic: env_entry_hash() for environment-level changes, run_specific_hash() for configuration changes
• Refactor register_run() to compute both env_id and run_id with nested directory structure: data/runs/{env_id}/{config_hash}/
• Update inference rules to use {env_id} wildcard for environment artifacts (in data/runs/{env_id}/) and {run_id} for run outputs
• Add ENV_CONFIGS global dict and collect_all_envs() function
• Add comprehensive unit tests for identity separation

Benefits

• Reuses environments across config changes (no squashfs rebuild)
• Reduces disk I/O burden on shared filesystems
• Clear separation of concerns: environment identity vs. run configuration
• Nested directory structure aligns with the proposed design in issue

Testing

• All existing tests pass
• 5 new tests verify identity separation behavior

[dnerini]

This is looking really good! Thanks!

One question I have: do I understand correctly that now, although we distinguish between model envs and actual model runs, we stack both together under data/runs? Wouldn't it perhaps make more sense to use something like data/envs/{env_id} and data/runs/{run_id} instead?

[frazane]

One question I have: do I understand correctly that now, although we distinguish between model envs and actual model runs, we stack both together under data/runs? Wouldn't it perhaps make more sense to use something like data/envs/{env_id} and data/runs/{run_id} instead?

Used to be exactly this, but I changed it in c689c99. I like that by organizing hierarchically we see immediately which models use which envs. What advantage do you see in separating them?

[dnerini]

Used to be exactly this, but I changed it in c689c99. I like that by organizing hierarchically we see immediately which models use which envs. What advantage do you see in separating them?

mmmh maybe I don't fully understand it, can you paste here an example of how it'd look like?

[Louis-Frey]

All the configs, with the exception of forecasters-co1e, run without error. forecasters-co1e failing is ok, it failed before and I will still fix it. Good to go from my side!

[frazane]

@dnerini

./output/data/runs
└── forecaster-b30a-d2c9
    ├── 3180
    │   ├── 202501010000
    │   ├── 202502010600
    │   └── 202503011200
    ├── 8772
    │   ├── 202501010000
    │   ├── 202502010600
    │   └── 202503011200
    ├── anemoi.json
    ├── inference-last.ckpt
    ├── requirements.txt
    └── venv.squashfs

[Louis-Frey]

Looks good from my side, good to go!