Streamline test suite with shared fixtures, markers, and per-notebook tests

[animmosmith]

Summary

Fixes #403. Folds in #237.

Built on top of #394 (assumed merged) — this branch includes its pyopia/tests/conftest.py and test_cli.py.

• Markers: registers slow and training pytest markers in pyproject.toml. pytest -m "not slow" gives a fast local loop; CI now runs with -m "not training" so model-training notebooks never run in routine CI.
• Shared fixtures: consolidates real-data downloads (example image, classifier model, training database, hologram) into session-scoped fixtures in conftest.py, shared across test_pipeline.py, test_classify.py, and test_cli.py instead of each downloading its own copy. The CLI-specific fixture copies into its own private directory before duplicating a file for its chunking-minimum workaround, so that duplicate can't leak into the fixtures other test files share.
• tmp_path everywhere: replaces manual tempfile.TemporaryDirectory() with pytest's built-in tmp_path/tmp_path_factory throughout.
• De-flaked test_classify.py: removes a hardcoded wall-clock timing assertion (we hit this exact flake earlier bumping keras) and a stray model/ directory it left in the repo root.
• test_notebooks.py rewrite: each notebook is now its own parametrized test (test_notebook[<name>.ipynb]) instead of one monolithic function, with markers reflecting real cost:
  • slow: real network/pipeline notebooks — existing ones plus new docs/notebooks coverage for Add docs/notebooks to test_notebooks #237 (montaging, stats, exploring_pipeline_data, pipeline_step_by_step, background_correction)
  • unmarked: cli.ipynb (no network dependency) and markdown-only notebooks (toml_config, processing_raw_data, big_datasets)
  • slow + training: the DINOv2 classifier training notebook (30 real training epochs, no CI-mode shortcut) — now excluded from routine CI entirely
  • Not included: docs/notebooks/STATSnc.ipynb loads a pre-existing test-STATS.nc file that no notebook produces at that path in isolation; it's designed to be read against a user's own prior processing run, not executed standalone.
• Two real bugs found and fixed by this new notebook coverage: docs/notebooks/config.toml referenced a non-existent keras_model.h5, and pipeline_step_by_step.ipynb referenced a stale imc key and relied on default segment_source/roi_source values that no longer match the current ImagePrep/Segment/CalculateStats implementation.
• README: adds a "Testing" section documenting the markers, shared fixtures, and notebook CI policy.

Test plan

• pytest -m "not slow" — 34 passed in ~10s
• pytest -m slow on test_pipeline.py/test_classify.py/test_cli.py — 10 passed in ~2 min, confirming shared fixtures work correctly across files
• All non-training notebook tests (13) — pass individually and together
• Confirmed -m training selects exactly the DINOv2 notebook, and -m "not training" (now CI's default) excludes only that one

🤖 Generated with Claude Code

[animmosmith]

Added `.github/workflows/scheduled-training-check.yml` to this PR.

What it does: runs the tests tagged `@pytest.mark.training` (currently just the DINOv2 classifier training notebook) on a weekly schedule (Mondays 06:00 UTC), plus on-demand via the "Run workflow" button (`workflow_dispatch`). These tests are intentionally excluded from the regular per-PR `build-and-test.yml` run, since they involve a real, uncapped multi-epoch training loop rather than a check of PyOPIA's own correctness — running them on every PR would be slow and mostly irrelevant to the change being reviewed.

Why schedule it at all rather than just excluding it: without this, the only way we'd find out that notebook broke (e.g. an upstream `torch`/`timm` release changing the DINOv2 backbone's behavior) is a user stumbling into it and reporting back. We actually found a live example of exactly that risk while building this PR — `pipeline_step_by_step.ipynb` had a stale key reference that had silently gone unnoticed for roughly two years, since nothing ever re-ran it. A weekly check catches that kind of drift automatically instead.

One thing to note: GitHub only evaluates the `schedule` cron trigger from the copy of the workflow file on the default branch (`main`) — so the weekly run won't actually start firing until this PR merges. Until then it can still be triggered manually from the Actions tab.

[emlynjdavies]

Given that DINOv2 was originally released under CC BY-NC 4.0 (non-commercial only) before being relicensed to Apache 2.0, I think we should be cautious about assuming the licences to pre-trained weights are stable. This workflow would quietly keep pulling and using new weights (potentially with a new licence) every Monday without anyone verifying.

Maybe we could consider if we can lock the weights version that gets downloaded by the DINOv2 notebook in the standard tests (but that would still remove the need for a scheduled test, with the benefit of more defined control over exactly which weights are used), and we would not capture the 'drift' effect from updated weights without it being an explicit PR where we update to newer weights. Perhaps that's anyway more robust: if it is possible to lock the weights version in that notebook downloads?

@nepstad any thoughts?

[animmosmith]

Good point on the licensing risk. I've prepared the pin as a ready branch: 404-pin-dinov2-weights-version (403-streamline-test-suite-with-shared-fixtures...404-pin-dinov2-weights-version) - pins timm to 1.0.28 in the notebook's install cell, with a comment explaining why, so bumping it becomes a deliberate, reviewed change rather than an implicit weekly re-download. Verified the pinned version installs cleanly and loads the real backbone.

Haven't touched the scheduled workflow itself yet, since that decision (keep it for catching our own code going stale vs. drop it entirely) seems like it should follow from what you and @nepstad land on here - happy to fold this in once there's agreement, or push it as-is if that's easier to review directly.