feat: PortalTaskArchive polls and caches instrument capabilities (§A.4)

[thusser]

Summary

PortalTaskArchive.get_instrument_capabilities() now actually returns live data instead of inheriting TaskArchive's None default — the last piece of specs/plans/2026-09-01-instrument-capability-duration-estimates.md's §A.

• New _poll_instrument_capabilities(), called from the existing _poll() alongside the tasks/projects check: re-downloads GET /api/instruments/ only when pyobs-portal#144's last_instrument_update marker moves, using the same marker-gated pattern _poll()/_update() already use for tasks/projects (pyobs-portal#84).
• Kept deliberately independent of the tasks/projects poll: a failure fetching or parsing instrument capabilities is caught and throttled locally, keeps serving the last-good InstrumentCapabilities rather than clearing it, and never blocks or forces a retry of the tasks/projects side.

Resolves both open review notes from #864/#867

• extra="forbid" conflict: a portal payload the pyobs-core models don't recognize now raises inside _poll_instrument_capabilities()'s own try/except, degrading like any other fetch failure rather than propagating.
• Pagination: reuses the existing http_request_paginated helper already used for tasks/projects, so no truncation risk on a fleet with >100 instruments.

Status of the plan

§A is now fully landed end-to-end in pyobs-core (#864, #865, #867, this PR). Real capability data only actually reaches the scheduler once pyobs-portal#144 (merged) and this PR are both deployed together with a PortalTaskArchive-backed scheduler module — and pyobs-portal's own pyobs-core dependency needs bumping to a release containing #864-867 first (see #144's review).

Test plan

• 8 new tests: first-poll download, marker-unchanged skip, marker-moved re-download, last-good-cache-kept on marker-fetch failure / download failure / unparseable payload, get_instrument_capabilities() default, _poll() actually calls the new method
• pytest tests/robotic/storage/portal/ — 59/59 passing (51 existing + 8 new)
• pytest tests/ (excluding integration/xmpp) — 1895 passed, no regressions
• ruff check / black --check / pyrefly check — clean

🤖 Generated with Claude Code

https://claude.ai/code/session_019XoNCe7YyJELc8xup1zVdE

[thusser]

Review: approve — clean, faithful §A.4 implementation

This genuinely resolves both review notes raised on #864/#867 (the extra="forbid" hard-fail risk and the >100-instrument pagination truncation), with regression tests for each. No blockers.

Verified

• Both #864/#867 review notes are resolved in code + tests. The unparseable-payload regression test ([{"some_unrecognized_future_field": 1}]) proves an extra="forbid" rejection degrades inside the poll's own try/except — last-good cache kept, marker not advanced — rather than propagating. Pagination uses http_request_paginated(..., strict=True), the same all-pages + no-silent-truncation helper as tasks/projects.
• Failure-handling details are correct. except Exception doesn't swallow CancelledError (BaseException); the throttle clear("instrument_capabilities") fires on any successful marker check (even when no download is needed), so a recovered streak resets; the marker advances only after a successful parse, so a mid-way failure retries next poll; _instrument_capabilities is never cleared on failure — the last-good object keeps being served.
• Consistent with the existing pattern. _poll_instrument_capabilities mirrors _poll's marker-gating and _check_for_changes' LogThrottle key/clear idiom; naming (_instrument_capabilities/_instrument_capabilities_marker) mirrors _tasks/_last_marker; the get_instrument_capabilities() override matches the base's InstrumentCapabilities | None signature (pyrefly confirms).
• Contract matches the merged portal side. The URL /api/instruments/last_instrument_update/ and the Time(res["last_instrument_update"]) epoch-safe parse match pyobs-portal#144 as merged (confirmed MERGED — the status claim in the PR body is accurate).
• Tooling/tests: 59/59 portal-archive tests pass (51 existing + 8 new, all well-targeted: first-poll download, unchanged-marker skip, moved-marker re-download, three keep-last-good failure modes, and the _poll() wiring test); ruff/black/pyrefly clean (0 errors). Full suite: 1888 passed here with only the 7 known environmental failures (sunpy config dir + /opt/pyobs read-only in this sandbox), consistent with the claimed 1895 in a healthy environment.

Notes (minor, non-blocking)

1. Strictly-greater marker comparison misses backward marker moves. If the admin deletes the row holding the current max(updated_at), the marker decreases and this poll never notices — the cached capabilities keep the removed device until some later edit pushes the marker forward. This exactly mirrors the pre-existing tasks-marker semantics (and the impact is estimate-only data that self-heals on the next edit), so it's fine — but != (with the existing None first-poll check) would also catch deletions at zero cost if you want instruments to be deletion-aware where tasks aren't.
2. Poll ordering coupling (asymmetric). _poll_instrument_capabilities() runs after the tasks block, so a tasks-side exception skips the instrument attempt for that cycle. In practice both fail together (portal down), and the docstring's promise — instrument failures never block/retry the tasks side — holds. Just noting the reverse direction isn't also isolated.
3. Persistent unparseable payload = full re-download every 5s. When the marker has advanced but parsing keeps failing (e.g. the portal deployed a new serializer field before this side catches up), each _poll cycle re-downloads the entire instrument list and fails again — same cadence and no backoff beyond http_request_with_retries' internal retries and the log throttle. Matches the tasks-side pattern; acceptable, but a short failure backoff would be a nice future hardening.

Summary

This completes §A end-to-end (models → plumbing → leaf scripts → portal fetch), the degradation semantics are exactly what the plan's "optional/degrade-to-None" convention calls for, and the two open review notes from the earlier PRs in this series are closed with genuine regression tests. Merge as-is.