Skip to content

Replace untyped probe pipeline internals: #86

Description

@henry0816191

Problem

The probe/source pipeline leaks several untyped or mutable internal shapes that callers can take dependencies on:

  • DataSource.diff() returns Any. protocols.py does define a @runtime_checkable DataSource Protocol whose diff is declared def diff(self, previous: Any, current: Any) -> Any (src/paperscout/protocols.py lines 34–36). The concrete WG21Index.diff() (src/paperscout/sources.py lines 239–247) has no return annotation at all — it defers to monitor._diff_paper_maps() (which does return a typed DiffResult), so the real value is typed, but the public method/protocol surface is Any. (ISOProber.diff() at lines 377–389 and OpenStdSource.diff() at lines 825–833 are already typed to list[ProbeHit] / list[OpenStdEntry].) Note: an earlier draft asserted protocols.py/DataSource do not exist in this clone — that is incorrect; both exist and the Any-typed diff is real.
  • Bare positional 6-tuple probe entry. _Entry = tuple[str, Tier, str, int, int, str] (src/paperscout/sources.py lines 317–319) is unpacked positionally inside run_cyclefor url, tier, prefix, num, rev, ext in urls (line 432) — and indexed positionally (u[1] at lines 409–410). It is produced across _build_probe_list (516–524), _build_hot_list (562–595, appends at 585/593), and _build_cold_slice (597–635, appends at 623/633).
  • ProbeHit is a non-frozen mutable dataclass. @dataclass(slots=True) (not frozen=True) at src/paperscout/models.py lines 169–184, so every field (is_recent, last_modified, front_text, …) is silently mutable after construction.
  • PollResult performs no validation. PollResult is a plain class (not a dataclass) whose __init__ accepts list/dict arguments and stores them as-is with no narrowing/validation (src/paperscout/monitor.py lines 107–120), unlike the adjacent typed DiffResult (lines 40–46) and CycleResult (models.py 203–224, which validates in __post_init__).

Acceptance Criteria

  • DataSource.diff (protocol) and WG21Index.diff carry a concrete return type (e.g. DiffResult) instead of Any/no annotation, so mypy sees a real type at the source boundary.
  • _Entry is replaced by a named, typed structure (e.g. a @dataclass(frozen=True, slots=True) ProbeTarget or a NamedTuple) so probe-list fields are accessed by name; run_cycle, _build_probe_list, _build_hot_list, _build_cold_slice, and _probe_one are updated to use it.
  • ProbeHit is made immutable (@dataclass(frozen=True, slots=True)) or otherwise prevented from post-construction mutation, with all construction sites in sources.py still passing.
  • PollResult.__init__ validates/normalizes its inputs (or becomes a typed dataclass) rather than storing arbitrary raw list/dict values for dp_transitions/per_user_matches.
  • mypy/type checks pass with no new Any-typed public return in the probe/source pipeline.
  • Tests pass in CI (if applicable)
  • PR approved by at least 1 reviewer

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions