Skip to content

fix(deploy): reference collector memory watermarks too small for current catalog #573

Description

@proerror77

Change contract

Production and shadow reference-collector units must not be throttled by cgroup memory.high during cold-start backfill at the current catalog size.

Production evidence (2026-08-01)

Shadow gate invocation 329edad2 (candidate 8cedff1c) failed: 'Rust shadow memory.events high grew or a MemoryMax/OOM event occurred'. The 672MiB/768MiB pair was calibrated in July for a smaller catalog; current state is 6509 tracked markets + 316k retained trade ids. Same defect class as the usdm archiver throttle stall (#534).

Acceptance

  • Both units raised with measured justification in README calibration history
  • control-plane/stage tests green
  • Runtime: host drop-in already applied to the shadow template (documented in PR); production drop-in at cutover

Out of scope

  • Collector memory reduction work

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-humanHuman implementation or judgment is requiredruntimeRequires deployment, runtime mutation, or live evidence for closure

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions