Docs audit — message recording — 2026-08-06 (mnfst/manifest PRs #2565–#2662)

[SebConejo]

Audited: topic message recording, window 30 days (mnfst/manifest PRs #2565, #2627, #2634, #2642, #2662 of Jul 29–30, plus historical context #2317, #1984, #1959, #2002), against the docs at commit 2b3cdaa. Docs pages audited: message-recording.mdx, observability.mdx, self-hosted.mdx, reference/telemetry.mdx, reference/environment-variables.mdx, and the five deploy pages.

Clean: no internal-leak found (no internal tool names in any audited page; env vars correctly scoped to self-hosted sections), retention table matches the code, per-attempt recording and "metadata stays, body goes" match, and the REQUEST_RECORDING_STORAGE=auto selection rule is documented correctly.

Findings

1 · [WRONG] The Settings section is called "Logs", not "Message recording"

• Doc: message-recording.mdx — "The toggle lives on each harness's Settings page, under Message recording."
• Platform: packages/frontend/src/pages/Settings.tsx:234, SettingsRecordingSection.tsx:35-63 — the section is titled "Logs", the switch is "Enable logs"; the UI never says "recording".
• Impact: a user looks for a "Message recording" section, finds nothing, and concludes the feature is missing from their plan or version.
• Options:
  1. (recommended) Keep the page name, correct the sentence to the real UI and bridge the terminology in one line.
  2. Rename the page and cross-links to "Logs" (large rename, "logs" collides with server logs).
  3. Ask product to rename the UI section (out of docs' hands).
• Suggested wording (option 1):

  The dashboard calls this feature **Logs**. The switch is **Enable logs**, in the **Logs** section of each harness's **Settings** page.

2 · [WRONG] "Older harnesses keep whatever they were set to" is false — the upgrade set them all to off

• Doc: message-recording.mdx — "New harnesses have it on. Older harnesses keep whatever they were set to, because only the default changed, so a harness from before the feature shipped probably needs the toggle flipped once." Echoed in reference/telemetry.mdx.
• Platform: migration 1801300000000-AddRequestRecordings.ts:8 (PR #2565) — every harness existing before the July upgrade was set to FALSE (the earlier column had been dropped by migration 1795000000000, so no prior setting survived). New-harness default TRUE comes from migration 1801500000000 (PR #2627).
• Impact: a user who had recording on before the July upgrade assumes it still runs; no bodies are stored since the upgrade and the lost period is unrecoverable.
• Options:
  1. (recommended) State the real rule plainly on both pages: created after the release = on; existed before = set to off, re-enable in Settings → Logs.
  2. Same, plus a dated "Upgraded in July 2026?" callout.
• Suggested wording (option 1), message-recording.mdx:

  Harnesses created after the July 2026 release record by default. The upgrade set every earlier harness to off, whatever it was set to before. Turn those back on once in **Settings → Logs**.

  reference/telemetry.mdx:

  New harnesses have it enabled; the July 2026 upgrade set every harness from before it to off, so re-enable those in the harness's **Settings → Logs**.

3 · [OBSOLETE] "Reading it back" describes a Messages tab "next to Attempts"; the drawer now has an Attempts sidebar and per-attempt tabs

• Doc: message-recording.mdx — "When recordings exist, the drawer has a Messages tab next to Attempts." Echo in observability.mdx ("a Messages tab next to Attempts").
• Platform: packages/frontend/src/components/RequestDrawer.tsx:240-257, 363-366 (PR #2565) — the drawer has an Attempts sidebar where you pick a provider attempt (initial call, fallbacks, and Auto-fix retries are separate attempts), then per-attempt tabs Details / Messages / Tools / Raw / Request headers / Model params, where Messages, Tools and Raw appear only when that attempt has a recording. The Messages tab's turn rail with search and role filters still exists (RequestMessages.tsx:232-263) and the doc's description of it stays accurate.
• Impact: the user looks for "Messages next to Attempts", misses that Messages is per attempt behind the Attempts sidebar, and never sees fallback or Auto-fix retry bodies.
• Options:
  1. (recommended) Rewrite the navigation part of the section against the current drawer (Attempts sidebar → per-attempt tabs), keep the accurate rail paragraph, fix the observability.mdx sentence, pair with the screenshot of finding 6.
  2. Conservative rewrite naming only the Attempts sidebar and the Messages tab, less detail, more future-proof.
• Suggested wording (option 1, replaces only the navigation sentences; the rail paragraph stays):

  Open a request in the dashboard's **Requests** log and pick a provider attempt in the drawer's **Attempts** sidebar. The initial call, each fallback, and an Auto-fix retry are separate attempts, and each one shows its own conversation.
  
  Every attempt has a **Details** tab; **Request headers** and **Model params** appear when the attempt carries them. **Messages**, **Tools**, and **Raw** appear only when that attempt has a recording.

4 · [MISSING-CASE] Upgraded self-hosted installs lose recordings without a one-time upgrade step

• Doc: self-hosted.mdx (Upgrading) — "Database migrations run automatically on boot, no manual steps." And message-recording.mdx (Docker tab) — "Nothing to configure."
• Platform: docker/install.sh:16-17, docker/docker-compose.yml:187, 231-232 (PR #2565) — pre-July compose files lack the manifest_request_recordings volume; without bash install.sh --upgrade (or a manual compose edit), recordings are written in the container's writable layer and destroyed on recreation.
• Impact: an upgraded self-hosted user follows the documented docker compose pull && up -d, enables logs, and silently loses every recording at the next container recreation. Same fresh-install-vs-upgrade gap as the Auto-fix case.
• Options:
  1. (recommended) Add the upgrade Note in self-hosted.mdx Upgrading (with a manual-compose Accordion) AND scope the "Nothing to configure" claim in message-recording.mdx.
  2. Fix only message-recording.mdx.
  3. Document only the manual compose diff.
• Suggested wording (option 1), self-hosted.mdx:

  <Note>
    **Installed before the July 2026 release?** Run the installer once with the upgrade flag, or recordings are written inside the container's writable layer and lost when the container is recreated:
  
    ```bash
    bash install.sh --upgrade
    ```
  
    It adds the `manifest_request_recordings` volume to your compose file. Later releases include it from the start.
  
    <Accordion title="Hand-managed compose file? Add the volume yourself">
      Declare the named volume and mount it on the backend service:
  
      ```yaml
      services:
        manifest:
          volumes:
            - manifest_request_recordings:/data/request-recordings
  
      volumes:
        manifest_request_recordings:
          name: manifest_request_recordings
      ```
    </Accordion>
  </Note>

  message-recording.mdx (Docker tab):

  Nothing to configure when your compose file includes that volume; an install from before the July 2026 release needs [one upgrade step](/self-hosted#upgrading) first.

5 · [IMAGE] Screenshot of Settings → Logs in "Turning it on"

• Doc: message-recording.mdx, "Turning it on" — no Frame/image on the page (autofix.mdx sets the precedent with its Frames).
• Platform: Settings.tsx:234, SettingsRecordingSection.tsx:35-63.
• Where and what: one screenshot in "Turning it on" showing the harness Settings page, "Logs" section visible, "Enable logs" switch on, including the storage copy (self-hosted local vs cloud workspace). Framed like autofix-settings-toggle.png. Especially valuable because the doc term and the UI term differ (finding 1): the picture removes the ambiguity even if wording drifts again.

6 · [IMAGE] Screenshot of the Attempts sidebar and Messages tab in "Reading it back"

• Doc: message-recording.mdx, "Reading it back" — no image.
• Platform: RequestDrawer.tsx:240-257, 363-366.
• Where and what: one screenshot at the top of the rewritten section (finding 3): the drawer open on a request with at least two provider attempts (ideally a fallback or Auto-fix retry), Attempts sidebar visible, Messages tab selected. The per-attempt navigation is the least guessable part of the feature. Suggested path: /images/message-recording-drawer.png.

---

Répondre avec le numéro du finding et l'option choisie
(ex: "3 → option 2", ou "3: autre idée, discutons").
Les PRs se font ensuite manuellement avec l'agent, jamais par le harness.
Findings rejetés = répondre "N → reject" : ils ne seront plus re-signalés.

[SebConejo]

Verdicts (recorded in references/decisions.md):

• 1 → option 1: use the UI's real name: Logs section, Enable logs switch.
• 2 → custom: do not narrate the feature's add/remove/re-add history. Replace the whole claim with the simple useful sentence: to make sure your logs are on, open the harness's Settings → Logs. Less beats confusing.
• 3 → option 1, with vocabulary: describe the current navigation using the product words request and attempt (the page is called Requests; the /messages URL will be renamed platform-side later). Also ADD glossary entries: request, attempt, and Autofix (its scope: it fixes the request, not the prompt), written in end-user language.
• 4 → custom: put a Cloud / Self-hosted toggle on the section. Self-hosted side: logs were reintroduced in ; older installs must upgrade, link to the Upgrading section of the self-hosted page.
• 5 → accepted: Seb produces the image (Settings page, Enable logs switch visible).
• 6 → accepted: Seb produces the image (a request opened, attempts list and Messages tab visible).

PRs to be made manually with the agent. Lessons recorded: findings must be written in plain language (the IMAGE findings were unreadable jargon).