Skip to content

Build privacy controls for transcripts, audio, cache, data export, and deletion#124

Merged
charneykaye merged 24 commits into
mainfrom
vibrator/issue-47-build-privacy-controls-for-transcripts-a
Jul 8, 2026
Merged

Build privacy controls for transcripts, audio, cache, data export, and deletion#124
charneykaye merged 24 commits into
mainfrom
vibrator/issue-47-build-privacy-controls-for-transcripts-a

Conversation

@charneykaye

@charneykaye charneykaye commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR implements comprehensive privacy controls for ConversationSimulator by adding user preferences for transcript and data persistence, combined with bulk and per-session data management controls. It addresses issue #47 by building a full privacy-centric feature set.

Key Features

Privacy Preferences

  • Save transcripts: Toggle whether conversation transcripts are persisted (default: enabled)
  • TTS cache: Toggle whether generated speech is cached locally (default: enabled)
  • Raw audio: Advanced option to save unprocessed microphone recordings for debugging (default: disabled)
  • All preferences are persisted to localStorage and applied to new sessions

Backend Privacy Enforcement

  • When save_transcript=false, event records are skipped during session operation (start, turn, end, debrief)
  • Real-time events are still delivered in API responses using synthetic event IDs for immediate UI display
  • Export endpoint returns empty event arrays for sessions created with transcript saving disabled, enforcing the user's stated privacy preference

Data Management UI

  • Clear all local data: Single-click (with confirmation) deletion of all sessions, transcripts, and caches
  • Per-session export: Download any session as JSON for archival or analysis
  • Per-session delete: Individually delete specific sessions with confirmation
  • Data folder info: Display of the local data directory path for user transparency

API Additions

  • GET /api/privacy/data-folder — retrieves the path where all local data is stored
  • POST /api/privacy/clear — deletes all sessions and events from the database
  • GET /api/sessions/:session_id/export — exports a session and its events as JSON
  • DELETE /api/sessions/:session_id — deletes a specific session
  • Client API methods: getDataFolder(), clearLocalData(), exportSession(), deleteSession()

Implementation Details

  • Privacy preferences are read from localStorage on component mount with sensible defaults
  • The backend respects save_transcript at session creation time and consistently enforces it across all event-persisting endpoints (start, turn, end, debrief)
  • Session exports transparently include the save_transcript setting in the setup JSON so users can see whether their preference was honored
  • Comprehensive test coverage for all privacy endpoints and transcript enforcement

Testing

  • 261 lines of API tests covering privacy routes (data folder, clear, export, delete)
  • 454 lines of UI tests for Settings component (preferences, session management, error handling)
  • Tests verify that save_transcript=false sessions do not persist events but still deliver them in real-time responses
  • Tests confirm state machine correctness for clear-data operation (idle → confirming → clearing → done/error)
  • Tests validate error handling and recovery in session deletion and export flows

Closes #47

charneykaye and others added 2 commits July 7, 2026 20:02
Implements issue #47: exposes privacy settings and backend actions so users
can manage all locally-stored conversation data.

Backend (apps/api):
- Add GET /api/privacy/data-folder — returns the path where data is stored.
- Add POST /api/privacy/clear — atomically deletes all sessions and events,
  returning the count of deleted sessions.
- Add GET /api/sessions/:session_id/export — returns a full JSON export of
  a session (metadata, parsed setup/state-vars, and all events).
- Wire setDataFolderPath() in index.ts so the live server reports the real
  data directory.
- Add privacy.test.ts: 15 backend tests covering all three new endpoints,
  idempotency, cascade deletion, and export shape.

Frontend (apps/web):
- Extend api/client.ts: add deleteSession, exportSession, getDataFolder,
  and clearLocalData helpers; make post() body optional and add a del() helper.
- Rewrite screens/Settings.tsx with a full Privacy Settings UI:
  - Prominent notice that no data is sent to external servers.
  - Save-transcripts toggle with plain-language explanation and live note
    (local-only / not-saved) that updates on toggle.
  - Cache-TTS-audio toggle.
  - Data-folder path display (loaded from API).
  - Two-step clear-all-local-data action: first click shows a confirmation
    alert; second click calls the API; result shows deleted count or error.
  - Advanced section (collapsed by default) revealing a raw-audio-saving
    toggle, which is off by default and shows a warning when enabled.
- Add __tests__/Settings.test.tsx: 25 frontend tests covering all toggles,
  the local-only/not-saved notes, the data-folder display, the two-step
  clear flow (confirm, cancel, success, error, singular count), and the
  advanced raw-audio section.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The Settings screen toggles (save transcripts, TTS cache, raw audio) were
pure local React state — values reset on every navigation and the transcript
toggle had no effect on the session setup form default.

Extract privacyPrefs.ts with typed keys and read/write helpers guarded for
environments where localStorage may be absent. Initialize each Settings toggle
from localStorage and write back on change. Have ScenarioSetupPage read the
persisted saveTranscripts preference as the initial save_transcript default.

Add a localStorage stub in setupTests.ts (jsdom 25 dropped it) and clear it
in Settings.test.tsx beforeEach to prevent inter-test state leakage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

charneykaye and others added 2 commits July 7, 2026 20:24
After a successful clear or an API error, the 'Clear all local data'
button remained visible and enabled but clicking it was a no-op —
handleClearLocalData only handled 'idle' and 'confirming', leaving
'done' and 'error' states with no transition back into the flow.

Treat 'done' and 'error' the same as 'idle' so a second click
restarts the two-step confirmation. Two new tests cover both paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

charneykaye and others added 2 commits July 7, 2026 20:34
The issue acceptance criteria required that a user can delete a session
and have it disappear, and that export session JSON is user-accessible.
Only the bulk clear-data action was exposed; individual sessions had no
UI even though the delete and export API routes existed.

- Add GET /api/sessions list endpoint (newest-first) with five tests
- Wire api.listSessions() in the web client
- Add "Your sessions" section in Settings: shows all sessions with
  per-session Export (JSON download) and Delete (two-step confirm)
  buttons; list refreshes after bulk clear or individual delete
- Add vi.restoreAllMocks() to beforeEach to prevent spy leak between
  Settings tests; add seven tests covering the new sessions UI

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

Session mock objects used plain string literals for `state`, causing TS
to widen the type to `string` instead of the `SessionState` union. Add
`as const` to keep the literal type, and cast `setup: {}` via
`as unknown as SessionCreateRequest` to satisfy the required fields.

The `exportSession` mock was also missing `ending_type`, `state_vars`,
and `turn_count` required by the `SessionExport.session` shape; spread
those fields in with appropriate defaults.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Addressed failing CI checks and pushed a fix (unknown checks).

charneykaye and others added 2 commits July 7, 2026 20:58
Both functions had no catch block, so API failures produced silent
unhandled promise rejections with no user-visible feedback. Added
deleteError and exportError state, display them as role="alert"
paragraphs in the sessions section, and added tests for both failure
paths to ensure the error message is surfaced and the session row
remains visible after a failed delete.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

…xport

Add rowid DESC as a tiebreaker to ORDER BY created_at DESC in GET /api/sessions
so sessions created in the same millisecond always return newest-first, making
the ordering test deterministic.

Replace the ...row spread in the export endpoint with explicit field selection to
prevent setup_json and state_vars_json (raw internal strings) from leaking into
the exported JSON alongside the already-parsed setup and state_vars objects.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

Settings.tsx used React.ReactNode in SectionHeading prop type without
importing the React namespace. With jsx: react-jsx and strict mode (no
allowUmdGlobalAccess), this is a type error. Import ReactNode directly
as a named type, matching the pattern used elsewhere (e.g. ErrorBoundary).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

…lk clear

loadSessions() now resets sessionsError to false on a successful reload so the
'Could not load sessions' banner disappears once the API recovers (e.g. after
the user clears local data and the list refreshes successfully).

handleClearLocalData() also clears any stale per-session deleteError/exportError
messages on success, since those sessions no longer exist after a bulk clear.

Added a regression test covering the sessionsError recovery path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

charneykaye and others added 2 commits July 7, 2026 21:25
Append the download anchor to document.body before clicking so that
Safari (and older WebKit) will honour the programmatic click — detached
elements are silently ignored in those engines.

Also optimistically clear the sessions list and any pending per-session
delete confirmation immediately after a successful clearLocalData call,
so the UI never shows deleted sessions even if the subsequent reload
fails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

charneykaye and others added 2 commits July 7, 2026 21:30
The merge of origin/main into this branch silently dropped the executable-file
and magic-byte security scanning that was added in main after this branch was
cut.  The conflict was resolved by keeping the older (pre-security) versions of
loader.ts and validator.py, removing FORBIDDEN_BINARY/FORBIDDEN_FILE checks and
their tests entirely.

This commit restores those five files verbatim from origin/main so the security
properties — rejecting symlinks, forbidden extensions, and disguised binaries in
scenario packs — are not regressed by the privacy-controls PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code, no issues found.

When a session is created with save_transcript=false, the start/turn/end/debrief
handlers no longer insert rows into session_events. HTTP responses still carry
the NPC opening and turn events so the frontend can display them in real time,
but using a synthetic event_id=0 that signals they are ephemeral. The export
endpoint therefore returns an empty events array for these sessions, matching
what the UI promises ("Transcripts will not be saved").

The privacy.test.ts test that previously documented the broken behaviour
("export still works" despite save_transcript=false) is updated to assert the
correct invariant and to verify the start response still delivers the NPC
opening for real-time display.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

The existing save_transcript=false test only covered the start event
(npc_opening). The turn handler has its own save-gate path, which was
code-correct but untested. This test verifies that player_turn and
npc_turn events are still returned in the API response for real-time
display but are not written to the database.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

…ence

The end and debrief handlers already skip insertEvent when save_transcript=false,
but there were no tests verifying this. Add two tests to privacy.test.ts that
cover session_ended and debrief_generated events for opt-out sessions, keeping
test coverage consistent with the existing start/turn save_transcript=false tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code, no issues found.

charneykaye and others added 2 commits July 7, 2026 23:05
renderSettings() now awaits both mount effects (getDataFolder +
listSessions) before returning, so all 38 call sites flush pending
state updates inside act() rather than after test cleanup. All 37
Settings tests still pass; no act() warnings remain.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

Settings.tsx and its tests reference getDataFolder, clearLocalData,
deleteSession, and exportSession on the api object, but these were not
defined. This wired them up to the existing backend endpoints
(GET /api/privacy/data-folder, POST /api/privacy/clear,
DELETE /api/sessions/:id, GET /api/sessions/:id/export) and put the
previously-declared-but-unused del() helper to use.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Addressed failing CI checks and pushed a fix (unknown checks).

…red in start/turn routes

- Import privacyRoutes and setDataFolderPath in index.ts; register privacyRoutes in buildApp()
  so that /api/privacy/data-folder, /api/privacy/clear, and /api/sessions/:id/export are reachable
- Start route: check shouldSaveTranscript before calling insertEvent for npc_opening; define
  openingPayload and openingAt before the transaction so the synthetic fallback event has real data
- Turn route: define turnAt before the response block; without it the else branch threw a
  ReferenceError, causing the turn to return a 500 with no events field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code and pushed fixes.

@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code, no issues found.

1 similar comment
@charneykaye

Copy link
Copy Markdown
Contributor Author

Reviewed code, no issues found.

@charneykaye charneykaye marked this pull request as ready for review July 8, 2026 06:29
@charneykaye charneykaye merged commit 42ad36c into main Jul 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build privacy controls for transcripts, audio, cache, data export, and deletion

1 participant