feat(audio): add opt-in local speech memory channel by Coldaine · Pull Request #5 · MooseGooseConsulting/screenpipe

Coldaine · 2026-08-08T07:31:24Z

User description

Summary

add an opt-in Windows audio capture/transcription channel with no raw audio persistence
preserve identical transcripts from distinct VAD utterance windows while deduplicating retries within one utterance
add dedicated Windows audio CI with pinned LLVM 18.1.8, Ninja 1.12.1, LIBCLANG_PATH, and CMAKE_GENERATOR=Ninja
update the pruning verifier and README for the optional audio workspace/feature

Validation

deterministic non-audio CI-equivalent suite: 200 passed
cargo test -p screenpipe-memory --test merge_contract: 38 passed
pinned-Ninja cargo test -p screenpipe-audio: 20 passed
pinned-Ninja cargo check -p screenpipe-cli --features audio: passed
scripts/verify-pruned.ps1: passed
workflow YAML, README PowerShell parsing, fmt, clippy, and diff checks: passed

Evidence boundaries

hosted CI is required before merge and is not claimed by local results
live foreground capture and live endpoint/model/database resource qualification remain unproven
resource figures are category-only whole-host synthetic/offline measurements; no device names or transcripts were recorded

Documentation impact

Durable behavior changed, so README is updated in this PR for opt-in semantics, utterance-scoped deduplication, and the qualified audio build path.

CodeAnt-AI Description

Add an opt-in local audio channel for private speech transcription

What Changed

Adds Windows audio capture with system-audio loopback by default and an explicit --microphone option for recording the room
Transcribes utterances locally with voice-activity boundaries, silence filtering, and no raw audio persistence
Records audio events in the existing event store with channel, endpoint role, model, confidence, and utterance metadata
Keeps repeated words in separate utterances while deduplicating retries from the same utterance
Adds audio doctor, foreground recording, and a separate audio scheduled service
Keeps audio disabled in default builds and prevents unsupported builds from installing a recording service

Impact

✅ Private on-device transcription
✅ Fewer silence and noise transcript entries
✅ Separate control for room microphone capture

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

A third channel beside screen and clipboard: WASAPI capture, WebRTC voice activity detection, and whisper.cpp in this process. Speech becomes `events` of kind `audio` in the same table, through the same writer, with no schema change at all. Off, and off in two independent ways. `audio` is a Cargo feature disabled by default, so the shipped binary has no capture path, no model loader, and nothing that can open a microphone - and building it needs neither CMake nor libclang. Even a build that HAS the feature records nothing until somebody runs `screenpipe audio run` or installs its own scheduled task; `screenpipe run` starts it never, and `screenpipe service install` installs it never. `audio service` is deliberately outside the feature gate, because a build that cannot record must still be able to remove a task an earlier build installed. Loopback is the default, not the microphone. Loopback hears what came out of the speakers; the microphone hears the room and everyone in it, which is a second decision the operator has to make explicitly, and every run that makes it says so at the top of the log. Silence decides the boundaries. Three voiced 20 ms frames open an utterance, 600 ms of silence closes it, the 200 ms before the trigger is kept so the first consonant survives, and the trailing silence is cut before the model sees it. Whisper is never asked where speech starts - that would cost several orders of magnitude more than a signal test built for the job. Each utterance is then its own event unless its transcript is identical to the open one. That is the clipboard's rule, and it is load-bearing here for a different reason: the writer REPLACES ocr_text on a merge, so merging two different transcripts would keep the second and lose the first. What it does merge is repetition, which is whisper's failure mode over a quiet room. What it refuses to record: any device name (the endpoint ROLE is stored instead - console or communications - because "Microphone (Realtek High Definition Audio)" identifies hardware in someone's house); audio under 250 ms; segments the model itself scores above 0.6 no-speech; and anything at all when nothing survives that filter. whisper.cpp's own stdout logging is routed into hooks that lead nowhere, because this process's stdout is a log file kept on disk and what whisper prints is somebody's speech. Three threads, because the capture thread must never block: the audio engine's buffer is finite and a stalled reader loses audio with no record that it happened. It hands closed utterances to the transcriber and drops them loudly if that queue is full. Dropping happens at the cheap end, never at the durable one. Two things found by running it rather than reasoning about it. A loopback stream delivers NOTHING while nothing is playing - not silence, nothing - so the first live run declared the stream stalled two seconds after opening on a perfectly healthy machine; a timed-out wait is now silence. And whisper-rs-sys's bundled bindings are glibc-generated and will not compile on MSVC, so libclang is genuinely required for this feature. Verified live: two spoken sentences captured over loopback, transcribed, and written to the disposable test database as kind='audio' rows with word-perfect transcripts, correct titles, and full merge_meta. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HhXifh8P7AhWdAcBnSSW5b

Found by review on the design PR, and it was a real defect rather than a documentation one. The merger's idle-gap test measures `next.captured_at - open.ended_at`, and an utterance's timestamp is when the speech STARTED - so the "silence" between two turns silently included the length of the first one. A 30-second sentence followed by a 35-second pause read as a 65-second gap and split at a 60-second threshold that 35 seconds of silence never crossed. The claim in the design document that the interval between two audio samples IS the silence was simply wrong. `ObservationSample.observed_until` fixes it at the root: an observation may now occupy a span, `None` means it was an instant, and the merger takes the event's `ended_at` from it. Screen and clipboard samples are instants and nothing about their windows moves - there is a test for that. It also makes the durable window true. `ended_at - started_at` is now how long the speech actually ran, so `merge_meta.audio.duration_ms` - added an hour ago as a workaround for the window being an instant - is gone rather than left as a second source of truth that can disagree. Also from that review: `audio service install` is refused in a build without the feature. Uninstall and status stay open in every build, which was deliberate - a build that cannot record must still be able to turn one off - but install would have registered a task running `audio run` from a binary that refuses `audio run`, and the operator's only evidence would be a log they have no reason to read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HhXifh8P7AhWdAcBnSSW5b

codeant-ai · 2026-08-08T07:31:28Z

🤖 CodeAnt AI — Review Status

Status	Commit	Started (UTC)	Finished (UTC)
✅ Reviewed your PR	`702c501`	Aug 08, 2026 · 07:31	07:34

codeant-ai · 2026-08-08T07:31:29Z

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

Coldaine · 2026-08-08T07:31:48Z

@codex review

Copilot

Pull request overview

Adds a Windows-only, opt-in audio capture/transcription “memory” channel that records transcripts as events(kind=audio) without persisting raw audio, while keeping default builds free of whisper.cpp/CMake/libclang requirements and adding dedicated CI coverage for the audio toolchain.

Changes:

Introduces screenpipe-audio (WASAPI capture + WebRTC VAD + local whisper.cpp transcription) and wires it into the CLI behind an audio feature flag.
Extends the merge contract (v6) to support utterance-window-scoped audio deduplication and span-based observations via observed_until + merge_meta.audio.
Updates workspace pruning verification, README build/usage docs, and CI to keep audio opt-in while still testing it on Windows with pinned LLVM/Ninja.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
scripts/verify-pruned.ps1	Verifies audio is a workspace member but not a default member, and keeps pruning constraints intact.
README.md	Documents opt-in audio semantics, build steps (incl. pinned Ninja/LLVM), and the durable audio event contract.
crates/screenpipe-screen/tests/merge_boundary_live.rs	Updates test sample construction for new `ObservationSample` fields.
crates/screenpipe-memory/tests/postgres_writer.rs	Adds persistence assertions for audio `kind/title/merge_meta.audio` and absence of audio key for non-audio rows.
crates/screenpipe-memory/tests/merge_contract.rs	Expands merge contract tests for audio utterance identity/dedup and span-based idle-gap behavior.
crates/screenpipe-memory/tests/debug_redaction.rs	Updates redaction test fixtures for new sample fields.
crates/screenpipe-memory/src/sample.rs	Adds `observed_until` + `AudioMeta` to `ObservationSample` and exposes `observed_until()` helper.
crates/screenpipe-memory/src/runner.rs	Updates runner test fixtures for new sample fields.
crates/screenpipe-memory/src/postgres.rs	Writes `merge_meta.audio` only when present (no `"audio": null` on other event kinds).
crates/screenpipe-memory/src/merge.rs	Adds `EventKind::Audio`, bumps merge contract version, and implements utterance-window-based audio merge hashing/splitting.
crates/screenpipe-memory/src/lib.rs	Re-exports `AudioMeta` for downstream (CLI/audio channel) use.
crates/screenpipe-cli/tests/cli_surface.rs	Adds CLI surface/consent assertions for `audio` help text and feature-gated behavior.
crates/screenpipe-cli/src/windows_source.rs	Updates screen sample construction for new sample fields.
crates/screenpipe-cli/src/service.rs	Splits service management into `ServiceKind::{Screen,Audio}` with separate task/binary/wrapper/log stems.
crates/screenpipe-cli/src/main.rs	Adds `audio` subcommands, feature-gated audio runtime loop/doctor, and routes audio service actions.
crates/screenpipe-cli/src/clipboard_source.rs	Updates clipboard sample construction for new sample fields.
crates/screenpipe-cli/src/audio_source.rs	New audio sample source implementing capture/transcribe/write pipeline and fixed diagnostic strings.
crates/screenpipe-cli/Cargo.toml	Introduces `audio` feature (off by default) with optional `screenpipe-audio` dependency.
crates/screenpipe-audio/src/vad.rs	New VAD segmentation (utterance boundaries, preroll, hangover, max length) with tests.
crates/screenpipe-audio/src/transcribe.rs	New local whisper.cpp transcription wrapper, model path guarding, and no-speech filtering.
crates/screenpipe-audio/src/lib.rs	New crate entrypoint and public API for capture/VAD/transcription/channel codes.
crates/screenpipe-audio/src/device.rs	New endpoint-role categorization logic avoiding device-name reads/persistence.
crates/screenpipe-audio/src/capture.rs	New WASAPI loopback/mic capture producing fixed-shape frames with timeout-as-silence handling.
crates/screenpipe-audio/Cargo.toml	Adds dependencies for WASAPI, WebRTC VAD, and whisper-rs (CPU-only defaults).
Cargo.toml	Adds audio crate to workspace members while keeping it out of default-members (opt-in build).
Cargo.lock	Records new dependency graph for audio + whisper-rs/wasapi/webrtc-vad.
.github/workflows/ci.yml	Splits non-audio CI from a dedicated Windows audio job with pinned LLVM/Ninja + env vars.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

qodo-code-review · 2026-08-08T07:38:27Z

PR Summary by Qodo

feat(audio): opt-in local speech capture/transcription channel

✨ Enhancement 🧪 Tests ⚙️ Configuration changes 📝 Documentation 🕐 40+ Minutes

AI Description

• Adds an opt-in Windows audio channel: WASAPI capture, WebRTC VAD, local whisper transcription.
• Keeps audio fully off by default via workspace default-members + CLI audio feature gate.
• Introduces audio events with utterance-window-scoped deduplication and new merge contract v6.

Diagram

graph TD
  F["screenpipe-cli audio"] --> A["WASAPI capture"] --> B["VAD utterances"] --> C["Whisper transcribe"] --> D["Merger / Runner"] --> E[("Postgres events")]

High-Level Assessment

The following are alternative approaches to this PR:

1. Ship a separate dedicated audio CLI binary

➕ Avoids feature-gating complexity in the main binary surface
➕ Can package whisper/toolchain requirements independently
➖ More artifacts to distribute and secure
➖ Duplicates service/CLI plumbing already shared in screenpipe-cli

2. Run transcription via an external process (whisper-cli)

➕ Reduces in-process native binding surface area
➕ Isolates crashes/ABI issues from the capture process
➖ Harder packaging on Windows; still needs LLVM/CMake at build or bundled binaries
➖ More IPC/IO complexity; transcript leakage risk shifts to process management

Recommendation: Current approach is appropriate: it keeps the privacy-sensitive channel compiled out by default, preserves a single shared event/merge/write pipeline (no schema change), and provides clean operational separation via distinct audio subcommands and a separate scheduled task identity. The main tradeoff (feature gating + native toolchain complexity) is mitigated by the pinned Windows CI job and the explicit README build path.

Files changed (27) +3323 / -67

Enhancement (13) +2531 / -61

Cargo.tomlIntroduce new screenpipe-audio crate manifest +31/-0

Introduce new screenpipe-audio crate manifest
• Creates the new crate with dependencies on wasapi (loopback capture), webrtc-vad (utterance segmentation), and whisper-rs (local transcription), plus hound for tests.
crates/screenpipe-audio/Cargo.toml

capture.rsImplement WASAPI capture with 16kHz mono framing +303/-0

Implement WASAPI capture with 16kHz mono framing

• Implements Windows-only capture for loopback/microphone into fixed-size frames for VAD, including special handling where loopback delivers no frames during silence (manufactures silence frames rather than treating it as a stall). Avoids device-name leakage in errors.

crates/screenpipe-audio/src/capture.rs

device.rsCategorize default endpoint role without reading device names +100/-0

Categorize default endpoint role without reading device names
• Adds DeviceCategory (console/communications) by comparing endpoint IDs (opaque, not persisted). Provides a 'doctor'-style endpoint role check without opening a stream.
crates/screenpipe-audio/src/device.rs

lib.rsDefine audio crate API and Channel semantics +75/-0

Define audio crate API and Channel semantics
• Exports capture/VAD/transcription primitives and defines 'Channel' (Loopback/Microphone) with stable persisted codes and app attribution keys/titles used for event labeling.
crates/screenpipe-audio/src/lib.rs

transcribe.rsAdd local whisper transcription with no-speech filtering +329/-0

Add local whisper transcription with no-speech filtering

• Implements ggml model loading and per-utterance transcription, suppressing whisper.cpp stdout printing to avoid transcript leakage. Drops too-short audio and high no-speech-probability segments, and persists only model stem (not path).

crates/screenpipe-audio/src/transcribe.rs

vad.rsAdd WebRTC VAD utterance segmentation with preroll and caps +577/-0

Add WebRTC VAD utterance segmentation with preroll and caps

• Implements a testable VAD-driven state machine: opens after 3 voiced frames, closes after 600ms silence, keeps 200ms preroll, trims trailing silence, and caps utterances at 30s. Includes scripted-detector tests to avoid committing real speech fixtures.

crates/screenpipe-audio/src/vad.rs

audio_source.rsWire capture→VAD→transcribe into SampleSource via 3 threads +386/-0

Wire capture→VAD→transcribe into SampleSource via 3 threads

• Adds AudioSampleSource and its worker loops: capture thread (never blocks, drops on backlog), transcription thread (owns model), and async loop (feeds merger/writer). Ensures fixed-string diagnostics and explicit no-speech discard behavior.

crates/screenpipe-cli/src/audio_source.rs

main.rsAdd 'audio' subcommand with run/doctor/service and feature gating +399/-5

Add 'audio' subcommand with run/doctor/service and feature gating

• Adds 'screenpipe audio' command surface with Run/Doctor/Service subcommands. Keeps 'audio service' available even without the feature so tasks can be removed; blocks installing an audio service from a non-audio build with actionable guidance.

crates/screenpipe-cli/src/main.rs

service.rsSupport separate Task Scheduler identities for screen vs audio +145/-38

Support separate Task Scheduler identities for screen vs audio
• Introduces ServiceKind to separate task name, binary name, wrapper, and log stem for screen and audio services, enabling independent install/uninstall without deleting a sibling artifact.
crates/screenpipe-cli/src/service.rs

lib.rsRe-export AudioMeta for downstream use +1/-1

Re-export AudioMeta for downstream use
• Exports 'AudioMeta' alongside 'ObservationSample' so producers/consumers can construct and inspect audio metadata.
crates/screenpipe-memory/src/lib.rs

merge.rsAdd EventKind::Audio and utterance-window-based merge identity +95/-15

Add EventKind::Audio and utterance-window-based merge identity

• Bumps MERGE_CONTRACT_VERSION to 6, introduces EventKind::Audio, computes audio merge identity from transcript hash + utterance start/end timestamps (dedupe retries within one utterance, preserve repeated phrases across utterances), and measures idle gap from 'observed_until'.

crates/screenpipe-memory/src/merge.rs

postgres.rsPersist merge_meta.audio only for audio events +24/-2

Persist merge_meta.audio only for audio events
• Extends merge_meta JSONB generation to include an 'audio' object when AudioMeta is present, while keeping the key absent for screen/clipboard rows (not null).
crates/screenpipe-memory/src/postgres.rs

sample.rsExtend ObservationSample with observed_until and AudioMeta +66/-0

Extend ObservationSample with observed_until and AudioMeta

• Adds 'observed_until' to represent span-based observations (utterances) and 'audio: Option<AudioMeta>' for audio-only metadata; includes 'observed_until()' helper and Debug behavior that keeps audio meta visible while redacting sensitive URLs.

crates/screenpipe-memory/src/sample.rs

Refactor (2) +4 / -0

clipboard_source.rsAdapt clipboard samples to new ObservationSample fields +2/-0

Adapt clipboard samples to new ObservationSample fields
• Populates 'observed_until: None' and 'audio: None' for clipboard samples to match the expanded ObservationSample struct.
crates/screenpipe-cli/src/clipboard_source.rs

windows_source.rsAdapt screen samples to new ObservationSample fields +2/-0

Adapt screen samples to new ObservationSample fields
• Populates 'observed_until: None' and 'audio: None' for screen samples to match the expanded ObservationSample struct.
crates/screenpipe-cli/src/windows_source.rs

Documentation (1) +140 / -2

README.mdDocument opt-in audio channel, boundaries, and build steps +140/-2

Document opt-in audio channel, boundaries, and build steps

• Adds detailed documentation for the audio channel: off-by-default semantics, pinned Ninja/LLVM setup, model location/override, privacy boundaries (no device names, no transcript logging), utterance segmentation, and utterance-scoped deduplication behavior.

README.md

Other (11) +648 / -4

ci.ymlSplit non-audio CI and add pinned Windows audio job +58/-2

Split non-audio CI and add pinned Windows audio job

• Renames the existing Windows job to explicitly cover non-audio builds and excludes screenpipe-audio from clippy/tests. Adds a dedicated Windows job that installs LLVM 18.1.8 + Ninja 1.12.1, sets LIBCLANG_PATH and CMAKE_GENERATOR=Ninja, then tests 'screenpipe-audio' and checks 'screenpipe-cli --features audio'.

.github/workflows/ci.yml

Cargo.lockLock new audio-related dependencies +97/-0

Lock new audio-related dependencies

• Updates the lockfile for the new audio crate and whisper build chain dependencies (e.g., cmake/fs_extra) and related transitive changes.

Cargo.lock

Cargo.tomlMake screenpipe-audio a non-default workspace member +16/-0

Make screenpipe-audio a non-default workspace member

• Adds 'crates/screenpipe-audio' to workspace members but keeps default-members limited to screen/memory/cli so normal builds/tests avoid the whisper.cpp toolchain unless explicitly requested.

Cargo.toml

Cargo.tomlAdd 'audio' feature flag and optional dependency +16/-0

Add 'audio' feature flag and optional dependency
• Introduces a new 'audio' feature (default off) that enables the optional 'screenpipe-audio' dependency; without it, the binary has no audio recording code paths.
crates/screenpipe-cli/Cargo.toml

cli_surface.rsAdd CLI contract tests for audio opt-in semantics +76/-0

Add CLI contract tests for audio opt-in semantics

• Adds tests asserting help text communicates OFF-by-default and consent guidance, microphone is a separate opt-in, audio service status works without the feature, and non-audio builds fail audio run/install with actionable error messages.

crates/screenpipe-cli/tests/cli_surface.rs

runner.rsUpdate runner test fixtures for new sample fields +2/-0

Update runner test fixtures for new sample fields
• Updates ObservationSample construction in tests to set 'observed_until: None' and 'audio: None'.
crates/screenpipe-memory/src/runner.rs

debug_redaction.rsUpdate debug redaction tests for new fields +2/-0

Update debug redaction tests for new fields
• Extends the sensitive sample fixture with 'observed_until' and 'audio' fields to keep redaction behavior tests compiling and correct.
crates/screenpipe-memory/tests/debug_redaction.rs

merge_contract.rsAdd audio merge contract test suite +253/-0

Add audio merge contract test suite

• Adds contract tests that pin audio event kind behavior, deduplication boundaries (distinct utterance windows vs same-window retries), idle gap semantics measured from utterance end, and presence/absence of audio metadata.

crates/screenpipe-memory/tests/merge_contract.rs

postgres_writer.rsAdd Postgres persistence tests for audio merge_meta and title +107/-0

Add Postgres persistence tests for audio merge_meta and title
• Adds integration tests verifying audio events persist with kind=audio, correct title via apps join, raw/readable transcript split, and merge_meta.audio; also asserts screen events do not include an audio key.
crates/screenpipe-memory/tests/postgres_writer.rs

merge_boundary_live.rsUpdate live screen test fixtures for new fields +2/-0

Update live screen test fixtures for new fields
• Updates the live observation fixture to include 'observed_until: None' and 'audio: None'.
crates/screenpipe-screen/tests/merge_boundary_live.rs

verify-pruned.ps1Verify workspace remains pruned while audio stays opt-in +19/-2

Verify workspace remains pruned while audio stays opt-in
• Adds 'screenpipe-audio' to expected workspace packages, verifies default-members still exclude it, and updates messaging/forbidden dependency checks to match the new opt-in model.
scripts/verify-pruned.ps1

qodo-code-review · 2026-08-08T07:39:12Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. ~~Flushed utterance can drop~~ ✓ Resolved 🐞 Bug ≡ Correctness

Description

On capture failure, capture_loop flushes the in-flight utterance but still sends it via offer(),
which uses a bounded try_send and drops the utterance when the queue is full. This can lose the
final utterance despite the comment claiming it will still be written.

Code

crates/screenpipe-cli/src/audio_source.rs[R225-229]

+                // The utterance in flight is worth more than the failure: flush
+                // before reporting, so a stream that dies mid-sentence still
+                // writes the sentence.
+                if let Some(utterance) = segmenter.flush() {
+                    offer(utterances, utterance);

Evidence

The capture error handler flushes and calls offer(), but offer() drops on TrySendError::Full, so the
flushed utterance is not guaranteed to reach the transcriber.

crates/screenpipe-cli/src/audio_source.rs[223-252]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
In the capture error path, the code flushes an in-flight utterance and intends to preserve it, but it is sent via `offer()` which may drop it when the transcribe queue is full.

## Issue Context
`offer()` uses `SyncSender::try_send` to avoid blocking the capture thread under normal operation. However, on a terminal capture error (the thread is about to return), preserving the in-flight utterance likely matters more than strict nonblocking behavior, and the current comment asserts a stronger guarantee than the implementation provides.

## Fix Focus Areas
- crates/screenpipe-cli/src/audio_source.rs[223-233]
- crates/screenpipe-cli/src/audio_source.rs[241-252]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. ~~Unverified CI binary downloads~~ ✓ Resolved 🐞 Bug ⛨ Security

Description

The new Windows audio CI job downloads and executes an LLVM installer and extracts a Ninja zip
without verifying cryptographic integrity, enabling supply-chain substitution to execute arbitrary
code in CI. This is a CI trust-boundary risk that could impact workflow secrets and test/build
correctness.

Code

.github/workflows/ci.yml[R135-137]

+          $uri = "https://github.com/llvm/llvm-project/releases/download/llvmorg-$env:LLVM_VERSION/LLVM-$env:LLVM_VERSION-win64.exe"
+          Invoke-WebRequest -Uri $uri -OutFile $installer
+          Start-Process -FilePath $installer -ArgumentList '/S' -Wait -WindowStyle Hidden

Evidence

The workflow uses Invoke-WebRequest to fetch a Windows LLVM installer and then executes it, and
separately downloads and extracts Ninja; neither path validates an expected digest/signature before
running the artifacts.

.github/workflows/ci.yml[131-151]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The audio CI job downloads LLVM and Ninja and executes/extracts them without verifying integrity (no SHA256/signature check). This creates a supply-chain execution risk in CI.

## Issue Context
The job pins versions via env vars but pinning does not authenticate the bytes. Add a checksum verification step before `Start-Process` / `Expand-Archive`.

## Fix Focus Areas
- .github/workflows/ci.yml[131-158]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. ~~String-matched shutdown signal~~ ✓ Resolved 🐞 Bug ☼ Reliability

Description

The audio run loop decides a terminal shutdown by checking whether the formatted anyhow error string
contains CAPTURE_STOPPED, which is brittle and can misclassify errors if messages change or are
wrapped differently. This can cause incorrect retry/exit behavior over time as error text evolves.

Code

crates/screenpipe-cli/src/main.rs[R1042-1046]

+                    Err(error) if format!("{error:#}").contains(CAPTURE_STOPPED) => {
+                        // Not retryable. The WASAPI stream and the model live
+                        // on threads that have exited; backing off would print
+                        // the same line every second until logoff. Exiting
+                        // hands off to the service wrapper's restart loop,

Evidence

The run loop’s terminal branch is selected based on whether the formatted error string contains
CAPTURE_STOPPED, rather than on a typed/structured error signal.

crates/screenpipe-cli/src/main.rs[1037-1050]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Audio shutdown behavior is triggered via substring matching on `format!("{error:#}")`, which relies on human-readable error text remaining stable.

## Issue Context
`CAPTURE_STOPPED` is a constant, but the matching happens on a formatted error chain; adding context, rewording, or wrapping can change the rendered string and break the classification.

## Fix Focus Areas
- crates/screenpipe-cli/src/main.rs[1037-1050]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 702c501c35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 762b085feb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Coldaine and others added 6 commits August 7, 2026 17:40

Merge origin/main into feat/audio-channel

6ebbb9b

fix(memory): scope audio dedup to one utterance

c2b4897

ci(audio): qualify the opt-in Windows feature

ea530ab

fix(ci): pin the proven Ninja audio generator

702c501

Copilot AI lite review requested due to automatic review settings August 8, 2026 07:31

Copilot started reviewing on behalf of Coldaine August 8, 2026 07:31 View session

codeant-ai Bot added the size:XXL This PR changes 1000+ lines, ignoring generated files label Aug 8, 2026

codeant-ai Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread scripts/verify-pruned.ps1 Outdated

codeant-ai Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread crates/screenpipe-memory/src/sample.rs Outdated

Copilot AI reviewed Aug 8, 2026

View reviewed changes

Comment thread Cargo.toml Outdated

codeant-ai Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread crates/screenpipe-audio/src/capture.rs Outdated

qodo-code-review Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread .github/workflows/ci.yml

Comment thread crates/screenpipe-cli/src/audio_source.rs Outdated

Comment thread crates/screenpipe-cli/src/main.rs Outdated

chatgpt-codex-connector Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread crates/screenpipe-cli/src/service.rs

Comment thread crates/screenpipe-cli/src/main.rs Outdated

Comment thread crates/screenpipe-audio/src/transcribe.rs Outdated

Comment thread crates/screenpipe-cli/src/main.rs Outdated

Coldaine added 7 commits August 8, 2026 03:10

fix(memory): preserve observation sample compatibility

e77bee8

fix(audio): propagate non-timeout wait failures

9a01646

docs: clarify optional audio workspace surface

6f3f291

fix(ci): verify audio toolchain artifacts

bd151aa

fix(audio): harden failure and readiness boundaries

cd078db

test(audio): exercise terminal and segment call paths

443d367

test(audio): cover no-speech probability boundaries

762b085

Coldaine merged commit 8de573a into main Aug 8, 2026
6 checks passed

chatgpt-codex-connector Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread vendor/wasapi/src/api.rs

Comment thread .github/workflows/ci.yml

Comment thread crates/screenpipe-cli/src/main.rs

Comment thread crates/screenpipe-audio/src/capture.rs

Comment thread crates/screenpipe-cli/src/main.rs

Comment thread crates/screenpipe-audio/src/transcribe.rs

Coldaine mentioned this pull request Aug 8, 2026

fix(audio): close post-merge capture and shutdown gaps #6

Merged

Uh oh!

Conversation

Coldaine commented Aug 8, 2026 • edited by codeant-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

User description

Summary

Validation

Evidence boundaries

Documentation impact

CodeAnt-AI Description

What Changed

Impact

Checking Your Pull Request

Talking to CodeAnt AI

Example

Preserve Org Learnings with CodeAnt

Example

Retrigger review

Check Your Repository Health

Uh oh!

codeant-ai Bot commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 CodeAnt AI — Review Status

Uh oh!

codeant-ai Bot commented Aug 8, 2026

Thanks for using CodeAnt! 🎉

Uh oh!

Coldaine commented Aug 8, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

qodo-code-review Bot commented Aug 8, 2026

PR Summary by Qodo

Uh oh!

qodo-code-review Bot commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Coldaine commented Aug 8, 2026 •

edited by codeant-ai Bot

Loading

codeant-ai Bot commented Aug 8, 2026 •

edited

Loading

qodo-code-review Bot commented Aug 8, 2026 •

edited

Loading