Skip to content

feat: literature-informed piezo processor v2#215

Merged
ng merged 7 commits intodevfrom
feat/piezo-processor-v2
Mar 16, 2026
Merged

feat: literature-informed piezo processor v2#215
ng merged 7 commits intodevfrom
feat/piezo-processor-v2

Conversation

@ng
Copy link
Copy Markdown
Contributor

@ng ng commented Mar 16, 2026

Summary

  • Complete rewrite of the piezo signal processing pipeline using validated BCG techniques from published literature
  • Pump noise rejection via dual-channel energy gating with 5s guard period
  • Hysteresis presence detection with autocorrelation quality as secondary feature
  • Subharmonic summation autocorrelation for harmonic-proof HR extraction (0.8-8.5 Hz band)
  • Hilbert envelope breathing rate (fixes locked 12.0 BPM)
  • Sub-window autocorrelation HRV with Hampel IBI filtering

Pod validation (2026-03-16)

Metric V1 V2
Right HR (occupied) 61-411 BPM 78.7-82.6 BPM
Left false presence (empty) 100% 13%
HR harmonic errors 3/8 windows 0/12 windows
Breathing rate Locked 12.0 16-25 BPM

Live deployment confirmed: HR 78.7-81.3 BPM over 5 minutes of streaming data.

Deliverables

  • modules/piezo-processor/main.py — production v2 processor
  • modules/piezo-processor/test_main.py — 57 unit tests (all passing)
  • modules/piezo-processor/prototype_v2.py — offline comparison tool
  • docs/piezo-processor-v2.md — full documentation with Mermaid diagrams, citations, and validation results

Closes #214

Test plan

  • 57 unit tests pass locally (uv run --with numpy --with scipy --with pytest -- pytest)
  • Live pod deployment producing clean vitals (HR 78-81, BR 17-18, HRV 10-12 ms)
  • Overnight validation with both sides occupied
  • Pod 3/4 validation (thresholds calibrated on Pod 5)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Pump noise filtering with adaptive guard periods
    • Presence detection system for conditional vital sign reporting
  • Improvements

    • Enhanced heart rate tracking with inter-window consistency
    • Stricter heart rate variability validation threshold (200ms max)
    • Refined breathing rate and HRV calculations
  • Tests

    • Added comprehensive signal processing test suite
  • Documentation

    • Added piezo-processor v2 architecture and processing documentation

Replace the entire signal processing pipeline with validated BCG
techniques. Resolves pump noise contamination, harmonic locking,
false presence detection, and locked breathing rate.

Pipeline changes:
- Pump gating: dual-channel energy correlation + 5s guard (Shin 2009)
- Presence: hysteresis state machine + autocorr quality (Paalasmaa 2012)
- HR: 0.8-8.5 Hz SOS bandpass + subharmonic summation autocorrelation
  (Hermes 1988; Bruser 2011) + inter-window tracking
- BR: Hilbert envelope of cardiac band (PMC9354426)
- HRV: sub-window autocorrelation IBI + Hampel filter (PMC9305910)

Pod validation (2026-03-16, right side occupied):
- HR: 78.7-82.6 BPM (was 61-411 BPM in v1)
- BR: 16-25 BPM (was locked 12.0)
- Empty side false presence: 13% (was 100%)
- Harmonic errors: 0/12 windows (was 3/8)

Includes 57 unit tests and full documentation with citations.

Closes #214

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 16, 2026

Warning

Rate limit exceeded

@ng has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 21 minutes and 44 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3ecac0c9-ee0f-4951-aa3f-daaf02787e41

📥 Commits

Reviewing files that changed from the base of the PR and between 81362ae and 56b3ab8.

📒 Files selected for processing (3)
  • docs/piezo-processor.md
  • modules/piezo-processor/main.py
  • modules/piezo-processor/test_main.py
📝 Walkthrough

Walkthrough

Piezo Processor v2 introduces pump-aware signal processing with stream-based gating, filtered presence detection, and modular vital sign computation. Adds PumpGate, PresenceDetector, and HRTracker classes alongside bandpass filtering and autocorrelation-based HR/BR/HRV estimation. Includes tests and documentation. Tightens HRV validation threshold in calibration.

Changes

Cohort / File(s) Summary
Core Processor Refactor
modules/piezo-processor/main.py
Replaced single-pass processing with stream-based PumpGate gating and presence-based filtering with 5-second guard period. Added PumpGate, PresenceDetector, HRTracker classes. Introduced modular signal processing: _bandpass, _autocorr_quality, subharmonic_summation_hr, compute_breathing_rate, compute_hrv, _compute_autocorr. Updated SideProcessor and main loop to instantiate and use new components. Added module constants (PUMP_ENERGY_MULTIPLIER, PUMP_CORRELATION_MIN, PUMP_GUARD_S, HR_BAND).
Prototype & Tests
modules/piezo-processor/prototype_v2.py, modules/piezo-processor/test_main.py
Added complete prototype script (515 lines) implementing v2 architecture with CLI runner; comprehensive test suite (853 lines) validating bandpass filtering, pump gating, presence detection, autocorrelation, HR/BR/HRV computation across synthetic and realistic signals with edge-case coverage.
Documentation
docs/piezo-processor.md
New 404-line document detailing v2 architecture, processing pipelines, pump gating algorithm, presence state machine, HR/BR/HRV extraction methods, validation results, configuration constants, and literature references.
Validation Threshold
modules/common/calibration.py
Tightened HRV_HARD_MAX from 300.0 ms to 200.0 ms to reflect BCG-derived artifact threshold.

Sequence Diagram(s)

sequenceDiagram
    participant Main as Main Loop
    participant PumpGate as PumpGate
    participant SideProc as SideProcessor
    participant PresenceDetector as PresenceDetector
    participant SignalProc as Signal Processing<br/>(HR/BR/HRV)
    participant Output as Vitals Output

    Main->>PumpGate: check(left_chunk, right_chunk)
    PumpGate->>PumpGate: detect dual-channel spikes
    PumpGate->>Main: return gating mask
    
    Main->>SideProc: ingest gated records
    SideProc->>SideProc: buffer windowed data
    
    SideProc->>PresenceDetector: update(window_std, acr_qual)
    PresenceDetector->>PresenceDetector: hysteresis state check
    PresenceDetector->>SideProc: return presence state
    
    alt Presence Detected
        SideProc->>SignalProc: compute_hrv(filtered_samples)
        SignalProc->>SignalProc: sub-window autocorr + Hampel
        SignalProc->>SideProc: return HRV
        
        SideProc->>SignalProc: subharmonic_summation_hr(samples)
        SignalProc->>SignalProc: bandpass + autocorr + harmonic scoring
        SignalProc->>SideProc: return HR candidate
        
        SideProc->>SignalProc: compute_breathing_rate(samples)
        SignalProc->>SignalProc: Hilbert envelope + bandpass + peaks
        SignalProc->>SideProc: return BR
        
        SideProc->>Output: write vitals (HR/BR/HRV/presence)
    else No Presence
        SideProc->>Output: skip vitals write
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

🐰 Hops of joy for v2 streams,
Pump gates guard our signal dreams,
Filtered presence, bandbassed clean,
Heart and breath now brightly seen!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'literature-informed piezo processor v2' directly and clearly summarizes the main change: a v2 rewrite of the piezo processor using literature-validated techniques, matching the core objective.
Linked Issues check ✅ Passed The PR comprehensively addresses all requirements from issue #214: bandpass filtering applied before presence detection, presence gating prevents HR/HRV/BR processing during pump noise, pump-noise rejection via dual-channel gating with guard period, BR extraction fixed, and HRV validation tightened.
Out of Scope Changes check ✅ Passed All changes align with stated objectives: piezo processor v2 rewrite with pump gating, presence detection, HR/BR/HRV estimation, comprehensive tests, documentation, and HRV threshold tightening in calibration.py are all in-scope.
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/piezo-processor-v2
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

ng and others added 2 commits March 16, 2026 13:12
BCG-derived RMSSD >200 ms is virtually always artifact. Even elite
athletes during deep sleep rarely exceed 200 ms (Shaffer & Ginsberg
2017). The previous 400 ms ceiling allowed garbage values through.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Align with the piezo processor's range gate. BCG-derived RMSSD >200 ms
is artifact per Shaffer & Ginsberg 2017.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ng ng requested a review from Copilot March 16, 2026 20:30
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/piezo-processor-v2.md`:
- Around line 211-213: The fenced code block containing the formula score(L) =
1.0 * ACR(L) + 0.8 * ACR(L/2) + 0.6 * ACR(L/3) is missing a language tag and
triggers markdown lint MD040; update that fenced block around the formula (the
triple-backtick block that contains the score(L) expression) to include a
language identifier such as "text" (e.g., ```text) so the block becomes a
labeled code block and satisfies the linter/CI.
- Line 404: Update the SHS scoring resolution note by replacing the incorrect
adjacent-lag difference value "0.12 s" with the correct "0.002 s (2 ms)" in the
SHS limitation sentence (the line referencing "SHS scoring resolution" / the
sentence that starts "At 500 Hz, adjacent lags differ by ..."). Ensure the
surrounding text about BPM resolution and linear interpolation remains
consistent after the change.
- Line 365: The table cell containing the Hampel threshold expression ("3.0 *
1.4826 * MAD") is being parsed as emphasis; update the cell so the expression is
treated as literal text by either wrapping the expression in inline code
backticks (e.g., `3.0 * 1.4826 * MAD`) or escaping the asterisks (3.0 \* 1.4826
\* MAD) next to the existing "Hampel `threshold`" text to prevent MD037
formatting.

In `@modules/piezo-processor/main.py`:
- Line 456: The HRV sliding-window loop currently uses range(0, len(filtered) -
sub_window, sub_window // 2) which omits the final full sub_window; update the
range stop to include the last valid window (e.g., use len(filtered) -
sub_window + 1 as the stop) so the loop over variable start yields the final 30s
segment when iterating over filtered with step sub_window // 2 in main.py.
- Around line 549-550: The slice loop computing stds currently uses range(0,
len(filt) - w, w) which omits the final full-length window and can bias med_std;
change the range to include the last complete w-sized segment (e.g., range(0,
len(filt) - w + 1, w)) so stds covers every full 5-second window of filt before
computing med_std.
- Around line 161-164: Replace wall-clock time usage with a monotonic clock for
the pump guard: change the comment for self._pump_until to "monotonic seconds
when guard expires", replace time.time() with time.monotonic() in
is_pump_active(), and replace the time.time() call where the guard period is
assigned to self._pump_until (the assignment that sets the suppression expiry)
so all three places use time.monotonic().

In `@modules/piezo-processor/prototype_v2.py`:
- Line 71: The loop in PumpGate.build_mask currently uses range(0, n -
self.chunk_size, self.chunk_size) which skips the final full chunk at start == n
- self.chunk_size; change the range to include that last start index (for
example use range(0, n - self.chunk_size + 1, self.chunk_size) or an equivalent
<= check) so the final full-sized chunk is evaluated for pump contamination;
update references in PumpGate.build_mask to use this inclusive range and run
tests covering edge case where n is an exact multiple of chunk_size.
- Line 315: The sliding-window range currently uses range(0, len(filtered) -
sub_window, sub_window // 2) which omits the final full sub_window; change the
iteration so the last valid 30s window is included by iterating up to and
including the last start index (e.g., use len(filtered) - sub_window + 1 or
explicitly append a final start of len(filtered) - sub_window when needed).
Update the loop that references filtered and sub_window (the for start in
range(...) in prototype_v2.py) so the final full HRV sub-window is processed.
- Around line 450-451: The median-std calculation currently uses range(0,
len(filt)-w, w) which skips the last full window; update the range used when
building stds (in the block computing stds = [...] and med_std) to include the
final full sub-window by using range(0, len(filt)-w+1, w) (so the last window
starting at len(filt)-w is included), keeping variables filt, w, stds and
med_std unchanged otherwise.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 76b17762-1ba4-4bf8-a93a-279069c20cae

📥 Commits

Reviewing files that changed from the base of the PR and between b6eb7dd and f4556e3.

📒 Files selected for processing (4)
  • docs/piezo-processor-v2.md
  • modules/piezo-processor/main.py
  • modules/piezo-processor/prototype_v2.py
  • modules/piezo-processor/test_main.py

Comment thread docs/piezo-processor.md Outdated
Comment thread docs/piezo-processor.md Outdated
Comment thread docs/piezo-processor.md Outdated
Comment thread modules/piezo-processor/main.py Outdated
Comment thread modules/piezo-processor/main.py Outdated
Comment thread modules/piezo-processor/main.py Outdated
mask = np.ones(n, dtype=bool)
guard = int(self.GUARD_S * self.fs)

for i in range(0, n - self.chunk_size, self.chunk_size):
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Process the final chunk in PumpGate.build_mask().

Line 71 excludes start == n - chunk_size, so the last full chunk is never evaluated for pump contamination.

Proposed fix
-        for i in range(0, n - self.chunk_size, self.chunk_size):
+        for i in range(0, n - self.chunk_size + 1, self.chunk_size):
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for i in range(0, n - self.chunk_size, self.chunk_size):
for i in range(0, n - self.chunk_size + 1, self.chunk_size):
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@modules/piezo-processor/prototype_v2.py` at line 71, The loop in
PumpGate.build_mask currently uses range(0, n - self.chunk_size,
self.chunk_size) which skips the final full chunk at start == n -
self.chunk_size; change the range to include that last start index (for example
use range(0, n - self.chunk_size + 1, self.chunk_size) or an equivalent <=
check) so the final full-sized chunk is evaluated for pump contamination; update
references in PumpGate.build_mask to use this inclusive range and run tests
covering edge case where n is an exact multiple of chunk_size.

min_lag = int(fs * 60 / 150)
max_lag = int(fs * 60 / 40)

for start in range(0, len(filtered) - sub_window, sub_window // 2): # 50% overlap
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Include the last full HRV sub-window.

Current bounds skip the final valid 30s window (50% overlap loop).

Proposed fix
-        for start in range(0, len(filtered) - sub_window, sub_window // 2):  # 50% overlap
+        for start in range(0, len(filtered) - sub_window + 1, sub_window // 2):  # 50% overlap
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@modules/piezo-processor/prototype_v2.py` at line 315, The sliding-window
range currently uses range(0, len(filtered) - sub_window, sub_window // 2) which
omits the final full sub_window; change the iteration so the last valid 30s
window is included by iterating up to and including the last start index (e.g.,
use len(filtered) - sub_window + 1 or explicitly append a final start of
len(filtered) - sub_window when needed). Update the loop that references
filtered and sub_window (the for start in range(...) in prototype_v2.py) so the
final full HRV sub-window is processed.

Comment on lines +450 to +451
stds = [np.std(filt[j:j+w]) for j in range(0, len(filt)-w, w)]
med_std = float(np.median(stds)) if stds else 0
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Presence median-std calculation drops the last 5s segment.

The range stop omits the final full sub-window, biasing med_std.

Proposed fix
-            stds = [np.std(filt[j:j+w]) for j in range(0, len(filt)-w, w)]
+            stds = [np.std(filt[j:j+w]) for j in range(0, len(filt)-w+1, w)]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@modules/piezo-processor/prototype_v2.py` around lines 450 - 451, The
median-std calculation currently uses range(0, len(filt)-w, w) which skips the
last full window; update the range used when building stds (in the block
computing stds = [...] and med_std) to include the final full sub-window by
using range(0, len(filt)-w+1, w) (so the last window starting at len(filt)-w is
included), keeping variables filt, w, stds and med_std unchanged otherwise.

ng and others added 3 commits March 16, 2026 13:31
This is the canonical piezo processor doc, not a versioned supplement.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This is the canonical piezo processor doc, not a versioned supplement.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Rewrites the SleepyPod piezo signal processing pipeline (presence/HR/BR/HRV) to a literature-informed v2 approach with pump-noise rejection, harmonic-robust HR extraction, improved BR, and more robust HRV—plus accompanying documentation and unit tests.

Changes:

  • Replaces v1 HeartPy-based processing with v2: PumpGate + hysteresis presence + SHS autocorrelation HR + Hilbert-envelope BR + sub-window autocorr HRV.
  • Adds an offline prototype_v2.py runner and a comprehensive test_main.py suite for the v2 signal processing functions.
  • Updates HRV validation hard cap and adds detailed documentation for the new pipeline and parameters.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
modules/piezo-processor/main.py Production v2 processing pipeline (pump gating, presence, HR/BR/HRV) and streaming ingestion changes.
modules/piezo-processor/test_main.py New unit tests for the v2 signal-processing primitives and state machines.
modules/piezo-processor/prototype_v2.py On-pod/offline prototype tool to evaluate v2 processing on RAW recordings.
modules/common/calibration.py Tightens HRV validator maximum to 200 ms to treat higher values as artifacts.
docs/piezo-processor-v2.md Full v2 design/parameter documentation with diagrams, rationale, and validation results.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread modules/piezo-processor/main.py Outdated
Comment on lines +456 to +461
for start in range(0, len(filtered) - sub_window, sub_window // 2):
chunk = filtered[start:start + sub_window]
acr = _compute_autocorr(chunk, fs)
if acr is None:
continue
search = acr[min_lag:min(max_lag, len(acr))]
Comment thread modules/piezo-processor/main.py Outdated
# --- HRV (sub-window autocorrelation IBI) ---
hrv_arr = np.array(self._hrv_buf)
hrv = compute_hrv(hrv_arr) if len(hrv_arr) >= int(
60 * SAMPLE_RATE) else None
Comment on lines +18 to +40
import sys

_stubs = {
"cbor2": type(sys)("cbor2"),
"common": type(sys)("common"),
"common.raw_follower": type(sys)("common.raw_follower"),
}
_stubs["common.raw_follower"].RawFileFollower = None
sys.modules.update(_stubs)

from main import ( # noqa: E402
_bandpass,
_autocorr_quality,
_compute_autocorr,
compute_breathing_rate,
compute_hrv,
subharmonic_summation_hr,
HRTracker,
PresenceDetector,
PumpGate,
SAMPLE_RATE,
PUMP_GUARD_S,
)
mask = np.ones(n, dtype=bool)
guard = int(self.GUARD_S * self.fs)

for i in range(0, n - self.chunk_size, self.chunk_size):
Comment on lines +185 to +199
if self._baseline is not None and self._baseline > 0:
spike = max(le, re) / self._baseline
if ratio > PUMP_CORRELATION_MIN and spike > PUMP_ENERGY_MULTIPLIER:
# Pump detected — set guard period
self._pump_until = time.time() + PUMP_GUARD_S
log.debug("Pump detected (spike=%.1f, ratio=%.2f), "
"guard until +%.0fs", spike, ratio, PUMP_GUARD_S)
return True

def is_present(signal: np.ndarray) -> bool:
return float(np.ptp(signal)) > PRESENCE_THRESHOLD
# Update baseline with exponential moving average (only on clean data)
avg = (le + re) / 2
if self._baseline is None:
self._baseline = avg
elif avg < self._baseline * 3:
self._baseline = 0.95 * self._baseline + 0.05 * avg
Comment thread modules/piezo-processor/main.py Outdated
filt = _bandpass(hr_arr, 1.0, 10.0, SAMPLE_RATE)
w = int(5 * SAMPLE_RATE)
stds = [np.std(filt[j:j + w])
for j in range(0, len(filt) - w, w)]
- Use time.monotonic() for pump guard (avoid wall-clock drift)
- Fix pump baseline initialization vulnerability on startup during pump
- Fix off-by-one in presence std window loop (include final segment)
- Fix off-by-one in HRV sub-window iteration (include last 30s window)
- Fix HRV acr slice to include max_lag endpoint
- Gate HRV on HRV_WINDOW_S (300s) not hardcoded 60s
- Fix docs: add code fence language tag, escape Hampel expression,
  correct SHS lag resolution (0.002s not 0.12s), remove \n in mermaid

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ng ng merged commit 25c8bce into dev Mar 16, 2026
1 of 5 checks passed
@ng ng deleted the feat/piezo-processor-v2 branch March 16, 2026 20:40
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 1.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Piezo presence detection runs on unfiltered signal — pump noise triggers false presence

2 participants