Skip to content

fix(sound): share pan sampling during playback - #869

Merged
martin-henz merged 3 commits into
masterfrom
replace-pr-838
Aug 3, 2026
Merged

fix(sound): share pan sampling during playback#869
martin-henz merged 3 commits into
masterfrom
replace-pr-838

Conversation

@martin-henz

Copy link
Copy Markdown
Member

Re-opens #838, which was automatically closed (not merged) when its base branch conductor-migration was deleted after being merged into master via #680. This branch cherry-picks the original commits from @11suixing11 unchanged, onto current master.

Summary

  • sample pan outputs once per timestamp and derive both PCM channels from that shared sample
  • sample both the source and pan modulator once per timestamp for pan_mod
  • preserve the internal shared stereo sampler when a Sound crosses the Conductor pair/array boundary, keyed by evaluator, stable wave closure IDs, and duration
  • keep the public Sound representation and left/right Wave accessors unchanged

Why the boundary handling matters

The pure functions.ts optimisation alone is lost in normal Source usage: a transformed Sound is encoded as [[left_wave, right_wave], duration] and decoded again by play. TypedValue and pair wrappers are recreated during the Python round-trip, so the sampler metadata is retained using evaluator-scoped closure IDs instead of object identity. A modified Sound does not match the cache and falls back to the existing per-wave sampling path.

Testing (from original PR, re-verify on CI)

  • counted generator-backed waves verify one source sample per timestamp for play(pan(...))
  • counted source and modulator waves verify one sample each per timestamp for play(pan_mod(...))
  • Conductor adapter tests verify fresh closure wrappers restore metadata without leaking across durations, closure pairs, or evaluators
  • yarn workspace @sourceacademy/bundle-sound test (68 passed)
  • yarn workspace @sourceacademy/bundle-sound tsc
  • yarn workspace @sourceacademy/bundle-sound lint

Fixes #800

Original author: @11suixing11

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@martin-henz

Copy link
Copy Markdown
Member Author

Reviewed the diff and checked out the branch to run the suite directly. This is a solid, well-motivated fix with good test coverage — one design question worth a second pair of eyes, no blockers.

The problem, as I traced it: pan/pan_mod build left/right channel Waves that both close over the same source wave (gainWave(wave, leftGain) and gainWave(wave, rightGain) in pan, similarly for pan_mod's amount/source waves). play() used to sample the left and right Waves independently via two separate sampleWave() calls, so the shared source wave got invoked twice per timestamp. For a pure deterministic wave that's just wasted compute, but per the comment in functions.ts, a Source-level student-defined wave threads all the way up through the CSE machine on each call — so this was doubling real interpreter work, and for any wave with actual side effects or non-determinism it would silently produce diverging left/right channels. The new countedWave tests demonstrate this directly (asserting the source is sampled exactly once per timestamp for both pan and pan_mod).

Why the boundary-crossing half is necessary, not just the pure fix: I confirmed this isn't hypothetical — under Conductor, pan(amount)(sound) and play(...) are both @moduleMethod-decorated (confirmed via index.ts:446+), so every real Source-level call like play(pan(0.5)(some_sound)) marshals the intermediate Sound out to [[left_wave, right_wave], duration] and back through conductorToSound/soundToConductor — exactly the round trip described in the PR body. So without conductorAdapters.ts, the functions.ts-level fix alone would be silently lost the moment a student actually calls pan from Source code, which is the normal case. Good catch, and the PR doesn't stop at the easy half of the fix.

conductorAdapters.ts cache design: WeakMap<evaluator, Map<leftClosureId, Map<rightClosureId, Map<duration, sampler>>>>, keyed by the closures' numeric Identifier (confirmed ClosureIdentifier is a branded number in @sourceacademy/conductor's types, so it's a stable, valid Map key — not object identity, which the comment correctly notes wouldn't survive the round trip). Scoping by evaluator via WeakMap means the whole cache is naturally cleaned up when a run's evaluator is GC'd. The "why this file is separate from index.ts" rationale (Vitest's transform doesn't handle this bundle's experimentalDecorators-based @moduleMethod decorators) checks out — index.ts does use @moduleMethod(...) decorators, so keeping the undecorated cache logic testable in isolation is a reasonable call, not incidental churn.

One design question, not a blocker: the cache keys on closure IDs + duration, scoped to one evaluator/run, with no explicit invalidation — entries just accumulate for the run's lifetime and are matched by (leftId, rightId, duration). That's fine if a given evaluator never reuses a closure ID for an unrelated closure within the same run. I couldn't verify that guarantee from this repo (the ID allocator lives in js-slang/py-slang, not here) — if IDs are ever recycled after GC within a run, a stale sampler could theoretically get matched to an unrelated Sound that happens to land on the same (id, id, duration) tuple. Worth a quick confirmation from whoever owns the evaluator-side ID allocation, but I don't think it should hold up this PR — the existing conductorAdapters.test.ts already covers the cases that matter for this bundle's own contract (no leakage across duration, closure pair, or evaluator).

Verified locally (checked out replace-pr-838):

  • yarn workspace @sourceacademy/bundle-sound test — 68/68 passing, matches the PR's claim
  • yarn workspace @sourceacademy/bundle-sound tsc — clean
  • yarn workspace @sourceacademy/bundle-sound lint — same 6 pre-existing warnings present on master too (an environment-dependent @sourceacademy/throw-runtime-error rule that's off outside CI), nothing new from this PR

Minor nit, not worth blocking on: in both pan and pan_mod, the SoundSampler closures are written as duration => samplePannedChannels(wave, duration, clamped), where the parameter shadows the outer duration already captured by the closure. Since sampleSound() always invokes sound.sampleChannels(sound.duration) and sound.duration is set to that same outer duration by make_stereo_sound_with_sampler, the parameter is always redundant with the closed-over value — harmless, just a bit of unnecessary indirection.

No changes requested from me — the core fix is correct, well-tested, and the harder boundary-crossing half is handled thoughtfully.

@martin-henz

Copy link
Copy Markdown
Member Author

Good work @11suixing11 . Thanks for contributing.

@martin-henz
martin-henz merged commit 0e30c51 into master Aug 3, 2026
14 checks passed
@martin-henz
martin-henz deleted the replace-pr-838 branch August 3, 2026 06:26
martin-henz added a commit that referenced this pull request Aug 3, 2026
- __ensureAudioContext() now treats a closed AudioContext as absent
  rather than reusing it: destroy() closes the context but never
  resets the field, so a playSamples() call still in flight when a
  Run ends (e.g. sampling was still running) could otherwise be
  handed back an already-closed, unusable context.
- Cap __players to the most recent MAX_PLAYER_BARS entries so a loop
  calling play_in_tab() many times in one Run can't grow the list
  (each holding a full WAV data URI plus a rendered <audio> element)
  without bound for the tab's lifetime.
- Associate each play bar's "Sound N" label with its <audio> control
  via aria-labelledby, so the control's accessible name matches what
  is shown on screen.

Also updates play_in_tab() to use the shared sampleSound() helper
(added by the just-merged #869) instead of directly sampling
leftWave/rightWave, so it gets the same "sample the source once"
fix play() already has for panned sounds - this one only surfaced
once resolving this PR's conflict with master pulled sampleSound()
in.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Dv6WCgTvMVbCJmyRUY9pL
martin-henz added a commit that referenced this pull request Aug 4, 2026
* fix(sound): play concurrently, add back play_in_tab (#841)

The Conductor migration accidentally serialized play()/play_wave()
(each call now waited for the previous one to finish) and dropped
play_in_tab() (and its tab UI) entirely, regressing two behaviours
the sound module used to have: genuinely overlapping playback, and a
tab showing per-sound play bars.

- Removed the host-side playback queue in SoundTabPlugin so repeated/
  looped play()/play_wave() calls start immediately and overlap,
  mixed by the shared AudioContext, instead of playing one after
  another.
- Brought back play_in_tab(): it samples the Sound and encodes it as
  a WAV data URI (pure computation, done module-side rather than via
  a host round trip), which the sound tab renders as a native
  <audio controls> play bar. Each call adds a new bar, stacked
  vertically below any earlier ones, per the clarification on #841.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3LRoXRMQ5ADeYMdVTChCZ

* fix(sound): address CodeRabbit findings on play_in_tab tab UI

- __ensureAudioContext() now treats a closed AudioContext as absent
  rather than reusing it: destroy() closes the context but never
  resets the field, so a playSamples() call still in flight when a
  Run ends (e.g. sampling was still running) could otherwise be
  handed back an already-closed, unusable context.
- Cap __players to the most recent MAX_PLAYER_BARS entries so a loop
  calling play_in_tab() many times in one Run can't grow the list
  (each holding a full WAV data URI plus a rendered <audio> element)
  without bound for the tab's lifetime.
- Associate each play bar's "Sound N" label with its <audio> control
  via aria-labelledby, so the control's accessible name matches what
  is shown on screen.

Also updates play_in_tab() to use the shared sampleSound() helper
(added by the just-merged #869) instead of directly sampling
leftWave/rightWave, so it gets the same "sample the source once"
fix play() already has for panned sounds - this one only surfaced
once resolving this PR's conflict with master pulled sampleSound()
in.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Dv6WCgTvMVbCJmyRUY9pL

* fix(sound): address Akshay's review on play_in_tab

- play_in_tab() now calls notifyConstructing() before sampling, like
  play() already does, so a long Sound shows "Constructing…" instead
  of looking stalled until the bar appears. addPlayerToTab() is the
  corresponding "done" signal (decrementing __constructingCount and
  recomputing status), matching how playSamples() does it for play().
- Deduplicated play()/play_in_tab()'s near-identical Sound argument
  validation into a shared assertPlayableSound() helper.
- Added browser-mode rendering tests for PlayerBarsView (the actual
  "multiple bars stacked vertically" UI the issue asked for), which
  previously only had coverage at the addPlayerToTab() data layer.
  Exported PlayerBarsView so it's directly renderable in tests, and
  brought in the same vitest-browser-react/@vitest/browser-playwright
  setup already used by the Rune/Matrix/Curve tabs.

The player-list cap Akshay's review also flagged (point 1) was
already fixed in the previous commit before the review landed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Dv6WCgTvMVbCJmyRUY9pL

* fix(sound): complete the play_in_tab review fixes

The previous commit's git add silently dropped these files (one bad
pathspec in a multi-path invocation aborts the whole add) - this is
the rest of that change:
- assertPlayableSound() dedup in functions.ts, and play_in_tab()'s
  new notifyConstructing() call
- addPlayerToTab()'s matching __constructingCount decrement
- the PlayerBarsView rendering tests and their vitest-browser-react/
  browser-playwright devDependencies
- 2 new bundle-level tests (notifyConstructing ordering) and 1 new
  tab-level test (constructing status closes out via addPlayerToTab)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Dv6WCgTvMVbCJmyRUY9pL

* fix(sound): show a placeholder instead of silently dropping zero-duration play_in_tab() calls

A zero-duration Sound is a valid neutral element (for consecutively()/
simultaneously(), e.g. as a reduce() starting value), not an error -
but play_in_tab() previously just silently added nothing to the tab
for one, which could read as the call having been dropped.

Adds a new addZeroDurationPlayerToTab() RPC method (deliberately
separate from addPlayerToTab(), not a variant of it): no sampling
happens for a zero-duration Sound, so play_in_tab() never calls
notifyConstructing() for one either, and folding this into
addPlayerToTab() would incorrectly decrement __constructingCount on
behalf of a call that never incremented it - which could cancel out
an unrelated, genuinely concurrent play_in_tab() call's still-in-
flight notifyConstructing().

PlayerBarEntry becomes a discriminated union so PlayerBarsView can
render a plain "zero duration sound" placeholder line in the entry's
call-order position, instead of a native <audio> control.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Dv6WCgTvMVbCJmyRUY9pL

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Akshay <131676168+Akshay-2007-1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sound: pan/pan_mod sample the shared source/modulator wave twice per channel instead of once

2 participants