Skip to content

Commit 60d7277

Browse files
cutterblcutterbl
committed
feat(audio-worklet): Add phase vocoder packages
Adds two new packages: - @soundtouchjs/stretch-phase-vocoder: FFT-based StretchPipe implementation using radix-2 Cooley-Tukey FFT, Hann-windowed overlap-add synthesis, and per-bin phase accumulation. Includes createPhaseVocoderFactory for drop-in use with SoundTouchOptions.stretchFactory. - @soundtouchjs/phase-vocoder-worklet: AudioWorklet integration wrapping PhaseVocoderNode (mirrors SoundTouchNode API) and PhaseVocoderProcessor (uses SoundTouch with the phase vocoder stretchFactory). Adds fftSize and overlapFactor constructor options. Full metrics observability support.
1 parent e3341fc commit 60d7277

35 files changed

Lines changed: 3225 additions & 1 deletion

‎nx.json‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,9 @@
102102
"@soundtouchjs/interpolation-strategy-linear",
103103
"@soundtouchjs/interpolation-strategy-hann",
104104
"@soundtouchjs/interpolation-strategy-blackman",
105-
"@soundtouchjs/interpolation-strategy-kaiser"
105+
"@soundtouchjs/interpolation-strategy-kaiser",
106+
"@soundtouchjs/stretch-phase-vocoder",
107+
"@soundtouchjs/phase-vocoder-worklet"
106108
],
107109
"changelog": {
108110
"automaticFromRef": false,

‎packages/phase-vocoder-worklet/README.md‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# @soundtouchjs/phase-vocoder-worklet
2+
3+
An `AudioWorklet` integration that uses the phase vocoder time-stretch algorithm from `@soundtouchjs/stretch-phase-vocoder`. Provides `PhaseVocoderNode` as a drop-in replacement for `SoundTouchNode` when smoother time-stretching at extreme ratios is required.
4+
5+
## Installation
6+
7+
```sh
8+
npm install @soundtouchjs/phase-vocoder-worklet @soundtouchjs/stretch-phase-vocoder
9+
```
10+
11+
## When to use this package
12+
13+
- Playback at extreme ratios (< 0.5× or > 2×) where the default WSOLA algorithm produces audible artifacts.
14+
- Smooth slow-motion or fast-forward effects where "phasiness" is acceptable.
15+
- You want the same `AudioWorkletNode` API as `SoundTouchNode` with a different internal stretch algorithm.
16+
17+
For moderate ratios (0.5–2×) the default `@soundtouchjs/audio-worklet` typically sounds better.
18+
19+
## Setup
20+
21+
### Resolving `processorUrl`
22+
23+
#### Vite
24+
25+
```ts
26+
import processorUrl from '@soundtouchjs/phase-vocoder-worklet/processor?url';
27+
```
28+
29+
#### webpack 5
30+
31+
```ts
32+
const processorUrl = new URL(
33+
'@soundtouchjs/phase-vocoder-worklet/processor',
34+
import.meta.url,
35+
).href;
36+
```
37+
38+
#### Static / CDN
39+
40+
Copy `.dist/phase-vocoder-processor.js` to your public directory and reference it by path:
41+
42+
```ts
43+
const processorUrl = '/phase-vocoder-processor.js';
44+
```
45+
46+
## Usage
47+
48+
```ts
49+
import { PhaseVocoderNode } from '@soundtouchjs/phase-vocoder-worklet';
50+
51+
const audioCtx = new AudioContext();
52+
await PhaseVocoderNode.register(audioCtx, processorUrl);
53+
54+
const node = new PhaseVocoderNode({
55+
context: audioCtx,
56+
fftSize: 2048, // optional, default 2048
57+
overlapFactor: 4, // optional, default 4
58+
});
59+
60+
node.pitch.value = 1.5; // pitch up 50 %
61+
node.pitchSemitones.value = 3; // or shift by semitones
62+
node.playbackRate.value = 0.5; // slow down 2×
63+
64+
node.connect(audioCtx.destination);
65+
66+
// Connect your source:
67+
sourceNode.connect(node);
68+
```
69+
70+
## API
71+
72+
### `PhaseVocoderNode`
73+
74+
Extends `AudioWorkletNode`. Same API as `SoundTouchNode` with additional `fftSize` / `overlapFactor` options.
75+
76+
#### Static methods
77+
78+
| Method | Description |
79+
|--------|-------------|
80+
| `PhaseVocoderNode.register(context, processorUrl)` | Registers the processor module. Must be called before constructing nodes. |
81+
| `PhaseVocoderNode.registerStrategyModule(context, moduleUrl)` | Loads an interpolation strategy plugin into worklet scope. |
82+
| `PhaseVocoderNode.processorName` | The registered processor identifier string. |
83+
84+
#### Constructor options
85+
86+
| Option | Type | Default | Description |
87+
|--------|------|---------|-------------|
88+
| `context` | `BaseAudioContext` | required | AudioContext or OfflineAudioContext |
89+
| `fftSize` | `512 \| 1024 \| 2048 \| 4096` | `2048` | FFT frame size — larger = better frequency resolution, higher latency |
90+
| `overlapFactor` | `2 \| 4 \| 8` | `4` | Overlap factor — higher = smoother output, more computation |
91+
| `outputChannelCount` | `1 \| 2` | `2` | Output channel count (set to `1` for mono destinations) |
92+
| `sampleBufferType` | `'circular' \| 'fifo'` | `'circular'` | Internal buffer strategy |
93+
| `interpolationStrategy` | `RateTransposerInterpolationStrategy` | `'lanczos'` | Initial rate-transposer interpolation strategy |
94+
95+
#### AudioParams
96+
97+
| Param | Default | Range | Description |
98+
|-------|---------|-------|-------------|
99+
| `pitch` | `1.0` | `0.1–8.0` | Pitch multiplier (k-rate) |
100+
| `pitchSemitones` | `0` | `-24–24` | Pitch shift in semitones, combined with `pitch` (k-rate) |
101+
| `playbackRate` | `1.0` | `0.1–8.0` | Playback rate multiplier — set this to match the source node's `playbackRate` (k-rate) |
102+
103+
#### Methods
104+
105+
| Method | Description |
106+
|--------|-------------|
107+
| `setInterpolationStrategy(strategy)` | Switches interpolation strategy at runtime. |
108+
| `setInterpolationStrategyParams(params)` | Updates parameters for the active strategy. |
109+
| `setStretchParameters(params)` | No-op for the phase vocoder (accepted for API parity with `SoundTouchNode`). |
110+
111+
#### Processor observability
112+
113+
The processor posts metrics every 100 render blocks. Access them via the `metrics` getter or the `metrics` CustomEvent:
114+
115+
```ts
116+
// Getter
117+
const m = node.metrics; // ProcessorMetrics | null
118+
119+
// Event
120+
node.addEventListener('metrics', (e) => {
121+
const { framesBuffered, underrunCount, blockCount } = (e as CustomEvent<ProcessorMetrics>).detail;
122+
console.log('underruns:', underrunCount);
123+
});
124+
```
125+
126+
`ProcessorMetrics` shape:
127+
128+
| Field | Description |
129+
|-------|-------------|
130+
| `framesBuffered` | Output frames available at the last render block |
131+
| `underrunCount` | Cumulative render blocks with fewer output frames than requested |
132+
| `blockCount` | Total render blocks processed |
133+
| `timestamp` | `performance.now()` when the metrics arrived on the main thread |
134+
135+
## Parameters
136+
137+
| Option | Default | Description |
138+
|--------|---------|-------------|
139+
| `fftSize` | 2048 | FFT frame size — larger gives better frequency resolution but higher latency (`fftSize` samples) |
140+
| `overlapFactor` | 4 | Overlap factor — `4` = 75 % overlap (good quality); `8` = 87.5 % overlap (smoother, 2× cost) |
141+
142+
## Trade-offs vs `SoundTouchNode`
143+
144+
| | `SoundTouchNode` (WSOLA) | `PhaseVocoderNode` |
145+
|--|--|--|
146+
| Quality at extreme ratios | Artifacts above 2× | Smooth at all ratios |
147+
| Transient preservation | Better (time-domain) | Worse (frequency smearing) |
148+
| Computation | Lower | Higher (FFT per hop) |
149+
| Startup latency | Lower | `fftSize` samples |
150+
| Artifacts | Clicks / repeats | "Phasiness" / smearing |
151+
152+
## License
153+
154+
MPL-2.0 — see [LICENSE](../../LICENSE) for details.

‎packages/phase-vocoder-worklet/package.json‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
{
2+
"name": "@soundtouchjs/phase-vocoder-worklet",
3+
"version": "1.0.0",
4+
"description": "AudioWorklet integration for the SoundTouchJS phase vocoder time-stretch algorithm",
5+
"license": "MPL-2.0",
6+
"licenseFile": "LICENSE",
7+
"type": "module",
8+
"sideEffects": false,
9+
"exports": {
10+
"./package.json": "./package.json",
11+
".": {
12+
"types": "./.dist/index.d.ts",
13+
"default": "./.dist/index.js"
14+
},
15+
"./processor": "./.dist/phase-vocoder-processor.js"
16+
},
17+
"main": "./.dist/index.js",
18+
"types": "./.dist/index.d.ts",
19+
"files": [
20+
".dist",
21+
"README.md",
22+
"LICENSE"
23+
],
24+
"keywords": [
25+
"audio",
26+
"WebAudio",
27+
"AudioWorklet",
28+
"TypeScript",
29+
"pitch",
30+
"tempo",
31+
"soundtouch",
32+
"phase-vocoder",
33+
"fft"
34+
],
35+
"author": "Steve 'Cutter' Blades",
36+
"contributors": [
37+
"Steve 'Cutter' Blades <web.admin@cutterscrossing.com> (https://cutterscrossing.com/)"
38+
],
39+
"repository": {
40+
"type": "git",
41+
"url": "git+https://github.com/cutterbl/SoundTouchJS.git",
42+
"directory": "packages/phase-vocoder-worklet"
43+
},
44+
"bugs": {
45+
"url": "https://github.com/cutterbl/SoundTouchJS/issues"
46+
},
47+
"homepage": "https://github.com/cutterbl/SoundTouchJS",
48+
"dependencies": {
49+
"@soundtouchjs/core": "workspace:*",
50+
"@soundtouchjs/stretch-phase-vocoder": "workspace:*"
51+
},
52+
"publishConfig": {
53+
"access": "public"
54+
}
55+
}

‎packages/phase-vocoder-worklet/project.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"name": "@soundtouchjs/phase-vocoder-worklet",
3+
"targets": {
4+
"build": {
5+
"executor": "nx:run-commands",
6+
"dependsOn": ["^build"],
7+
"inputs": ["production", "^production"],
8+
"outputs": ["{projectRoot}/.dist"],
9+
"cache": true,
10+
"options": {
11+
"cwd": "{projectRoot}",
12+
"commands": ["tsc --build tsconfig.lib.json", "vite build"],
13+
"parallel": false
14+
}
15+
}
16+
}
17+
}

0 commit comments

Comments
 (0)