Voice Lens

Acoustic metrics for speech: arousal and expressiveness, computed entirely on-device from pitch, loudness, and rhythm. No model, no network, no upload — just a small dependency-free DSP library.

A tool by Inworld Voice Lab.

---

What it measures

Two scores, each normalised to 0..1:

Score	What it captures	How it is built
arousal	energetic vs. flat delivery	mean of loudness, pitch variation, speaking rate
expressiveness	dynamic vs. monotone delivery	mean of pitch range, loudness variation, pitch variation

Under the hood the library runs a faithful, pure-JS DSP pipeline on a 16 kHz mono mixdown of each clip:

• Pitch (F0) — probabilistic YIN (CMNDF → beta/Boltzmann trough probabilities → banded Viterbi)
• Rhythm — mel-power onset-strength flux with a peak-picking state machine
• Loudness — framewise RMS energy

---

Repository layout

.
├── audio-metrics.js   # the library — zero dependencies, single file
├── demo.html          # browser demo: pick a file, see the scores
├── demo-node.js       # Node demo: score a file from the CLI
├── demo.mp3           # sample clip used by both demos
└── README.md

---

Installation

There is no package to install — the library is a single dependency-free file. Copy audio-metrics.js into your project, then:

Browser — load it with a classic <script> tag; it attaches a global AudioMetrics:

<script src="audio-metrics.js"></script>

Node / bundlers — it also sets module.exports:

const AudioMetrics = require("./audio-metrics.js");

Everything operates on Float32Array sample buffers. The decode/resample helpers (analyzeFile, decodeAudioFile, resampleTo16k) use the Web Audio API and are browser-only; the analysis itself (analyzeExpressiveness and all low-level DSP) is pure JS and runs anywhere.

---

Demo 1 — browser (demo.html)

Serve the repo folder over http and open the page:

npx serve            # or: python3 -m http.server 8000
# then open http://localhost:3000/demo.html

Pick any wav/mp3 with the file button (this also works over file://), or click Analyze bundled demo.mp3 (this one needs the http server). The page decodes the file, resamples to 16 kHz, runs the analysis, and shows the two headline scores plus the underlying features.

The entire integration is one call:

<input type="file" id="file" accept="audio/*">
<script src="audio-metrics.js"></script>
<script>
  document.getElementById("file").addEventListener("change", async (e) => {
    const r = await AudioMetrics.analyzeFile(e.target.files[0]);   // decode → 16 kHz → analyze
    console.log(`arousal ${r.arousal.toFixed(3)} | expressiveness ${r.expressiveness.toFixed(3)} | f0 ${r.f0mean.toFixed(0)} Hz`);
  });
</script>

Demo 2 — Node (demo-node.js)

Node has no built-in audio decoder, so the demo shells out to ffmpeg (brew install ffmpeg / apt install ffmpeg) to turn any audio format into 16 kHz mono float32 samples, then runs the pure-JS analysis:

node demo-node.js                 # scores the bundled demo.mp3
node demo-node.js path/to/clip.wav

demo.mp3 — 3.38 s @ 16 kHz

  arousal                     0.451  █████████░░░░░░░░░░░
  expressiveness              0.726  ███████████████░░░░░

  f0 mean ± std               211.6 ± 91.5 Hz
  ...

The core of it — decode with ffmpeg, feed samples to analyzeExpressiveness:

const { spawnSync } = require("child_process");
const AudioMetrics = require("./audio-metrics.js");

const ff = spawnSync("ffmpeg", [
  "-v", "error", "-i", "clip.mp3",
  "-f", "f32le", "-acodec", "pcm_f32le", "-ac", "1", "-ar", "16000",
  "pipe:1",
], { maxBuffer: 1 << 30 });

const raw = ff.stdout;
const samples = new Float32Array(raw.buffer.slice(raw.byteOffset, raw.byteOffset + raw.length));

const r = AudioMetrics.analyzeExpressiveness(samples, 16000);
console.log(r.arousal, r.expressiveness, r.f0mean);

No ffmpeg? Any path to 16 kHz mono Float32 samples works — e.g. read a 16-bit PCM WAV yourself and divide by 32768.

---

API reference

Function	Runs in	Returns
analyzeFile(fileOrBlob)	browser	{ sr, dur, ...scores } — decode → resample 16 kHz → analyze
analyzeExpressiveness(samples, sr = 16000)	browser + node	{ arousal, expressiveness, ...features } (pure DSP; expects 16 kHz mono)
decodeAudioFile(fileOrBlob)	browser	{ mono: Float32Array, sr, dur }
resampleTo16k(mono, sr)	browser	Float32Array at 16 kHz
rmsTrack(x)	browser + node	Float64Array — per-frame RMS energy
pyinF0(x)	browser + node	Float64Array — F0 in Hz, NaN where unvoiced
onsetEnvelope(x)	browser + node	Float64Array — onset strength per frame
peakPick(x)	browser + node	number[] — onset peak frame indices
TARGET_SR	—	16000

analyzeExpressiveness / analyzeFile result fields:

Field	Meaning
arousal, expressiveness	the two headline scores, 0..1
f0mean, f0std, f0min, f0max	pitch statistics over voiced frames (Hz)
pdq	pitch dynamic quotient, (f0max - f0min) / f0mean
f0cv	pitch coefficient of variation, f0std / f0mean
rmsMean	mean loudness
intensityCV	loudness coefficient of variation
rate	onset peaks per second
nVoiced, nFrames, nPeaks	frame/peak counts (diagnostics)
sr, dur	native sample rate (Hz) and duration (s) — analyzeFile only

Score definitions

energy = min(rmsMean * 10, 1)
pitch  = min(f0cv, 1)
tempo  = min(rate / 10, 1)

arousal        = (energy + pitch + tempo) / 3
expressiveness = (min(pdq, 1) + min(intensityCV, 1) + pitch) / 3

---

Notes & limitations

• Analysis runs at 16 kHz mono. analyzeExpressiveness assumes its input is at 16 kHz; resample before calling it in Node (the browser path does this for you).
• Pure DSP, no ML. The scores are deterministic; a 220 Hz tone yields arousal ≈ 0.334, f0 ≈ 220 Hz.
• The bundled-demo button needs a server. Fetching demo.mp3 over file:// is blocked by browsers; the file-picker path works straight from disk.

---

Credits

Built by Inworld Voice Lab. Released under the MIT License.