Skip to content
BranchesTags

Repository files navigation

react-native-audio-intelligence

Native audio analysis for React Native via Nitro Modules. Decodes WAV files on-device, extracts DSP features (RMS, transient density, windowing), detects content heuristics, scores prominence, and selects highlight segments — all without leaving the JS thread blocked.

Features

  • WAV decoding — iOS via AVFoundation; Android via a built-in 16-bit PCM parser
  • Overlapping window analysis — 1.5 s windows with 0.75 s hop
  • Transient density — per-window sharp energy-change detection (transients/s)
  • Event detection — heuristic labels: speech, music, impact, ambient, silence
  • Prominence scoring — weighted score in [0, 1] with quality penalties
  • Highlight selection — up to 3 non-overlapping peak windows
  • Synchronous version info — getVersion() and getPlatformInfo() run on the native thread

Requirements

Installation

npm install react-native-audio-intelligence react-native-nitro-modules
# or
yarn add react-native-audio-intelligence react-native-nitro-modules

Then rebuild the native app:

npx pod-install   # iOS
npx react-native run-ios
npx react-native run-android

Quick start

import AudioIntelligence from 'react-native-audio-intelligence'

const version = AudioIntelligence.getVersion()
// => "1.0.0"

const result = await AudioIntelligence.analyzeAudioFile('/path/to/audio.wav')
console.log(result.score) // 0.0 – 1.0 prominence
console.log(result.detectedEvents) // [{ label: "speech", confidence: 0.47 }, ...]
console.log(result.windows.length) // overlapping analysis windows

The file path must be an absolute path readable by native code. React Native asset bundles are not supported directly — copy or record to a temp path first.

Platform file paths

Platform Example path Notes
iOS /tmp/real_test.wav Simulator/device temp directory
Android /data/local/tmp/test.wav Push with adb push audio.wav /data/local/tmp/ 
# Push a test file to an Android device/emulator
adb push ./fixtures/test.wav /data/local/tmp/test_transient.wav

API

getVersion(): string

Returns the native module version synchronously.

getPlatformInfo(): PlatformInfo

Returns the host OS name and version.

{ platform: 'ios' | 'android', version: string }

analyzeAudioFile(path: string): Promise<AudioAnalysisResult>

Analyzes a WAV file at the given absolute path. Resolves with a full result object; rejects if the file is missing, unreadable, or not a supported format.

Supported formats

  • iOS: any format AVFoundation can decode (WAV, AAC, MP3, etc.)
  • Android: 16-bit PCM WAV only

Types

AudioAnalysisResult

Field Type Description
score number Overall prominence score, clamped to [0, 1]
tags string[] Top-level tags (e.g. ["voice"], ["silence"])
detectedEvents DetectedEvent[] Heuristic content detections, sorted by confidence
highlights Highlight[] Up to 3 peak windows suitable for preview clips
quality AudioQuality Clipping, silence, and wind-noise flags
windows AudioWindow[] Overlapping per-window DSP metrics

AudioWindow

Field Type Description
startMs number Window start offset in milliseconds
durationMs number Window length in milliseconds
rms number Root-mean-square energy [0, 1]
transientDensity number Sharp energy rises per second

DetectedEvent

{
  label: string
  confidence: number
} // confidence in [0, 1]

Labels: speech, music, impact, ambient, silence.

Highlight

{ startMs: number; durationMs: number; score: number; tags: string[] }

AudioQuality

{
  clipping: boolean
  windNoise: boolean
  silence: boolean
}

Example

A minimal screen that runs analysis on launch and renders the results inline. Adjust TEST_WAV_PATH for your platform.

import React, { useCallback, useEffect, useState } from 'react'
import {
  ActivityIndicator,
  Platform,
  ScrollView,
  StyleSheet,
  Text,
  View,
} from 'react-native'
import AudioIntelligence, {
  type AudioAnalysisResult,
} from 'react-native-audio-intelligence'

const TEST_WAV_PATH =
  Platform.OS === 'ios'
    ? '/tmp/real_test.wav'
    : '/data/local/tmp/test_transient.wav'

const LOG = '[AudioIntelligence]'

function logAnalysis(result: AudioAnalysisResult, platform: string) {
  console.log(`${LOG} ── Analysis complete (${platform}) ──`)
  console.log(`${LOG} Prominence score: ${result.score.toFixed(5)}`)
  console.log(`${LOG} Tags: ${result.tags.join(', ') || '—'}`)
  console.log(
    `${LOG} Quality: clipping=${result.quality.clipping} ` +
      `silence=${result.quality.silence} wind=${result.quality.windNoise}`
  )
  console.log(`${LOG} Windows (${result.windows.length}):`)
  result.windows.forEach((w, i) => {
    console.log(
      `${LOG}   [${i}] ${w.startMs.toFixed(0)}ms–` +
        `${(w.startMs + w.durationMs).toFixed(0)}ms  ` +
        `rms=${w.rms.toFixed(4)}  transients=${w.transientDensity.toFixed(2)}/s`
    )
  })
  if (result.highlights.length > 0) {
    console.log(`${LOG} Highlights (${result.highlights.length}):`)
    result.highlights.forEach((h, i) => {
      console.log(
        `${LOG}   [${i}] ${h.startMs.toFixed(0)}ms  score=${h.score.toFixed(3)}`
      )
    })
  }
  console.log(`${LOG} Events (${result.detectedEvents.length}):`)
  result.detectedEvents.forEach((e, i) => {
    console.log(
      `${LOG}   [${i}] ${e.label} (${(e.confidence * 100).toFixed(1)}%)`
    )
  })
}

function formatPercent(value: number) {
  return `${(value * 100).toFixed(1)}%`
}

export default function App() {
  const [status, setStatus] = useState<'idle' | 'loading' | 'done' | 'error'>(
    'idle'
  )
  const [error, setError] = useState<string | null>(null)
  const [result, setResult] = useState<AudioAnalysisResult | null>(null)
  const [version, setVersion] = useState<string | null>(null)

  const runAnalysis = useCallback(async () => {
    setStatus('loading')
    setError(null)
    setResult(null)

    try {
      const v = AudioIntelligence.getVersion()
      setVersion(v)
      console.log(`${LOG} Module v${v} · ${Platform.OS}`)
      console.log(`${LOG} File: ${TEST_WAV_PATH}`)

      const analysis = await AudioIntelligence.analyzeAudioFile(TEST_WAV_PATH)
      logAnalysis(analysis, Platform.OS)
      setResult(analysis)
      setStatus('done')
    } catch (e) {
      const message = e instanceof Error ? e.message : String(e)
      console.error(`${LOG} Analysis failed:`, message)
      setError(message)
      setStatus('error')
    }
  }, [])

  useEffect(() => {
    runAnalysis()
  }, [runAnalysis])

  return (
    <View style={styles.root}>
      <View style={styles.header}>
        <Text style={styles.title}>Audio Intelligence</Text>
        {version && <Text style={styles.subtitle}>v{version}</Text>}
      </View>

      {status === 'loading' && (
        <View style={styles.centered}>
          <ActivityIndicator size="large" color="#6C8EFF" />
          <Text style={styles.muted}>Analyzing audio…</Text>
        </View>
      )}

      {status === 'error' && (
        <View style={styles.card}>
          <Text style={styles.errorLabel}>Analysis failed</Text>
          <Text style={styles.errorText}>{error}</Text>
        </View>
      )}

      {result && (
        <ScrollView
          style={styles.scroll}
          contentContainerStyle={styles.scrollContent}
        >
          <View style={styles.scoreCard}>
            <Text style={styles.scoreLabel}>Prominence</Text>
            <Text style={styles.scoreValue}>{formatPercent(result.score)}</Text>
            <Text style={styles.tags}>{result.tags.join(' · ')}</Text>
          </View>

          <Section title={`Events (${result.detectedEvents.length})`}>
            {result.detectedEvents.map((e, i) => (
              <Row
                key={`${e.label}-${i}`}
                label={e.label}
                value={formatPercent(e.confidence)}
              />
            ))}
          </Section>

          <Section title={`Windows (${result.windows.length})`}>
            {result.windows.map((w, i) => (
              <Row
                key={i}
                label={`${w.startMs.toFixed(0)}ms`}
                value={`rms ${w.rms.toFixed(3)} · ${w.transientDensity.toFixed(1)}/s`}
              />
            ))}
          </Section>

          {result.highlights.length > 0 && (
            <Section title={`Highlights (${result.highlights.length})`}>
              {result.highlights.map((h, i) => (
                <Row
                  key={i}
                  label={`${h.startMs.toFixed(0)}ms`}
                  value={formatPercent(h.score)}
                />
              ))}
            </Section>
          )}

          <Section title="Quality">
            <Row
              label="Clipping"
              value={result.quality.clipping ? 'yes' : 'no'}
            />
            <Row
              label="Silence"
              value={result.quality.silence ? 'yes' : 'no'}
            />
            <Row
              label="Wind noise"
              value={result.quality.windNoise ? 'yes' : 'no'}
            />
          </Section>
        </ScrollView>
      )}
    </View>
  )
}

function Section({
  title,
  children,
}: {
  title: string
  children: React.ReactNode
}) {
  return (
    <View style={styles.section}>
      <Text style={styles.sectionTitle}>{title}</Text>
      <View style={styles.sectionBody}>{children}</View>
    </View>
  )
}

function Row({ label, value }: { label: string; value: string }) {
  return (
    <View style={styles.row}>
      <Text style={styles.rowLabel}>{label}</Text>
      <Text style={styles.rowValue}>{value}</Text>
    </View>
  )
}

const styles = StyleSheet.create({
  root: {
    flex: 1,
    backgroundColor: '#0D0F14',
  },
  header: {
    paddingTop: 60,
    paddingBottom: 20,
    paddingHorizontal: 24,
    borderBottomWidth: StyleSheet.hairlineWidth,
    borderBottomColor: '#1E2230',
  },
  title: {
    fontSize: 28,
    fontWeight: '700',
    color: '#F0F2F8',
    letterSpacing: -0.5,
  },
  subtitle: {
    marginTop: 4,
    fontSize: 14,
    color: '#6C8EFF',
    fontWeight: '500',
  },
  centered: {
    flex: 1,
    justifyContent: 'center',
    alignItems: 'center',
    gap: 12,
  },
  muted: {
    color: '#6B7280',
    fontSize: 15,
  },
  scroll: {
    flex: 1,
  },
  scrollContent: {
    padding: 20,
    gap: 16,
    paddingBottom: 40,
  },
  scoreCard: {
    backgroundColor: '#161A24',
    borderRadius: 16,
    padding: 24,
    alignItems: 'center',
    borderWidth: 1,
    borderColor: '#1E2230',
  },
  scoreLabel: {
    fontSize: 13,
    fontWeight: '600',
    color: '#6B7280',
    textTransform: 'uppercase',
    letterSpacing: 1,
  },
  scoreValue: {
    fontSize: 48,
    fontWeight: '700',
    color: '#6C8EFF',
    marginTop: 4,
  },
  tags: {
    marginTop: 8,
    fontSize: 14,
    color: '#9CA3AF',
  },
  section: {
    backgroundColor: '#161A24',
    borderRadius: 12,
    overflow: 'hidden',
    borderWidth: 1,
    borderColor: '#1E2230',
  },
  sectionTitle: {
    fontSize: 13,
    fontWeight: '600',
    color: '#6B7280',
    textTransform: 'uppercase',
    letterSpacing: 0.8,
    paddingHorizontal: 16,
    paddingTop: 14,
    paddingBottom: 8,
  },
  sectionBody: {
    paddingBottom: 4,
  },
  row: {
    flexDirection: 'row',
    justifyContent: 'space-between',
    alignItems: 'center',
    paddingHorizontal: 16,
    paddingVertical: 11,
    borderTopWidth: StyleSheet.hairlineWidth,
    borderTopColor: '#1E2230',
  },
  rowLabel: {
    fontSize: 15,
    color: '#D1D5DB',
    textTransform: 'capitalize',
  },
  rowValue: {
    fontSize: 15,
    color: '#9CA3AF',
    fontVariant: ['tabular-nums'],
  },
  card: {
    margin: 20,
    backgroundColor: '#1C1014',
    borderRadius: 12,
    padding: 20,
    borderWidth: 1,
    borderColor: '#3B1C24',
  },
  errorLabel: {
    fontSize: 16,
    fontWeight: '600',
    color: '#F87171',
    marginBottom: 8,
  },
  errorText: {
    fontSize: 14,
    color: '#FCA5A5',
    lineHeight: 20,
  },
})

Sample console output

[AudioIntelligence] Module v0.3.0 · android
[AudioIntelligence] File: /data/local/tmp/test_transient.wav
[AudioIntelligence] ── Analysis complete (android) ──
[AudioIntelligence] Prominence score: 0.38922
[AudioIntelligence] Tags: voice
[AudioIntelligence] Quality: clipping=false silence=false wind=false
[AudioIntelligence] Windows (4):
[AudioIntelligence]   [0] 0ms–1500ms  rms=0.0413  transients=0.67/s
[AudioIntelligence]   [1] 750ms–2250ms  rms=0.0598  transients=1.33/s
[AudioIntelligence]   [2] 1500ms–3000ms  rms=0.0890  transients=1.33/s
[AudioIntelligence]   [3] 2250ms–3000ms  rms=0.1099  transients=1.33/s
[AudioIntelligence] Events (1):
[AudioIntelligence]   [0] speech (47.0%)

How scoring works

The prominence score combines five weighted components:

Component Weight Source
Volume 0.35 Whole-file RMS
Transients 0.25 Average transient density across windows
Frequency richness 0.20 RMS variance across windows
Voice presence 0.10 speech event confidence
Contrast 0.10 Peak-to-floor RMS ratio across windows

Penalties are applied for clipping (−0.15), silence (−0.50), and wind noise (−0.10). The final value is clamped to [0, 1].

Event detection is currently DSP-heuristic (not ML). ONNX model integration is planned for a future release.

Error handling

analyzeAudioFile rejects when:

  • The file does not exist at the given path
  • The file cannot be decoded (unsupported format, corrupt data)
  • On Android: the file is not 16-bit PCM WAV

Always use a try/catch (or .catch()) around the promise and surface the error message to the user.

License

MIT

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages