Skip to content

feat(sound): add FineLAP audio event search - #130

Merged
tulayha merged 4 commits into
mainfrom
docs/multimodal-model-direction
Aug 28, 2026
Merged

feat(sound): add FineLAP audio event search#130
tulayha merged 4 commits into
mainfrom
docs/multimodal-model-direction

Conversation

@tulayha

@tulayha tulayha commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a first-class FineLAP sound capability for music, environmental sounds, and general audio events
  • store both global 10-second window embeddings and dense timestamped activations at 160 ms intervals
  • expose sound search through the shared CLI, HTTP, MCP, browser, and Desktop contracts
  • pin FineLAP and its RoBERTa text assets for reproducible local preparation
  • refresh the multimodal model direction and benchmark documentation, with LongVALE retained as the next sound-quality evaluation target

Validation

  • uv lock --check
  • uv run --no-sync ruff check .
  • uv run --no-sync python -m pytest -q — 706 passed, 5 skipped, 102 subtests passed
  • npm --prefix desktop run check — typecheck, ESLint, 41 tests, and production build passed
  • npx --yes markdownlint-cli2@0.23.2 "**/*.md" "#node_modules"
  • real local FineLAP prepare, model-load, audio-indexing, and text-to-sound search smoke test

Notes

  • FineLAP inference is local and does not require a paid model API.
  • Sound retrieval quality has not yet been scored on LongVALE; the docs identify that as follow-up benchmark work.
  • Link checking could not be run because Lychee was unavailable and the Docker daemon was not running.

@tulayha
tulayha merged commit c8942c6 into main Aug 28, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant