Skip to content

along528/echolocate

Repository files navigation

EchoLocate

EchoLocate

echolocate.app — EchoLocate is a music discovery system powered by MCP (Model Context Protocol), Google Cloud Run, DuckDB, and audio vector search. It exposes sonic similarity search and playlist generation via an MCP server, backed by a high-performance vector search service.

Architecture

  • mcp/: Remote MCP server exposing 6 EchoLocate tools (echolocate_*). Handles OAuth2/JWT authentication and proxies requests to the vector service using Google Cloud ID tokens.
  • vector/: Python vector search service using DuckDB. Serves audio embeddings and supports sonic interpolation. Used as a build dependency for the Rust service.
  • vector-rs/: Rust/Axum vector search service (primary deployment). Baked-index architecture — DuckDB index is embedded in the container image, eliminating cold-start latency.
  • frontend/: Browser UI for exploring tracks with semantic search and interpolation. Publicly accessible at echolocate.app.
  • embeddings/: Scripts for processing audio files, generating MERT and CLAP embeddings, and building the DuckDB database.
  • fma-ingest/: Cloud Run job for ingesting Free Music Archive data from GCS.
  • firestore/: Firestore security rules and deployment config (used by the semantic search cache).

Features

  • Sonic Search: Find similar tracks based on audio embeddings (MERT model, 768-dim).
  • Semantic Search: Text-to-audio search using CLAP embeddings (512-dim). Queries are expanded by Vertex AI before embedding to improve recall on short or terse descriptions.
  • Sonic Interpolation: Generate smooth playlists between two tracks using recursive bisection — each step finds the nearest real track to the vector midpoint of the current segment, producing a gradual sonic path between two very different starting points. Supports an optional steering track to bend the path via Bézier interpolation.
  • Frontend Explorer: Browser UI with text search, semantic search, and interpolation playlist builder.

Performance

Cold start was reduced from ~45–60s to ~10–15s through: baked-index architecture (DuckDB index embedded in the container image), parallelised HNSW index warming with ONNX model loading, and migration to Rust for the vector service.

Metric Value
Cold start ~10–15s
Semantic search p50 ~250ms
Semantic search p99 ~600ms
Compute cost ~$0–$0.10/user/day (Cloud Run, min 0 instances)
Audio storage ~$0.30/day (~1TB FMA audio, GCS nearline)

Security Model

No load balancer — each service is accessed directly via its Cloud Run URL:

  • Frontend: Public (--allow-unauthenticated, --no-iap). Custom domain echolocate.app via Cloud Run domain mapping.
  • Vector service: Public (--allow-unauthenticated) — read-only search. CORS restricted to https://echolocate.app. The MCP server has roles/run.invoker for service-to-service calls.
  • MCP server: Publicly reachable for the OAuth2 handshake (/authorize, /token). All sensitive routes protected by AuthMiddleware.

Deployment

The entire stack deploys to Google Cloud Run.

Prerequisites

  • Google Cloud SDK (gcloud) with beta component, authenticated.
  • A Google Cloud Project with Cloud Run and Secret Manager enabled.
  • A GCS bucket containing your DuckDB file (for the vector service), or a baked data/index.duckdb for the Rust service.

Deploy all services

./deploy.sh

This deploys in order: vector service → MCP server → frontend.

Set up Custom Domain

After deploying, map the custom domain:

gcloud beta run domain-mappings create \
    --service=cloud-crate-frontend \
    --domain=echolocate.app \
    --region=us-central1 \
    --project=<YOUR_PROJECT>

Add the A/AAAA records shown by gcloud beta run domain-mappings describe --domain=echolocate.app --region=us-central1 at your domain registrar. Cloud Run provisions a managed TLS certificate automatically.

Tear down old load balancer (one-time)

./teardown_lb.sh

Local Development

# EchoLocate MCP Server (port 8080)
cd mcp && python main.py

# Vector Service — Rust (port 8080)
cd vector-rs && INDEX_DB_PATH=../data/index.duckdb cargo run

# Vector Service — Python legacy (port 8000)
cd vector && uvicorn main:app --reload

The frontend automatically uses http://localhost:8001 as the vector API base when served from localhost.

Verification

python vector/verify_service.py <VECTOR_URL>
python mcp/verify_auth.py <MCP_URL>

# Frontend (expect 200)
curl -s -o /dev/null -w '%{http_code}' https://echolocate.app/

Troubleshooting

Vector API calls fail with CORS errors

Cause: CORS_ALLOW_ORIGINS on the vector service doesn't include the frontend origin.

Fix: Verify the vector service has CORS_ALLOW_ORIGINS=https://echolocate.app. Redeploy with cd vector && ./deploy.sh if needed.

Roadmap

Interpolation

  • Asymmetric phase matching — Score transitions using the delta between outro and intro segment vectors, treating similarity as directional rather than symmetric.
  • k-NN shortest path — Build a nearest-neighbor graph and find actual shortest paths rather than recursive bisection.
  • Maximum marginal relevance — Balance similarity to the target path with diversity to avoid repetitive track selections.

Explainability

  • Audio-to-text generation — Generate natural language descriptions from audio embeddings to explain why tracks are considered similar.

Embeddings

  • Wider audio sampling — Average embeddings across multiple windows per track rather than a single 5-second segment.
  • CLAP for interpolation — Explore using CLAP embeddings alongside MERT for the interpolation path.

Acknowledgements

About

AI-powered semantic search and sonic interpolation of the Free Music Archive

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors