TubeTranscript MCP

Local-first YouTube transcript extraction, translation, summary, and batch processing demo built with Next.js, FastAPI, in-process FastMCP tools, Kafka, and PostgreSQL.

Runtime

• Python 3.12 for FastAPI, worker, providers, and core logic
• Next.js UI in apps/web
• FastAPI backend in apps/api
• SerpAPI transcript provider configured inside FastAPI
• Azure OpenAI translation and summary provider configured inside FastAPI
• FastMCP tool definitions hosted in the FastAPI backend package
• Kafka and PostgreSQL through Docker Compose

The Python Docker image sets PYTHONPATH=/app so FastAPI and the worker can import shared core and infrastructure packages. The UI uses a separate Node image and calls FastAPI through a server-side Next.js proxy route, so API_KEY is not exposed to the browser.

Current Architecture

Next.js UI
  -> Next.js server proxy /api/backend/*
    -> FastAPI REST API
      -> Core transcript use case
        -> SerpAPI provider or local fake provider
        -> Azure OpenAI translation provider or local fake provider

FastAPI batch endpoint
  -> PostgreSQL batch/job records
  -> Kafka transcript.jobs.requested topic
  -> Worker consumer
    -> Core transcript use case
    -> PostgreSQL job result/failure update
    -> Kafka completed/failed topic

FastMCP tools live in apps/api/fastmcp_tools.py. They share the same .env configuration and secrets as FastAPI. There are no separate MCP Docker containers in the local MVP.

Docker Compose starts:

• ui - Next.js frontend on http://localhost:3000
• api - FastAPI backend on http://localhost:8000
• worker - Kafka transcript worker
• kafka - local Kafka broker
• database - local PostgreSQL

Setup

Create a local .env from the example:

cp .env.example .env

Fill these values when you are ready to call real providers:

SERPAPI_API_KEY=...
AZURE_OPENAI_ENDPOINT=...
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_API_VERSION=...
AZURE_OPENAI_TRANSLATION_DEPLOYMENT=...

Until those values are configured, the API uses local placeholder transcript and translation providers so the UI and API can still be exercised.

Set API_KEY to protect transcript endpoints during local demos. When configured, the Next.js server proxy sends it to FastAPI as X-API-Key.

Important Docker settings:

UI_PORT=3000
API_BASE_URL=http://api:8000
TRANSCRIPT_PROVIDER=fastapi_fastmcp_serpapi
TRANSLATION_PROVIDER=fastapi_fastmcp_azure_openai
MCP_SERPAPI_TRANSPORT=in_process
MCP_AZURE_OPENAI_TRANSPORT=in_process
POSTGRES_HOST=database
POSTGRES_PORT=5432
POSTGRES_DB=transcript_ai
POSTGRES_USER=transcript_user
POSTGRES_PASSWORD=replace_with_local_password
DATABASE_URL=
KAFKA_BOOTSTRAP_SERVERS=kafka:9092

DATABASE_URL is optional. If it is empty, the backend builds it from POSTGRES_HOST, POSTGRES_PORT, POSTGRES_DB, POSTGRES_USER, and POSTGRES_PASSWORD. Set POSTGRES_PASSWORD in your local .env; it is required by Docker Compose and should not be committed.

PostgreSQL is exposed on host port 5433 by default to avoid conflicts with a local PostgreSQL already using 5432. Inside Docker Compose, services still connect to database:5432.

Run With Docker Compose

docker compose up --build

Useful URLs:

• Next.js UI: http://localhost:3000
• FastAPI docs: http://localhost:8000/docs
• API health: http://localhost:8000/health

If the Postgres volume was created with an older image or mount path, reset local dev data once:

docker compose down -v
docker compose up --build

Run Without Docker

Backend:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements/dev.txt
uvicorn apps.api.main:app --reload

Frontend:

cd apps/web
npm install
API_BASE_URL=http://localhost:8000 npm run dev

Worker:

python -m apps.worker.consumer

For local non-Docker backend services, point .env at local Kafka/PostgreSQL:

POSTGRES_HOST=localhost
POSTGRES_PORT=5433
KAFKA_BOOTSTRAP_SERVERS=localhost:9092

API Examples

Sync transcript:

curl -X POST http://localhost:8000/v1/transcripts/sync \
  -H 'Content-Type: application/json' \
  -d '{
    "youtube_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    "include_metadata": true,
    "include_chunks": true,
    "language_code": "en",
    "translate": false
  }'

Transcript summary:

curl -X POST http://localhost:8000/v1/transcripts/summary \
  -H 'Content-Type: application/json' \
  -d '{
    "title": "Example video",
    "target_language": "en",
    "transcript_text": "Transcript text goes here..."
  }'

Batch submission:

curl -X POST http://localhost:8000/v1/transcripts/batch \
  -H 'Content-Type: application/json' \
  -d '{
    "videos": [
      "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
    ],
    "include_metadata": true,
    "include_chunks": true,
    "language_code": "en"
  }'

Batch status:

curl http://localhost:8000/v1/transcripts/batch/{batch_id}

Job result:

curl http://localhost:8000/v1/transcripts/result/{job_id}

Tests

Python checks:

pip install -r requirements/dev.txt
pytest

Frontend checks:

cd apps/web
npm install
npm run build

The default Python test suite does not require real SerpAPI or Azure OpenAI credentials.

Current MVP Limitations

• FastMCP tools are defined in-process, but an external MCP HTTP/SSE endpoint is not mounted yet.
• Database schema creation is currently simple startup table creation, not a full Alembic migration flow.
• Kafka retry, dead-letter, and idempotency behavior are still minimal.
• Observability endpoints are intentionally lightweight.
• Real SerpAPI, Azure OpenAI translation, and Azure OpenAI summary flows need smoke testing after .env credentials are added.

Security Notes

• .env is ignored by Git and should hold real provider credentials.
• The browser never receives SerpAPI, Azure OpenAI, or FastAPI API_KEY secrets.
• Next.js proxies API calls through /api/backend/* and injects X-API-Key server-side when configured.
• FastAPI transcript endpoints support optional API_KEY protection via X-API-Key.
• YouTube metadata enrichment uses a backend-constructed YouTube oEmbed URL from a parsed video ID, not arbitrary user-provided URLs.
• Request models cap URL, batch, language, title, and summary text sizes.
• Kafka and PostgreSQL are configured for local development only; production would need network isolation, authentication, managed secrets, rate limiting, and stronger audit/observability.