Transcription Server

A server that integrates AI transcription services like Fieldy AI with MCP (Model Context Protocol) clients like Claude Desktop, enabling natural language search and retrieval of recorded conversations.

Note

This is a self-hosted server, so setup is fairly technical. At the very least, you'll need to have some experience with the command line and a computer you can leave on 24/7. If that sounds like you, great! If you get stuck, ChatGPT or Claude AI are great resources.

What It Does

Transcription Server bridges the gap between AI transcription services and AI assistants by:

• 🔗 Receiving transcriptions from AI services via webhooks
• 📚 Organizing conversations using graph relationships between transcriptions
• 🔍 Enabling search through MCP tools for date-based and semantic queries
• 🎯 Delivering results to AI clients in structured, readable formats

How It Works

Data Flow

1. AI Transcription Service (Fieldy) processes recorded audio
2. Webhook sends transcription data to this server
3. Server processes transcriptions into searchable conversation groups
4. AI clients (like Claude Desktop) query the server through MCP tools
5. Server returns relevant conversation excerpts and metadata

MCP Integration

The server exposes two primary search capabilities:

• search_by_date - Find conversations by natural language date ranges
  • Examples: "yesterday", "last week", "2025-01-15 to 2025-01-20"
  • Returns conversation summaries with timestamps and duration
• search_by_text - Find conversations by content (coming in Phase 2)
  • Semantic search across transcription content
  • Vector-based similarity matching

Quick Start

Prerequisites

• Bun runtime
• SQLite (comes bundled with Bun)
• Linux/macOS environment you can run 24/7 (e.g., a Raspberry Pi or an old laptop with sufficient storage)
• Tailscale (for remote access)

Installation

To use the MCP server and collect transcriptions with the webhook, the server and tunnel should be available at all times. This server was designed to run on a local Linux machine (see "Production Deployment with Tailscale" below), but you may also use Fly.io for deployment (instructions coming soon).

# Clone the repository
git clone https://github.com/jacksteamdev/transcription-server.git
cd transcription-server

# Install dependencies
bun install  # or npm install

Configuration

Copy the environment template and configure:

cp .env.example .env

Edit .env with your settings:

# Database path
SQLITE_DB_PATH=./data/transcriptions.db

# Server configuration
PORT=3000
WEBHOOK_TOKEN=your_secure_webhook_token

# MCP server settings
FASTMCP_PORT=8080
FASTMCP_TRANSPORT=http-stream

Running the Server

# Development mode
bun run dev

# Production mode
bun run start

The server will be available at:

• Main API: http://localhost:3000
• MCP Server: http://localhost:8080/mcp

Deployment with Tailscale

Fieldy's webhook will trigger many times a day, upto once every 30 seconds. If you deploy on a local machine, you'll want to ensure it's always running and can be accessed from the internet. A tunneling service is the recommended way to do this.

We recommend Tailscale Funnels, which has a generous free tier that is relatively easy to configure. Other tunneling services like ngrok or Cloudflare Tunnels can also be used, but may require a paid plan to handle the volume of webhook requests.

On Linux, you may use the provided systemd services to run the server and tunnel:

Quick Setup (Linux)

cd docs/services
./install.sh          # Interactive installation
./install.sh --dry-run # Preview changes first

# After installation, edit .env with your Google AI API key
nano .env
# GOOGLE_API_KEY=your_api_key_here

Manual Installation

For manual setup, see docs/services/README.md

Usage with Fieldy App (Webhook)

Open the Fieldy app settings and scroll down to "Developer Settings". Here you will find the "Webhook Endpoint URL". Paste your server's webhook URL into this field (e.g., http://your-server-ip:3000/webhook/<WEBHOOK_TOKEN>, replacing <WEBHOOK_TOKEN> with your actual WEBHOOK_TOKEN from the .env file).

View server logs to verify that the webhook URL is correct and receiving data (e.g., sudo journalctl -u transcription-server.service -f). Speak into the device and after about 30 seconds, you should see log entries indicating successful webhook processing. If you're running in development mode, logs will appear directly in your terminal.

Usage with MCP Clients

Claude Desktop Integration

1. Install Claude Desktop

2. Open the Settings menu

3. Select the Connectors tab

4. Scroll down and click "Add custom connector"

5. Enter the MCP Server URL (e.g., http://localhost:8080/mcp) into the provided field. If you're using Tailscale, this will be your Tailscale IP address or Funnel URL. For example, https://your-funnel-url.ts.net/mcp.

6. Start querying conversations in Claude:

Find transcriptions from last week
Show me yesterday's conversations
When did Alice ask about the budget?

Postman Testing

Use the included Postman collection (docs/postman_collection.json) to:

• ✅ Test webhook endpoints with sample transcription data
• ✅ Verify MCP tool responses with various search queries
• ✅ Monitor server health and data ingestion

Architecture Overview

Tech Stack

• Runtime: BunJS (high-performance JavaScript)
• Language: TypeScript for type safety
• Web Framework: Hono (edge-optimized)
• Database: SQLite with graph schema (nodes/edges)
• MCP: FastMCP server implementation
• Validation: Arktype runtime type checking

Data Model

Uses a graph database approach to organize transcriptions:

Nodes: Individual transcription segments
Edges: Relationships between segments (time-based clustering)
Conversations: Grouped segments with shared context

API Endpoints

Endpoint	Method	Purpose
/webhook/<WEBHOOK_TOKEN>	POST	Receive transcription data
/mcp	GET/POST	MCP protocol endpoint
/health	GET	Server health check

Development & Testing

Available Scripts

# Development
bun run dev          # Start dev server with hot reload
bun run types        # Type checking
bun run lint         # Code quality check

# Testing
bun run test         # Run test suite

# MCP Inspection
bun run inspect:mac  # Test MCP tools locally

Testing MCP Integration

# Test date-based search
npx @wong2/mcp-cli --url http://localhost:8080/mcp

Deployment Options

Docker Deployment

# Build and run with Docker
docker build -t transcription-server .
docker run -p 3000:3000 -p 8080:8080 transcription-server

Production Deployment

• Fly.io: Included fly.toml configuration
• Linux Systems: Use provided systemd service files
• Bare Metal: Direct BunJS execution in production

Security Considerations

• Webhook Authentication: Token-based verification in URL path
• Input Validation: Comprehensive payload validation with Arktype
• SQL Injection Prevention: Parameterized queries only
• Rate Limiting: Consider implementing on webhook endpoints

Troubleshooting

Common Issues

"MCP client times out"

• Check MCP server is running on correct port (8080)
• Verify MCP transport configuration matches client expectations

"No transcriptions found"

• Confirm webhook endpoint is receiving data
• Check webhook token authentication
• Verify database has been initialized

"Date parsing issues"

• Use natural language like "yesterday" or "last week"
• Try ISO format: "2025-01-15T10:30:00Z"

Logs and Debugging

# View server logs
bun run start 2>&1 | tee server.log

# Test webhook manually
curl -X POST -H "Content-Type: application/json" \
  -d @test-webhook.json \
  http://localhost:3000/webhook/YOUR_TOKEN/transcription

Phase Roadmap

✅ Phase 1: Date-Based Search (Complete)

• Webhook ingestion into SQLite graph database
• MCP server with search_by_date tool
• Basic conversation clustering

✅ Phase 2: Semantic Search (In Progress)

• Vector database integration (LanceDB)
• Embedding generation for transcriptions
• Semantic search tool implementation

📋 Phase 3: Advanced Features (Planned)

• Multi-user support
• Conversation summarization
• Audio processing integration

Contributing

This project follows these development principles:

• TypeScript Strict: Full type coverage required
• Module-Based: No classes, functional exports only
• Graph-First: Data relationships using nodes/edges schema
• MCP-Compliant: FastMCP standards for tool interfaces

License

MIT License - see LICENSE file for details.

Support

For questions or issues:

• Check the Postman collection for examples
• Review the memory-bank files for detailed project insights
• Open GitHub issues for bugs and feature requests