diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 0000000..4f87aba --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,318 @@ +# Development Environment Setup for ChatGPT MCP Integration + +This guide documents how to set up a **development environment** to test your MCP (Model Context Protocol) server with ChatGPT. This is NOT a production deployment guide. + +## Overview + +To integrate your local MCP server with ChatGPT, you need to expose it over HTTPS. ChatGPT requires: +- HTTPS endpoint (not HTTP) +- Support for Server-Sent Events (SSE) +- Publicly accessible URL + +## Common MCP Server Architectures + +MCP servers can be structured in different ways depending on your needs: + +### Single-Server Architecture +- **MCP Server**: One server handling both MCP protocol and serving assets +- Simple to set up and maintain +- Suitable for servers with minimal asset requirements + +### Two-Server Architecture +- **MCP Server**: Implements the MCP protocol over SSE +- **Asset Server**: Static file server for widget JavaScript, CSS, and images +- Allows independent scaling and caching strategies +- Widget HTML references asset server URLs (configured via environment variables) + +## Tunneling Approaches for Development + +When developing locally, you need to expose your MCP server to the internet. Here are three approaches we've evaluated: + +### Approach 1: ngrok (Free Tier) ❌ + +**Why Try It:** Popular, easy to use, free tier available + +**Limitations:** +- Free tier shows an interstitial warning page for browser user-agents +- This warning page blocks ChatGPT from connecting to the MCP endpoint +- Paid tier would work, but not suitable for free development + +**Command:** +```bash +ngrok http +``` + +**Result:** Connection blocked by warning page + +--- + +### Approach 2: Cloudflare Tunnels ❌ + +**Why Try It:** Free, no interstitial pages, official Cloudflare product + +**Limitations:** +- Cloudflare Tunnels **buffer GET requests** including SSE streams +- The MCP protocol relies on SSE (Server-Sent Events) for real-time communication +- Even with `X-Accel-Buffering: no` header, buffering persists +- Known issue: https://github.com/cloudflare/cloudflared/issues/1449 + +**Commands:** +```bash +# Quick tunnels (temporary URLs) +cloudflared tunnel --url http://localhost: + +# Named tunnels (requires domain) +cloudflared tunnel create mcp-tunnel +cloudflared tunnel route dns mcp-tunnel mcp.yourdomain.com +cloudflared tunnel run mcp-tunnel +``` + +**Result:** SSE buffering prevents MCP from working + +--- + +### Approach 3: SSH Reverse Tunnel + nginx ✅ **RECOMMENDED** + +**Why This Works:** +- No SSE buffering issues +- Full control over nginx configuration +- Uses existing VM infrastructure (no additional cost) +- Professional SSL setup with Let's Encrypt + +**Architecture:** +``` +Local Machine VM (your-domain.com) Internet +─────────────── ──────────────────── ──────── +MCP Server ────┐ ┌→ localhost:TUNNEL_PORT_1 + │ │ + SSH Tunnel ──────┤ ChatGPT + │ │ ↓ +Assets (opt)────┘ └→ localhost:TUNNEL_PORT_2 HTTPS:443 + ↓ ↓ + nginx ←─────────────────────── + (routes by path) +``` + +**How It Works:** + +1. **SSH Reverse Tunnel** forwards local ports to VM: + ```bash + ssh -f -N \ + -R :localhost: \ + -R :localhost: \ + @ + ``` + - `-R :localhost:`: VM port forwards to your MCP server + - `-R :localhost:`: VM port forwards to your assets (if using two-server architecture) + - Tunnel ports only listen on localhost (127.0.0.1) on the VM for security + +2. **nginx** on VM handles SSL termination and routing: + ```nginx + location /mcp { + proxy_pass http://127.0.0.1:; + proxy_buffering off; # Critical for SSE + proxy_cache off; + # ... SSE-specific settings (see docs/setup/README.md) + } + + location / { + proxy_pass http://127.0.0.1:; # Assets (if applicable) + } + ``` + +3. **Let's Encrypt** provides free SSL certificate +4. **DNS** points your domain to VM IP + +## Setup Instructions + +Detailed step-by-step instructions are in [`docs/setup/README.md`](setup/README.md), including: +- DNS configuration +- Firewall setup (ports 22, 80, 443) +- nginx installation and configuration +- SSL certificate acquisition with certbot +- SSH tunnel service setup for persistence + +## Quick Start (Assuming VM is Ready) + +```bash +# 1. Start your MCP server locally +cd && # e.g., pnpm start + +# 2. (Optional) Start asset server if using two-server architecture + # e.g., pnpm run serve + +# 3. Create SSH tunnel +ssh -f -N \ + -R :localhost: \ + -R :localhost: \ + @ + +# 4. Test MCP endpoint +MCP_URL=https:///mcp pnpm exec tsx tests/test-mcp.ts + +# 5. Add to ChatGPT +# Go to ChatGPT → Settings → Add MCP Server → https:///mcp +``` + +## Lessons Learned + +### Port Conflicts +- VMs may have services using common ports (8080, 8081, etc.) +- Check before choosing tunnel ports: `ssh @ 'sudo lsof -i:'` +- Choose unused high ports (9000+) to avoid conflicts + +### CAA DNS Records +- Let's Encrypt requires CAA record allowing `letsencrypt.org` (not `.com`) +- Check existing CAA: `dig CAA yourdomain.com` +- Add if needed: + ``` + Type: CAA + Name: @ (or yourdomain.com) + Tag: issue + Value: letsencrypt.org + ``` + +### Firewall Configuration +- Cloud providers often have external firewalls (check your provider's dashboard) +- Required ports: + - 22 (SSH) - For tunnel connection + - 80 (HTTP) - For Let's Encrypt validation + - 443 (HTTPS) - For ChatGPT access +- Tunnel ports should NOT be exposed publicly (only listen on 127.0.0.1) + +### SSL Certificate Setup +- nginx config can't reference non-existent SSL certs +- Deploy temporary HTTP-only config first +- Get certificate with certbot +- Then deploy production HTTPS config + +### SSE Requirements +Critical nginx settings for SSE: +```nginx +proxy_buffering off; +proxy_cache off; +proxy_set_header Connection ''; +chunked_transfer_encoding off; +proxy_http_version 1.1; +``` + +## Testing + +Run MCP protocol tests: +```bash +# Local (if your MCP server runs locally) +pnpm exec tsx tests/test-mcp.ts + +# Remote (through tunnel) +MCP_URL=https:///mcp pnpm exec tsx tests/test-mcp.ts +``` + +See [`tests/README.md`](../tests/README.md) for detailed testing documentation. + +## Troubleshooting + +### SSH Tunnel Disconnects +- Use autossh or systemd service for persistence +- Check `docs/setup/example-tunnel.service` for systemd template +- Verify tunnel: `ps aux | grep ssh | grep ` + +### nginx 502 Bad Gateway +- Verify SSH tunnel is active: `ps aux | grep ssh` +- Check tunnel ports on VM: `ssh @ 'ss -tlnp | grep -E "|"'` +- Restart tunnel if needed + +### ChatGPT Can't Connect +- Test endpoint: `curl https:///mcp` +- Should see SSE event stream starting +- Check nginx logs: `ssh @ 'sudo tail -f /var/log/nginx/error.log'` + +### Widget Assets Don't Load +- Check if asset URLs are correctly embedded in your build output +- Verify asset server is running: `lsof -i:` +- Test asset URL directly: `curl https:///path/to/asset.html` + +## Security Notes + +- SSH tunnel is encrypted end-to-end +- Tunnel ports only listen on localhost (not exposed to internet) +- nginx handles SSL termination with Let's Encrypt certificates +- Firewall only exposes ports 22, 80, 443 +- Consider adding authentication if your MCP server handles sensitive data + +## What's Next? + +This setup is for **development and testing only**. For production deployment: +- See [`docs/DEPLOYMENT.md`](DEPLOYMENT.md) for production guidelines +- Set up monitoring and logging +- Implement rate limiting +- Use process managers (PM2, systemd) for server persistence +- Regular security updates +- Backup and disaster recovery planning + +## Files Reference + +- `docs/setup/` - Example nginx configs, systemd services, and setup scripts +- `tests/` - MCP protocol test suite +- Your MCP server implementation files + +--- + +## Appendix: Pizzaz Example Implementation + +The **pizzaz** MCP server is a concrete example from this repository that uses the two-server architecture. Here's how it implements the concepts above: + +### Architecture + +The pizzaz implementation uses two separate servers: + +1. **MCP Server** (port 8000): Node.js server implementing MCP protocol + - Located in `pizzaz_server_node/` + - Reads widget HTML from filesystem + - Serves via SSE to ChatGPT + - Returns `structuredContent` for widget rendering + +2. **Asset Server** (port 4444): Static file server + - Serves JavaScript, CSS, and images + - Widget HTML embeds URLs pointing to this server + - Uses `serve` package for simple HTTP serving + +The `BASE_URL` environment variable (embedded during build) connects them: +```bash +BASE_URL=https://pizzaz.lazzloe.com pnpm run build +``` + +This embeds the production URL into the widget HTML for asset loading. + +### Quick Start for Pizzaz + +```bash +# 1. Build widgets with your domain +BASE_URL=https:// pnpm run build + +# 2. Start local servers +cd pizzaz_server_node && pnpm start # Terminal 1 (port 8000) +pnpm -w run serve # Terminal 2 (port 4444) + +# 3. Start SSH tunnel (example using ports 9080/9081) +ssh -f -N \ + -R 9080:localhost:8000 \ + -R 9081:localhost:4444 \ + ubuntu@ + +# 4. Test MCP endpoint +MCP_URL=https:///mcp pnpm exec tsx tests/test-mcp.ts + +# 5. Add to ChatGPT +# Go to ChatGPT → Settings → Add MCP Server → https:///mcp +``` + +### Pizzaz-Specific Files + +- `pizzaz_server_node/src/server.ts` - MCP server implementation +- `build-all.mts` - Widget build script (embeds BASE_URL) +- `src/pizzaz*/` - Widget source code +- `assets/` - Generated HTML, JS, and CSS bundles + +### Port Configuration Note + +The example uses ports 9080/9081 for tunneling to avoid conflicts with common services that might be running on the VM (like web servers on 8080/8081). diff --git a/docs/setup/README.md b/docs/setup/README.md new file mode 100644 index 0000000..99fc19d --- /dev/null +++ b/docs/setup/README.md @@ -0,0 +1,336 @@ +# MCP Server Development Environment Setup Guide + +> **Note:** This guide is for setting up a **development environment** to test your MCP server with ChatGPT. For production deployment, see [`../DEPLOYMENT.md`](../DEPLOYMENT.md). + +This guide will help you set up your MCP server to be accessible via ChatGPT using SSH reverse tunneling through a VM with nginx. This approach avoids SSE buffering issues found in some cloud tunnel services. + +## Architecture Overview + +``` +ChatGPT + ↓ HTTPS +your-domain.com (VM) + ├─ nginx (:443) + │ ├─ /mcp → localhost:TUNNEL_PORT_1 + │ └─ /* → localhost:TUNNEL_PORT_2 (optional, for assets) + ↓ SSH Reverse Tunnel +Your Local Machine + ├─ MCP Server (port MCP_PORT) + └─ Asset Server (port ASSETS_PORT, optional) +``` + +## Prerequisites + +- VM with public IP address +- Root/sudo access to the VM +- Domain name with DNS management access +- SSH access from your local machine to the VM +- SSH key pair for authentication + +## Setup Steps + +### 1. DNS Configuration + +Add an A record for your subdomain: + +``` +Type: A +Name: (e.g., mcp or your-app-name) +Value: +TTL: 3600 (or default) +``` + +**Verify DNS propagation:** +```bash +dig . +# or +nslookup . +``` + +### 2. Firewall Configuration + +Ensure your VM firewall allows: +- Port 22 (SSH) - For tunnel connection +- Port 80 (HTTP) - For Let's Encrypt validation +- Port 443 (HTTPS) - For ChatGPT access + +**Check for cloud provider firewalls:** +Many cloud providers (AWS, GCP, Azure, Hetzner, etc.) have external firewalls separate from the VM's local firewall. Check your provider's dashboard. + +### 3. VM Setup + +**Step 3.1:** Install nginx and certbot: +```bash +ssh @ +sudo apt update +sudo apt install -y nginx certbot python3-certbot-nginx +``` + +**Step 3.2:** Create nginx configuration with SSE support: + +See [`example-nginx-dev.conf`](example-nginx-dev.conf) for a template. Key points: +- Proxy `/mcp` path to your MCP server tunnel port +- Disable buffering for SSE (`proxy_buffering off`, `proxy_cache off`) +- Use HTTP/1.1 and clear Connection header +- Optional: proxy root `/` to assets server tunnel port + +Copy your config to nginx: +```bash +sudo cp /path/to/your-config.conf /etc/nginx/sites-available/ +sudo ln -s /etc/nginx/sites-available/ /etc/nginx/sites-enabled/ +sudo nginx -t # Test configuration +``` + +**Step 3.3:** Obtain SSL certificate: + +**Important:** nginx cannot start with SSL config if certificates don't exist yet. Use this approach: + +```bash +# First, deploy a temporary HTTP-only config (see example-nginx-temp.conf) +sudo cp example-nginx-temp.conf /etc/nginx/sites-available/ +sudo ln -s /etc/nginx/sites-available/ /etc/nginx/sites-enabled/ +sudo nginx -t && sudo systemctl reload nginx + +# Obtain certificate +sudo certbot --nginx -d . + +# Then deploy your full HTTPS config +sudo cp your-full-config.conf /etc/nginx/sites-available/ +sudo nginx -t && sudo systemctl reload nginx +``` + +**Note:** You'll need to provide your email for Let's Encrypt certificate registration. + +### 4. CAA DNS Record (if using Let's Encrypt) + +Let's Encrypt requires CAA record allowing `letsencrypt.org` (NOT `.com`): + +**Check existing CAA:** +```bash +dig CAA +``` + +**Add if needed:** +``` +Type: CAA +Name: @ (or ) +Tag: issue +Value: letsencrypt.org +``` + +### 5. SSH Reverse Tunnel Setup (Local Machine) + +**Step 5.1:** Ensure you have SSH key access to your VM: +```bash +ssh-copy-id @ +# or manually copy your public key to ~/.ssh/authorized_keys on the VM +``` + +**Step 5.2:** Test the SSH tunnel manually: +```bash +# Single-server architecture (MCP only) +ssh -N -R :localhost: @ + +# Two-server architecture (MCP + assets) +ssh -N -R :localhost: \ + -R :localhost: \ + @ +``` + +**Step 5.3:** Set up systemd service for persistent tunnel (recommended): + +See [`example-tunnel.service`](example-tunnel.service) for a template. Customize: +- `` - Your VM username +- `` - Your VM IP or hostname +- `` - Path to your SSH private key +- ``, `` - Your port numbers +- Add second `-R` flag if using two-server architecture + +Install the service: +```bash +sudo cp your-tunnel.service /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable your-tunnel.service +sudo systemctl start your-tunnel.service +``` + +**Step 5.4:** Check tunnel status: +```bash +sudo systemctl status your-tunnel +sudo journalctl -u your-tunnel -f +``` + +Verify ports are listening on VM: +```bash +ssh @ 'ss -tlnp | grep -E "|"' +``` + +### 6. Start Your MCP Server + +Start your MCP server locally: +```bash +cd + # e.g., pnpm start, npm start, python main.py +``` + +If using two-server architecture, also start your asset server: +```bash + # e.g., pnpm run serve +``` + +### 7. Testing + +**Step 7.1:** Test the MCP endpoint: +```bash +curl https://./mcp +# Should see SSE event stream starting +``` + +**Step 7.2:** Run MCP protocol tests (if available): +```bash +MCP_URL=https://./mcp pnpm exec tsx tests/test-mcp.ts +``` + +**Step 7.3:** Test in ChatGPT: +1. Go to ChatGPT → Settings → Add MCP Server +2. Enter URL: `https://./mcp` +3. Invoke a tool from your MCP server +4. Verify widgets render correctly (if applicable) + +## Troubleshooting + +### SSH Tunnel Not Working + +Check the tunnel status: +```bash +sudo systemctl status your-tunnel +sudo journalctl -u your-tunnel -f +``` + +Ensure tunnel ports are listening on the VM: +```bash +ssh @ 'ss -tlnp | grep -E ""' +``` + +### Port Conflicts on VM + +If your tunnel ports conflict with existing services: +```bash +ssh @ 'sudo lsof -i:' +``` + +Choose different high ports (9000+) that are available. + +### nginx Errors + +Check nginx logs on the VM: +```bash +ssh @ 'sudo tail -f /var/log/nginx/error.log' +``` + +Test nginx configuration: +```bash +ssh @ 'sudo nginx -t' +``` + +Common issues: +- **502 Bad Gateway**: Tunnel not active or wrong port +- **SSL errors**: Certificate path incorrect or doesn't exist + +### SSL Certificate Issues + +Check certificate status: +```bash +ssh @ 'sudo certbot certificates' +``` + +Renew certificate manually: +```bash +ssh @ 'sudo certbot renew' +``` + +### MCP Connection Failing + +Verify local servers are running: +```bash +curl http://localhost:/mcp +``` + +Check firewall rules (VM and cloud provider). + +## Scaling to Multiple Apps + +To add another MCP server on the same VM: + +1. Create DNS record: `. → VM_IP` +2. Create new nginx config with different tunnel ports +3. Obtain SSL certificate: `sudo certbot --nginx -d .` +4. Add ports to SSH tunnel or create separate tunnel service +5. Start your second MCP server on different local port + +## Files in This Directory + +- `example-nginx-dev.conf` - Example nginx configuration with SSE support +- `example-nginx-temp.conf` - Temporary HTTP-only config for SSL setup +- `example-tunnel.service` - Systemd service template for SSH tunnel +- `vm-setup.sh` - Example automated VM setup script +- `README.md` - This file + +## Architecture Notes + +**Why SSH Tunnel?** +- Avoids SSE buffering issues with some cloud tunnel services +- No additional cost (uses existing VM) +- Full control over nginx configuration +- Easily reproducible for multiple apps +- Encrypted connection + +**Security Considerations:** +- SSH tunnel is encrypted end-to-end +- nginx handles SSL termination with Let's Encrypt +- Tunnel ports only listen on 127.0.0.1 (not exposed to internet) +- Let's Encrypt provides free, auto-renewing certificates +- Firewall limits access to SSH, HTTP, and HTTPS ports only + +--- + +## Example: Pizzaz MCP Server Setup + +The pizzaz MCP server uses a two-server architecture. Here's the specific configuration: + +### Ports +- **Local MCP Server**: 8000 +- **Local Asset Server**: 4444 +- **VM Tunnel Port 1**: 9080 (forwards to MCP) +- **VM Tunnel Port 2**: 9081 (forwards to assets) +- **Domain**: pizzaz.lazzloe.com + +### Build Command +```bash +BASE_URL=https://pizzaz.lazzloe.com pnpm run build +``` + +### Start Commands +```bash +# Terminal 1: MCP Server +cd pizzaz_server_node && pnpm start + +# Terminal 2: Asset Server +pnpm -w run serve +``` + +### SSH Tunnel +```bash +ssh -f -N \ + -R 9080:localhost:8000 \ + -R 9081:localhost:4444 \ + ubuntu@ +``` + +### nginx Configuration +See the original `setup/nginx-pizzaz.conf` for the pizzaz-specific implementation. + +### Testing +```bash +MCP_URL=https://pizzaz.lazzloe.com/mcp pnpm exec tsx tests/test-mcp.ts +``` diff --git a/docs/setup/example-nginx-dev.conf b/docs/setup/example-nginx-dev.conf new file mode 100644 index 0000000..6ede80a --- /dev/null +++ b/docs/setup/example-nginx-dev.conf @@ -0,0 +1,64 @@ +# Example nginx configuration for MCP server development environment +# This is a generic template - customize with your values: +# - - Your full domain (e.g., mcp.example.com) +# - - VM port forwarding to your MCP server +# - - VM port forwarding to your assets (if using two-server architecture) + +server { + listen 80; + server_name ; + + # Redirect HTTP to HTTPS + return 301 https://$server_name$request_uri; +} + +server { + listen 443 ssl http2; + server_name ; + + # SSL certificates (Let's Encrypt will populate these) + ssl_certificate /etc/letsencrypt/live//fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live//privkey.pem; + + # SSL configuration + ssl_protocols TLSv1.2 TLSv1.3; + ssl_prefer_server_ciphers on; + ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384; + + # MCP endpoints - proxy to tunnel port + location /mcp { + proxy_pass http://127.0.0.1:; + proxy_http_version 1.1; + + # SSE-specific settings - CRITICAL for Server-Sent Events + proxy_buffering off; + proxy_cache off; + proxy_set_header Connection ''; + chunked_transfer_encoding off; + + # Standard proxy headers + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + + # Increase timeouts for long-lived SSE connections + proxy_read_timeout 3600s; + proxy_send_timeout 3600s; + } + + # Static assets - proxy to assets tunnel port (optional, for two-server architecture) + # Comment out this block if using single-server architecture + location / { + proxy_pass http://127.0.0.1:; + proxy_http_version 1.1; + + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + + # Cache static assets + proxy_cache_valid 200 1h; + } +} diff --git a/docs/setup/example-nginx-temp.conf b/docs/setup/example-nginx-temp.conf new file mode 100644 index 0000000..1006bbc --- /dev/null +++ b/docs/setup/example-nginx-temp.conf @@ -0,0 +1,17 @@ +# Temporary nginx configuration for SSL certificate acquisition +# Use this config BEFORE obtaining Let's Encrypt certificate +# Replace with full SSL config (example-nginx-dev.conf) after getting cert +# +# Replace with your actual domain (e.g., mcp.example.com) + +server { + listen 80; + server_name ; + + # Temporary config for getting SSL certificate + # Let's Encrypt will use this to verify domain ownership + location / { + return 200 'Temporary config for SSL cert acquisition'; + add_header Content-Type text/plain; + } +} diff --git a/docs/setup/example-tunnel.service b/docs/setup/example-tunnel.service new file mode 100644 index 0000000..910387a --- /dev/null +++ b/docs/setup/example-tunnel.service @@ -0,0 +1,48 @@ +# Example systemd service for persistent SSH reverse tunnel +# This keeps your SSH tunnel running and auto-restarts on failure +# +# Customize the following: +# - - Your local username +# - - VM port forwarding to MCP server +# - - Your local MCP server port +# - - VM port forwarding to assets (optional, remove if single-server) +# - - Your local assets port (optional, remove if single-server) +# - - Path to your SSH private key +# - - Username on the VM +# - - VM IP address or hostname +# +# Installation: +# 1. Customize this file with your values +# 2. Copy to: /etc/systemd/system/mcp-tunnel.service +# 3. Run: sudo systemctl daemon-reload +# 4. Run: sudo systemctl enable mcp-tunnel.service +# 5. Run: sudo systemctl start mcp-tunnel.service + +[Unit] +Description=SSH Reverse Tunnel for MCP Server +After=network.target + +[Service] +Type=simple +User= +Restart=always +RestartSec=10 +StartLimitIntervalSec=0 + +# SSH tunnel to VM +# For single-server architecture, remove the second -R flag +ExecStart=/usr/bin/ssh \ + -o ServerAliveInterval=60 \ + -o ServerAliveCountMax=3 \ + -o ExitOnForwardFailure=yes \ + -N \ + -R :localhost: \ + -R :localhost: \ + -i \ + @ + +# Graceful shutdown +ExecStop=/bin/kill -TERM $MAINPID + +[Install] +WantedBy=multi-user.target diff --git a/docs/setup/vm-setup.sh b/docs/setup/vm-setup.sh new file mode 100644 index 0000000..e347e08 --- /dev/null +++ b/docs/setup/vm-setup.sh @@ -0,0 +1,77 @@ +#!/bin/bash +# Generic VM Setup Script for MCP Server Development Environment +# Customize this script before running +# +# Required customizations: +# - YOUR_DOMAIN - Your full domain (line 30, 56) +# - YOUR_EMAIL - Your email for Let's Encrypt (line 56) +# - CONFIG_FILE - Your nginx config filename (line 30-31) +# +# Usage: +# 1. Upload your nginx config to VM: scp your-config.conf user@vm:/tmp/ +# 2. Upload this script to VM: scp vm-setup.sh user@vm:/tmp/ +# 3. SSH to VM and run: sudo bash /tmp/vm-setup.sh + +set -e + +echo "===================================================" +echo "MCP Server - VM Setup" +echo "===================================================" +echo "" + +# Check if running as root +if [ "$EUID" -ne 0 ]; then + echo "Please run as root (use sudo)" + exit 1 +fi + +# CUSTOMIZE THESE VARIABLES +DOMAIN="" # e.g., mcp.example.com +EMAIL="" # e.g., you@example.com +CONFIG_FILE="" # e.g., example-nginx-dev.conf + +echo "Step 1: Installing nginx and certbot..." +apt-get update +apt-get install -y nginx certbot python3-certbot-nginx + +echo "" +echo "Step 2: Copying nginx configuration..." +# Expecting the config file in /tmp/ +if [ -f "/tmp/${CONFIG_FILE}" ]; then + cp "/tmp/${CONFIG_FILE}" "/etc/nginx/sites-available/${DOMAIN}" + ln -sf "/etc/nginx/sites-available/${DOMAIN}" /etc/nginx/sites-enabled/ +else + echo "ERROR: ${CONFIG_FILE} not found in /tmp/" + echo "Please upload your nginx config to the VM first" + exit 1 +fi + +echo "" +echo "Step 3: Testing nginx configuration..." +nginx -t + +echo "" +echo "Step 4: Starting nginx..." +systemctl enable nginx +systemctl restart nginx + +echo "" +echo "Step 5: Obtaining Let's Encrypt SSL certificate..." +echo "This will request a certificate for ${DOMAIN}" +echo "Email: ${EMAIL}" +echo "" + +# Note: Remove --non-interactive if you want to review the prompts +certbot --nginx -d "${DOMAIN}" --non-interactive --agree-tos --email "${EMAIL}" + +echo "" +echo "===================================================" +echo "VM Setup Complete!" +echo "===================================================" +echo "" +echo "Next steps:" +echo "1. Ensure DNS A record points ${DOMAIN} to this VM" +echo "2. On your local machine, set up the SSH reverse tunnel" +echo "3. Start your MCP server locally" +echo "4. Test with: curl https://${DOMAIN}/mcp" +echo "" diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..5f88c1a --- /dev/null +++ b/tests/README.md @@ -0,0 +1,92 @@ +# MCP Protocol Tests + +This directory contains tests for verifying the MCP (Model Context Protocol) server implementation. + +## test-mcp.ts + +Comprehensive test suite that validates: +- MCP server connectivity over SSE (Server-Sent Events) +- Tool listing and metadata +- Tool invocation with proper return format +- Widget resource configuration +- MIME types and ChatGPT compatibility + +## Running Tests + +### Against Local Server + +```bash +# Start local servers first +cd pizzaz_server_node && pnpm start # Terminal 1 +pnpm -w run serve # Terminal 2 + +# Run tests +pnpm exec tsx tests/test-mcp.ts +``` + +### Against Production/Dev Environment + +```bash +# Test against your tunneled endpoint +MCP_URL=https://pizzaz.lazzloe.com/mcp pnpm exec tsx tests/test-mcp.ts +``` + +### Using Environment Variables + +The test automatically discovers the MCP URL in this order: + +1. `MCP_URL` - Full MCP endpoint URL +2. `MCP_TUNNEL_ID` - Cloudflare tunnel ID (constructs URL) +3. `ASSETS_TUNNEL_ID` - Falls back to asset tunnel +4. Default: `http://localhost:8000/mcp` + +## Test Output + +Successful test output: +``` +🔍 Discovering MCP URL... +MCP Endpoint: https://pizzaz.lazzloe.com/mcp + +🔌 Connecting to MCP server... +✅ Connected to MCP server + +✓ Testing list tools... + ✅ All 5 tools present with correct metadata +✓ Testing tool invocation... + ✅ Tool invocation successful with correct metadata +✓ Testing widget resource... + ✅ Widget resource has correct MIME type and metadata +✓ Testing list resources... + ✅ All 5 resources properly configured + +============================================================ +🎉 All MCP protocol tests passed! +``` + +## What the Tests Verify + +1. **SSE Connection**: Ensures the server properly handles Server-Sent Events +2. **Tool Metadata**: Validates OpenAI-specific metadata fields: + - `openai/outputTemplate` + - `openai/toolInvocation/invoking` + - `openai/toolInvocation/invoked` + - `openai/widgetAccessible` + - `openai/resultCanProduceWidget` +3. **Tool Invocation**: Confirms tools return `structuredContent` for widget rendering +4. **Resource Configuration**: Verifies widget resources use `text/html+skybridge` MIME type +5. **ChatGPT Compatibility**: Ensures all requirements for ChatGPT Apps integration + +## Troubleshooting + +**Error: SSE error** +- Check that the MCP server is running +- Verify the URL is accessible (try with curl) +- For tunneled endpoints, ensure firewall allows HTTPS traffic + +**Error: Tool invocation missing structuredContent** +- Update pizzaz_server_node/src/server.ts to return `structuredContent` +- Rebuild and restart the server + +**Connection timeout** +- Increase timeout in test file if testing slow connections +- Check SSH tunnel is active: `ps aux | grep ssh` diff --git a/tests/test-mcp.ts b/tests/test-mcp.ts new file mode 100644 index 0000000..b4f3775 --- /dev/null +++ b/tests/test-mcp.ts @@ -0,0 +1,214 @@ +#!/usr/bin/env tsx +/** + * MCP Protocol Tests - Validates ChatGPT-compatible MCP implementation + * Environment-agnostic: works in local dev (with tunnels) and CI/CD (with env vars) + */ + +// Polyfill EventSource for Node.js +import { EventSource } from 'eventsource'; +(global as any).EventSource = EventSource; + +import { Client } from '@modelcontextprotocol/sdk/client/index.js'; +import { SSEClientTransport } from '@modelcontextprotocol/sdk/client/sse.js'; + +async function discoverMcpUrl(): Promise { + // Priority 1: Full MCP URL from environment (CI/CD, production, or dev override) + if (process.env.MCP_URL) { + return process.env.MCP_URL; + } + + // Priority 2: Construct from Cloudflare tunnel ID (local dev) + if (process.env.MCP_TUNNEL_ID) { + return `https://${process.env.MCP_TUNNEL_ID}.cfargotunnel.com/mcp`; + } + + throw new Error( + 'No MCP URL found. Please either:\n' + + ' 1. Run: ./scripts/setup-tunnels.sh (creates .env with tunnel IDs), or\n' + + ' 2. Set MCP_URL environment variable' + ); +} + +async function main() { + console.log('🔍 Discovering MCP URL...\n'); + + const mcpEndpoint = await discoverMcpUrl(); + console.log(`MCP Endpoint: ${mcpEndpoint}\n`); + + console.log('🔌 Connecting to MCP server...'); + + const transport = new SSEClientTransport(new URL(mcpEndpoint)); + const client = new Client( + { name: 'test-client', version: '1.0.0' }, + { capabilities: {} } + ); + + await client.connect(transport); + console.log('✅ Connected to MCP server\n'); + + let errors = 0; + + // Test 1: List tools + try { + console.log('✓ Testing list tools...'); + const { tools } = await client.listTools(); + + const expectedTools = ['pizza-map', 'pizza-carousel', 'pizza-albums', 'pizza-list', 'pizza-shop']; + const foundTools = tools.map(t => t.name); + + if (tools.length !== expectedTools.length) { + console.error(` ❌ Expected ${expectedTools.length} tools, got ${tools.length}`); + errors++; + } + + for (const expected of expectedTools) { + if (!foundTools.includes(expected)) { + console.error(` ❌ Missing tool: ${expected}`); + errors++; + } + } + + // Validate critical metadata + const sampleTool = tools[0]; + if (!sampleTool._meta || !sampleTool._meta['openai/outputTemplate']) { + console.error(' ❌ Tool missing openai/outputTemplate metadata'); + errors++; + } + if (!sampleTool._meta['openai/widgetAccessible']) { + console.error(' ❌ Tool missing openai/widgetAccessible metadata'); + errors++; + } + + if (errors === 0) { + console.log(` ✅ All ${tools.length} tools present with correct metadata`); + } + } catch (error) { + console.error(` ❌ List tools failed: ${error}`); + errors++; + } + + // Test 2: Call a tool + try { + console.log('✓ Testing tool invocation...'); + const result = await client.callTool({ + name: 'pizza-map', + arguments: { pizzaTopping: 'pepperoni' } + }); + + // Validate response structure + if (!result.content || result.content.length === 0) { + console.error(' ❌ Tool result missing content'); + errors++; + } + + if (!result._meta) { + console.error(' ❌ Tool result missing _meta'); + errors++; + } else { + if (!result._meta['openai/toolInvocation/invoking']) { + console.error(' ❌ Missing openai/toolInvocation/invoking metadata'); + errors++; + } + if (!result._meta['openai/toolInvocation/invoked']) { + console.error(' ❌ Missing openai/toolInvocation/invoked metadata'); + errors++; + } + } + + if (!result.structuredContent) { + console.error(' ❌ Tool result missing structuredContent'); + errors++; + } + + if (errors === 0) { + console.log(' ✅ Tool invocation successful with correct metadata'); + } + } catch (error) { + console.error(` ❌ Tool invocation failed: ${error}`); + errors++; + } + + // Test 3: Read a widget resource + try { + console.log('✓ Testing widget resource...'); + const result = await client.readResource({ + uri: 'ui://widget/pizza-map.html' + }); + + if (!result.contents || result.contents.length === 0) { + console.error(' ❌ Resource missing contents'); + errors++; + } else { + const content = result.contents[0]; + + if (content.mimeType !== 'text/html+skybridge') { + console.error(` ❌ Wrong MIME type: ${content.mimeType} (expected text/html+skybridge)`); + errors++; + } + + if (!content.text || content.text.length === 0) { + console.error(' ❌ Resource HTML is empty'); + errors++; + } + + if (!content._meta || !content._meta['openai/widgetAccessible']) { + console.error(' ❌ Resource missing widget metadata'); + errors++; + } + } + + if (errors === 0) { + console.log(' ✅ Widget resource has correct MIME type and metadata'); + } + } catch (error) { + console.error(` ❌ Resource read failed: ${error}`); + errors++; + } + + // Test 4: List resources + try { + console.log('✓ Testing list resources...'); + const { resources } = await client.listResources(); + + if (resources.length !== 5) { + console.error(` ❌ Expected 5 resources, got ${resources.length}`); + errors++; + } + + const allHaveMetadata = resources.every(r => r._meta && r.mimeType === 'text/html+skybridge'); + if (!allHaveMetadata) { + console.error(' ❌ Some resources missing metadata or wrong MIME type'); + errors++; + } + + if (errors === 0) { + console.log(` ✅ All ${resources.length} resources properly configured`); + } + } catch (error) { + console.error(` ❌ List resources failed: ${error}`); + errors++; + } + + await client.close(); + console.log('\n' + '='.repeat(60)); + + if (errors === 0) { + console.log('🎉 All MCP protocol tests passed!\n'); + console.log('✓ ChatGPT-compatible MCP implementation verified'); + console.log('✓ All tools have correct metadata'); + console.log('✓ Widget resources use proper MIME types'); + console.log('✓ Tool invocations return required fields\n'); + console.log('📋 Ready for ChatGPT:'); + console.log(` ${mcpEndpoint}\n`); + process.exit(0); + } else { + console.error(`❌ ${errors} test(s) failed\n`); + console.error('Fix these issues before connecting to ChatGPT\n'); + process.exit(1); + } +} + +main().catch(error => { + console.error('Fatal error:', error); + process.exit(1); +});