Extractor API

Public API service for extracting content from TikTok, YouTube, articles, and more

Extract content from various platforms with a simple HTTP API. Perfect for n8n workflows, student projects, and automation.

Quick Start

curl -X POST https://api.prompt-stack.dev/v1/extract/tiktok \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://tiktok.com/@user/video/123456789"}'

Features

✅ TikTok: Extract video transcripts, metadata, engagement metrics
✅ YouTube: Get video transcripts and metadata
✅ Articles: Extract clean content from any URL
✅ Reddit: Get post content and comments
✅ Auto-detect: Smart platform detection from URL
✅ n8n Ready: Works out-of-the-box with n8n workflows
✅ Rate Limited: Fair usage for all users
✅ Well Documented: Auto-generated API docs

Supported Platforms

Platform	Endpoint	What You Get
TikTok	`/v1/extract/tiktok`	Transcript, views, likes, comments, shares
YouTube	`/v1/extract/youtube`	Transcript, metadata, duration
Articles	`/v1/extract/article`	Clean content, title, author, word count
Reddit	`/v1/extract/reddit`	Post content, comments, upvotes
Auto-detect	`/v1/extract`	Automatically detects platform

Installation (Self-Hosting)

Prerequisites

Docker & Docker Compose
Git
Python 3.11+ (for local development)
Node.js 18+ (for extractors)

Setup

# Clone the repository
git clone https://github.com/prompt-stack/extractor-api
cd extractor-api

# Initialize git submodules (extractors)
git submodule update --init --recursive

# Copy environment file
cp .env.example .env

# Edit .env and add your API keys
# API_KEYS=your-key-1,your-key-2,your-key-3

# Run with Docker Compose
docker-compose up --build

# API will be available at http://localhost:8000

API Documentation

Authentication

All endpoints require an API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Endpoints

Universal Extract

Auto-detects platform and extracts content:

POST /v1/extract
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY

{
  "url": "https://tiktok.com/@user/video/123",
  "options": {}
}

Platform-Specific Extract

Extract from specific platform:

POST /v1/extract/{platform}
# platforms: tiktok, youtube, article, reddit

{
  "url": "https://...",
  "options": {}
}

Response Format

{
  "success": true,
  "platform": "tiktok",
  "data": {
    "title": "Video title",
    "author": "username",
    "content": "Transcript...",
    "metadata": {
      "views": 1000000,
      "likes": 50000
    }
  },
  "extracted_at": "2025-10-19T20:00:00Z",
  "credits_used": 1
}

Interactive Documentation

Visit these URLs when running:

Swagger UI: http://localhost:8000/docs
ReDoc: http://localhost:8000/redoc

Usage Examples

cURL

curl -X POST http://localhost:8000/v1/extract/tiktok \
  -H "Authorization: Bearer demo-key-123" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://tiktok.com/@user/video/123"}'

Python

import requests

API_KEY = "your-api-key"
API_URL = "https://api.prompt-stack.dev/v1/extract"

def extract_content(url):
    response = requests.post(
        API_URL,
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"url": url}
    )
    return response.json()

# Extract from any platform
result = extract_content("https://tiktok.com/@user/video/123")
print(result["data"])

JavaScript/Node.js

const API_KEY = 'your-api-key';
const API_URL = 'https://api.prompt-stack.dev/v1/extract';

async function extractContent(url) {
  const response = await fetch(API_URL, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ url }),
  });

  return response.json();
}

// Usage
const result = await extractContent('https://tiktok.com/@user/video/123');
console.log(result.data);

n8n Workflow

Add HTTP Request node
Configure:
- Method: POST
- URL: https://api.prompt-stack.dev/v1/extract/tiktok
- Authentication: Generic Credential Type
  - Header Auth
  - Name: Authorization
  - Value: Bearer YOUR_API_KEY
- Body: JSON
```
{
  "url": "{{$json.tiktok_url}}"
}
```

See complete n8n guide

Development

Local Development (Without Docker)

# Install Python dependencies
cd gateway
pip install -r requirements.txt

# Setup extractors (git submodules)
cd ..
git submodule update --init --recursive

# Install extractor dependencies
cd extractors/tiktok && npm install
cd ../article && npm install

# Run the API
cd ../../gateway
python main.py

API runs at http://localhost:8000

Project Structure

extractor-api/
├── gateway/              # FastAPI gateway
│   ├── main.py          # Main application
│   ├── auth/            # Authentication
│   ├── routers/         # API routes
│   └── requirements.txt # Python deps
├── extractors/          # Git submodules
│   ├── tiktok/         → github.com/prompt-stack/tiktok-extractor
│   ├── article/        → github.com/prompt-stack/article-extractor
│   └── youtube/        → github.com/prompt-stack/youtube-extractor
├── docs/                # Documentation
├── Dockerfile
├── docker-compose.yml
└── README.md

Deployment

Railway

# Install Railway CLI
npm install -g @railway/cli

# Login
railway login

# Create new project
railway init

# Set environment variables
railway variables set API_KEYS="key1,key2,key3"

# Deploy
railway up

# Get public URL
railway domain

Fly.io

# Install flyctl
curl -L https://fly.io/install.sh | sh

# Login
fly auth login

# Launch app
fly launch

# Set secrets
fly secrets set API_KEYS="key1,key2,key3"

# Deploy
fly deploy

Configuration

Environment Variables

Variable	Description	Default
`API_KEYS`	Comma-separated API keys	`demo-key-123`
`PORT`	Server port	`8000`

Adding API Keys

Edit .env file:

API_KEYS=student-key-1,student-key-2,instructor-key

For production, use secure key generation:

python -c "import secrets; print(secrets.token_urlsafe(32))"

Rate Limiting

Currently no rate limiting (development mode). For production, add middleware for:

Requests per minute per API key
Concurrent request limits
Monthly quota

Error Handling

Error Response Format

{
  "detail": "Error message",
  "platform": "tiktok",
  "url": "https://..."
}

Common Errors

Status Code	Meaning
401	Invalid or missing API key
400	Invalid request (malformed URL, etc.)
404	Platform not supported
500	Extraction failed
504	Extraction timeout (>60s)

Contributing

Fork the repository
Create feature branch (git checkout -b feature/amazing-feature)
Commit changes (git commit -m 'Add amazing feature')
Push to branch (git push origin feature/amazing-feature)
Open Pull Request

License

MIT License - see LICENSE file for details

Related Projects

content-engine - Full-stack content management system
tiktok-extractor - Standalone TikTok extractor
article-extractor - Standalone article extractor

Support

Documentation: docs/
Issues: GitHub Issues
Discussions: GitHub Discussions

Built with ❤️ for students and developers

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
docs/examples		docs/examples
extractors		extractors
gateway		gateway
.env.example		.env.example
.gitignore		.gitignore
DEPLOYMENT.md		DEPLOYMENT.md
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
nixpacks.toml		nixpacks.toml
quick-start.sh		quick-start.sh
railway.json		railway.json

prompt-stack/extractor-api

Folders and files

Latest commit

History

Repository files navigation