A FastAPI backend with Celery asynchronous task queue that extracts structured JSON data from PDF documents using OpenAI's GPT-4o Vision API. It is designed to be highly reliable, featuring exponential backoff retries, SHA-256 file hashing for deduplication, and polling endpoints for frontend apps to consume.
- FastAPI: High-performance HTTP server providing predictable REST endpoints.
- Celery & Redis: Background job processing so the main thread never blocks when calling external ML APIs.
- PostgreSQL: Stores job statuses, extracted JSON payload, and timestamps.
- GPT-4o Vision: Multi-modal model used to understand document structure perfectly via
pdf2image. - Deduplication: SHA-256 hash checking prevents redundant API calls for exactly the same document.
POST /documents/upload- Start a new extraction background task. Returns HTTP 202 and ajob_id.GET /documents/{job_id}/status- Check the status (queued,processing,done,failed).GET /documents/{job_id}/result- Get the final structured JSON extraction.GET /documents/- Paginated history table endpoint.DELETE /documents/{job_id}- Remove a job.
- Copy
.env.exampleto.envand configure your API keys:cp .env.example .env
- Set your
OPENAI_API_KEYin.env. The API validates keys at runtime. - Start the Docker Compose stack:
docker compose up -d --build
A web dashboard will be available at http://localhost:5173. Swagger UI is accessible at http://localhost:8000/docs. By default, polling is handled gracefully via React useQuery every 2 seconds until the backend reports done or failed.