v2.0.1

ThinkParse 2.0.1

Patch release on the 2.0 line. The reconciler now fills configured MinerU slots in one cycle instead of submitting one job at a time.

Highlights

• Parallel polling and submission, bounded by THINKPARSE_SLOTS and a 32-thread cap
• Each claim records its target MinerU URL before upload, so concurrent dispatch stays within each address's slots
• Chinese deployment guide, security and operations notes

Upgrade

git fetch origin
git checkout v2.0.1
docker compose --env-file .env -f docker/docker-compose.yml build --no-cache gateway reconciler
docker compose --env-file .env -f docker/docker-compose.yml up -d --force-recreate gateway reconciler

GET /api/v1/health reports version 2.0.1. No database migration.

Docs

• Changelog
• Deployment

v2.0.0

ThinkParse 2.0.0

Major release: enterprise document parsing rebuilt around MinerU 4.0, with durable orchestration instead of the 1.x Celery/Redis stack.

Highlights

• MinerU 4.0 integration (optional Docling for selected non-PDF formats)
• Persistent jobs in PostgreSQL; sources/artifacts on a local directory or external S3 for multi-host setups
• Gateway + reconciler (no Celery / Redis task bus)
• Deploy modes: gpu, cpu, external — multi-GPU and distributed-friendly
• Public /api/v1 (async tasks, for existing clients) and /api/v2 (uploads, jobs, files, tiers, stats, batches)
• Stable projection for content_list / middle_json

Breaking changes vs 1.x

2.0 is a different architecture from 1.x (MinerU 3.x + Celery/Redis). Treat this as a new deployment line, not an in-place upgrade:

	1.x	2.0
Engine	MinerU 3.x	MinerU 4.0
Orchestration	Celery + Redis	Gateway + reconciler + PostgreSQL
Branch	1.x	main
Latest tag	v1.4.2	v2.0.0

/api/v1 task submit/poll remains available for existing async clients; verify against your environment before cutting over.

Install / deploy

git clone https://github.com/wzdavid/ThinkParse.git
cd ThinkParse
git checkout v2.0.0
cp .env.example .env
# COMPOSE_PROFILES=gpu | cpu | (empty → external MinerU)

docker compose --env-file .env -f docker/docker-compose.yml up -d --build
curl -fsS http://127.0.0.1:8000/api/v1/health/live

Still need 1.x (MinerU 3.x)?

git clone -b 1.x https://github.com/wzdavid/ThinkParse.git
# or: git checkout v1.4.2

Docs

• README
• Changelog
• Deployment
• API

Full changelog

See CHANGELOG.md.

v1.4.3

ThinkParse 1.4.3 (1.x / MinerU 3.x)

Maintenance release for the 1.x line.

Fixed

• All-in-one images no longer crash on Python 3.12 with ModuleNotFoundError: No module named 'asynchat'. Debian apt supervisor 4.2.1 still imports removed stdlib modules; GPU and CPU all-in-one Dockerfiles now install supervisor>=4.2.5 via pip and start that binary.

Install / rebuild

git clone -b 1.x https://github.com/wzdavid/ThinkParse.git
cd ThinkParse
git checkout v1.4.3
cd docker && sh build.sh --allinone

Version lines

Line	Branch	Engine	Latest tag
2.0 (current)	main	MinerU 4.0	v2.0.0
1.x (maintenance)	1.x	MinerU 3.x	v1.4.3

This release is not marked Latest; v2.0.0 remains the default GitHub Latest release.

v1.4.2

Stop local parse storage from filling the disk and halting Redis.

Highlights

• Free-space watermarks on the filesystem that holds temp and output data. Defaults keep an absolute reserve (16 GiB reclaim, 8 GiB floor), capped to a percentage of small disks
• Cleanup runs as soon as the process starts, then checks disk headroom on DISK_CHECK_INTERVAL_MINUTES even when CLEANUP_INTERVAL_HOURS is long
• New submissions return HTTP 507, and queued tasks wait, while free space is under the reclaim line
• /health/ready reports storage unavailable while headroom is low; liveness does not restart the API for a full disk
• Pressure deletion skips symlinks and paths outside the configured directories, and never removes data younger than TASK_TIME_LIMIT

Upgrade notes

• Pull v1.4.2 and recreate API, Worker, and cleanup containers. Those directories are bind-mounted, so docker compose restart does not pick up the new code or a changed cleanup command
• If docker/.env still has CLEANUP_INTERVAL_HOURS=24, set it to 6. The old value no longer delays the first disk check, but age-based cleanup would still run only once a day
• New DISK_FREE_* settings are optional. Defaults apply when they are unset
• To keep outputs for about 12 hours, set RESULT_EXPIRES=43200 and CLEANUP_EXTRA_HOURS=0. For about 8 hours, use RESULT_EXPIRES=28800. RESULT_EXPIRES must stay greater than BROKER_VISIBILITY_TIMEOUT_SECONDS
• /health/deep reports version 1.4.2

See CHANGELOG.md.

v1.4.1

Productized multi-GPU and multi-worker deployment. Operators set card count and workers per card; Compose service blocks no longer need to be copied by hand.

Highlights

• Add docker/gpu-up.sh and docker/render-gpu-workers.py to generate docker-compose.gpus.yml from MINERU_GPU_COUNT and MINERU_WORKERS_PER_GPU
• Start all generated workers with profile mineru-multi-gpu; use MINERU_GPU_WORKER_ONLY=1 on dedicated GPU hosts
• Keep GPU_WORKER_CONCURRENCY=1; scale by adding containers on the same card
• Stop stale GPU workers (legacy names and extra slots) after Compose validation, before starting the new set
• Validate the renderer in CI

Upgrade notes

• Multi-GPU hosts: set MINERU_GPU_COUNT / MINERU_WORKERS_PER_GPU in docker/.env, then cd docker && sh gpu-up.sh
• Do not also enable profile mineru-gpu alongside the generated workers
• docker-compose.multi-gpu.yml remains as the legacy one-container-per-card template; do not run it beside gpu-up.sh
• ThinkExtract: set INGESTION_MINERU_MAX_INFLIGHT ≈ worker total (or slightly higher) and keep backfill replicas above that limit
• /health/deep now reports version 1.4.1

See CHANGELOG.md and docker/README.md for details.

v1.4.0

Single-container deployment for schedulers that accept only one container (Tianhe / HPC). The four-container Compose stack remains the default.

Highlights

• Add GPU and CPU all-in-one images (mineru-allinone, mineru-allinone-cpu)
• Run Redis, API, Worker, and cleanup in one container under supervisord
• Force in-container Redis at 127.0.0.1:6379 so a copied four-container .env cannot break DNS
• Add docker-compose.allinone.yml and build.sh --allinone / --allinone-cpu
• Add English and Chinese single-container deployment guides
• Validate all-in-one entrypoint, supervisor programs, and Docker files in CI

Upgrade notes

• Four-container Compose deployments do not need to change
• Tianhe / single-container schedulers should build mineru-allinone and submit one replica on port 8000
• Mount /tmp/mineru_output and /tmp/mineru_temp; mount /data/redis if the queue must survive restarts
• Do not set REDIS_URL=redis://redis:6379/0 for the all-in-one image
• /health/deep now reports version 1.4.0

See CHANGELOG.md and single-container deployment for details.

v1.3.0

Production stability and observability upgrade with MinerU 3.4.5, native long-document processing, cancellable engine isolation, and layered health diagnostics.

Highlights

• Upgrade the parsing engine to MinerU 3.4.5
• Reuse MinerU models in a persistent isolated process while keeping active parses cancellable
• Use MinerU's native bounded processing windows for long PDFs by default
• Add liveness, readiness, and detailed operational health endpoints
• Report Redis queue depth, Worker heartbeats, task runtime, GPU utilization, memory, temperature, and engine state
• Add engine timeout recovery and a Worker watchdog for stalled tasks
• Default to one active MinerU task per GPU for safer production operation
• Refresh the English and Chinese READMEs with a clearer quick start and deployment guidance

Reliability and compatibility

• Move blocking Celery waits and inspections off the FastAPI event loop
• Persist cancellation requests in Redis and safely terminate the isolated parsing engine
• Restart the engine automatically after cancellation, timeout, crash, or parse failure
• Probe configured S3 buckets during readiness checks with bounded waits
• Keep existing task submission and result response shapes compatible
• Retain physical PDF splitting only as a deprecated compatibility fallback

Upgrade notes

• Compare your existing .env with .env.example; repository updates do not modify existing environment files
• Rebuild the MinerU base and Worker images: cd docker && sh build.sh --worker-gpu --rebuild-base
• Restart API, Worker, and cleanup containers so every service runs the new version
• Keep WORKER_CONCURRENCY=1 per GPU and scale by adding GPU workers
• Ensure WORKER_WATCHDOG_TIMEOUT_SECONDS is greater than MINERU_ENGINE_TIMEOUT_SECONDS, and BROKER_VISIBILITY_TIMEOUT_SECONDS is greater than the watchdog timeout
• Cancellation now returns cancel_requested; poll until the task reaches cancelled or another terminal state
• Restrict /api/v1/health/deep to trusted operators because it exposes detailed deployment diagnostics

See CHANGELOG.md for the complete change history.

v1.2.0

Production hardening: keep Redis as a slim Celery queue (no parse bodies), isolate Redis disk from temp/output, shorten temp cleanup defaults, and document single-host multi-GPU plus multi-node scale-out.

Added

• shared/task_result.py: slim Celery/Redis results; hydrate markdown / images[] / JSON from storage for the status API (response shape unchanged for ThinkDoc / ThinkExtract)
• docker/docker-compose.multi-gpu.yml: one worker per GPU (mineru-gpu-0 … mineru-gpu-7)
• docker/docker-compose.worker-only.yml: disable local API/cleanup on dedicated GPU hosts
• Docs: multi-node production guide (docs/PRODUCTION_MULTI_NODE.md / .zh.md); Redis REDIS_DATA_PATH guidance in deployment docs
• TEMP_MAX_AGE_HOURS (default 6) wired through cleanup scheduler / Compose

Changed

• Celery task results no longer store full markdown, base64 images[], or inline content_list / middle_json in Redis (AOF/memory grow only with queue metadata)
• Default cleanup interval CLEANUP_INTERVAL_HOURS: 24 -> 6; temp orphan age: 24 -> 6 (above TASK_TIME_LIMIT=2h)
• Docker Redis volume can bind via REDIS_DATA_PATH (keep off the temp/output disk)

Fixed

• S3 download_to_local: preserve remote file suffix so get_file_type() still selects MinerU for PDFs/images

Upgrade notes

• Rebuild/restart API + Worker + cleanup images/containers so slim-result + hydrate code is live
• Production: set REDIS_DATA_PATH on a disk separate from mineru_temp / mineru_output
• Multi-GPU single host: COMPOSE_FILE=docker-compose.yml:docker-compose.multi-gpu.yml and profiles mineru-gpu-0,... (do not also enable mineru-gpu)
• Multi-node GPU hosts: also load docker-compose.worker-only.yml; point REDIS_URL / S3 at shared services
• Configure S3 temp bucket lifecycle when using MINERU_STORAGE_TYPE=s3
• Public HTTP API fields are unchanged; clients need no code change

v1.1.0

ThinkParse v1.1.0

This release rebrands the project as ThinkParse (part of the Think product family) and upgrades the default parsing engine to MinerU 3.4.0.

Changed

• Rebrand project from MinerU-API to ThinkParse (wzdavid/ThinkParse)
• Docker service and image names (mineru-api, mineru-worker, etc.) are unchanged for deployment compatibility
• Upgrade MinerU parsing engine from 2.x to 3.4.0 (mineru[pipeline]==3.4.0)
• Pin mineru[core]==3.4.0 in the GPU base Docker image for version consistency
• Update README third-party license for MinerU (Apache 2.0-based MinerU Open Source License, formerly AGPL-3.0)

Fixed

• Fix KeyError on MINERU_DEVICE_MODE when MinerU 3.4.0 spawns subprocesses that re-import worker/tasks.py (use os.environ.pop instead of del)

Upgrade notes

1. Rebuild Worker Docker images after upgrading:

   cd docker && sh build.sh

2. Always run Docker Compose from the docker/ directory:

   cd docker && docker compose up -d

3. MinerU 3.4.0 includes new models (e.g. PP-OCRv6); models are downloaded automatically during image build

Full changelog

See CHANGELOG.md.