Repository navigation
v1.2.0
Production hardening: keep Redis as a slim Celery queue (no parse bodies), isolate Redis disk from temp/output, shorten temp cleanup defaults, and document single-host multi-GPU plus multi-node scale-out.
Added
shared/task_result.py: slim Celery/Redis results; hydrate markdown /images[]/ JSON from storage for the status API (response shape unchanged for ThinkDoc / ThinkExtract)docker/docker-compose.multi-gpu.yml: one worker per GPU (mineru-gpu-0…mineru-gpu-7)docker/docker-compose.worker-only.yml: disable local API/cleanup on dedicated GPU hosts- Docs: multi-node production guide (
docs/PRODUCTION_MULTI_NODE.md/.zh.md); RedisREDIS_DATA_PATHguidance in deployment docs TEMP_MAX_AGE_HOURS(default 6) wired through cleanup scheduler / Compose
Changed
- Celery task results no longer store full markdown, base64
images[], or inlinecontent_list/middle_jsonin Redis (AOF/memory grow only with queue metadata) - Default cleanup interval
CLEANUP_INTERVAL_HOURS: 24 -> 6; temp orphan age: 24 -> 6 (aboveTASK_TIME_LIMIT=2h) - Docker Redis volume can bind via
REDIS_DATA_PATH(keep off the temp/output disk)
Fixed
- S3
download_to_local: preserve remote file suffix soget_file_type()still selects MinerU for PDFs/images
Upgrade notes
- Rebuild/restart API + Worker + cleanup images/containers so slim-result + hydrate code is live
- Production: set
REDIS_DATA_PATHon a disk separate frommineru_temp/mineru_output - Multi-GPU single host:
COMPOSE_FILE=docker-compose.yml:docker-compose.multi-gpu.ymland profilesmineru-gpu-0,...(do not also enablemineru-gpu) - Multi-node GPU hosts: also load
docker-compose.worker-only.yml; pointREDIS_URL/ S3 at shared services - Configure S3 temp bucket lifecycle when using
MINERU_STORAGE_TYPE=s3 - Public HTTP API fields are unchanged; clients need no code change