Replies: 3 comments
-
|
Hi @sukki37! I'm Dosu and I’m helping the MinerU team. This is a known category of crash. The "corrupted double-linked list" abort under concurrency has been reported by multiple users and stems from thread-unsafe C/C++ libraries (specifically PDFium/pypdfium2 and CUDA resources) that cannot be protected by Python-level locks [1] [2]. To answer your questions directly:
# Option A: multiple remote instances
mineru-router \
--upstream-url http://server1:8003 \
--upstream-url http://server2:8004
# Option B: auto-managed local GPUs
mineru-router --local-gpus autoFor a single A800, you could also run multiple local workers on the same GPU via As additional context: maintainers have indicated that fully async pipeline support is planned for a future version [3], and several past releases have incrementally improved concurrency — e.g., v1.3.0 replaced paddle with paddleocr2torch for thread safety, and v2.6.4 added To reply, just mention @dosu. Share context across your team and agents. Try Dosu.
|
Beta Was this translation helpful? Give feedback.
-
|
I tested uploading two PDF documents at the same time and calling the pipeline backend twice, but I was unable to reproduce the abnormal situation. |
Beta Was this translation helpful? Give feedback.
-
|
@sukki37 The "corrupted double-linked list" crash with To answer your questions directly:
# Auto-managed local workers (one per GPU)
mineru-router --local-gpus auto
# Or explicitly point to multiple instances
mineru-router \
--upstream-url http://localhost:8003 \
--upstream-url http://localhost:8004For a single A800, you can run multiple workers on the same GPU via Note that the maintainer @myhloli indicated above they were unable to reproduce the crash with concurrency=3 on their setup, so this may be sensitive to specific hardware configurations, CUDA driver versions, or timing conditions. Your combination of To reply, just mention @dosu. Docs are dead. Just use Dosu.
|
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Description
We are seeing a reproducible native crash when running MinerU 3.1.4 with the pipeline backend through
mineru-apiand the async/tasksAPI under server-side concurrency 2.This looks similar to the locked issue #3106, but we are seeing it on MinerU 3.1.4.
Environment
pipelinemineru-api+/tasksasync APIMINERU_MODEL_SOURCE=modelscopeMINERU_API_MAX_CONCURRENT_REQUESTS=2Startup command
The service starts successfully and logs:
Reproduction
mineru-apiwithMINERU_API_MAX_CONCURRENT_REQUESTS=2./tasks./tasks/{task_id}and fetch/tasks/{task_id}/result.The PDFs used in the test are small research-report PDFs, around 3 pages each.
Key server log
The crash ends with:
Relevant log sequence before the crash includes:
What we checked
/tasks, not synchronous/file_parse.dmesg/journalctlshow no OOM killed event.Questions
pipelinebackend expected to be safe withMINERU_API_MAX_CONCURRENT_REQUESTS > 1inside a singlemineru-apiprocess?mineru-apiinstances withMAX_CONCURRENT_REQUESTS=1behindmineru-routeror an external round-robin/load balancer?Thanks!
Beta Was this translation helpful? Give feedback.
All reactions