Pre-production fixes and test added

Production-readiness release. Thresholds, weights and word lists are unchanged and batch output is identical, so DOC_LINE_CATEG_1210 stays valid.

Changed for API callers

• /process and /score_record now run the CPU pre-filter first, as the batch pipeline does. Numeric, code-like and blank lines come back Non-text or Empty (score 0.0, language N/A) without a model call; every other line is scored on its repaired text. Such lines that were answered Clear, Noisy or Trash before change category. A page's score averages Clear, Noisy and Trash lines only.
• PPL_BATCH_LINES defaults to 32. A .env copied from an older .env.example still sets 128.
• GPU work is serialised per process, and GET /health?deep=true fails while no model is loaded.
• Paradata names the perplexity model and layoutlmv3 when they ran, so the recorded licence no longer understates an API run. /score_record takes its page and line caps from the configuration.
• A JSON or text upload without page onto a multi-page record is skipped with a warning instead of landing on page 1.

Reliability

• A perplexity batch that runs out of GPU memory is split and retried; only a line that fails on its own scores 99999. Values are unchanged.
• The ordering of CATEG_TRASH_SCORE_MAX < CATEG_NOISY_SCORE_MAX, SHORT_PPL_CAP < PERPLEXITY_THRESHOLD_MAX and the positive weight sum is checked at startup: the service refuses to start and classify_TEXT.py exits before loading a model.
• Per-document CSVs are written as <doc>.csv.part and renamed when complete, so a failed re-classification keeps the previous file. A leftover .part is safe to delete.
• Exit status is non-zero when run_pipeline.py aborts a stage, when classify_TEXT.py loses its GPU or fails every document, and when page_split.py finds no input directory. Orchestrators that treated these as success will now see them.
• Config files are read as UTF-8, and the Qwen-scale warning no longer fires on every run.

Packaging

• API-only installs list pandas and tqdm, and the torch spec matches the requirements.
• The security scan targets the API image.
• setup/setup_api_server.sh fetches FastText at the Dockerfile's pinned revision and checks its SHA-256.

Documentation and tests (#5)

• docs/categorization_logic.md is rebuilt from the code: order of evaluation, overrides with what blocks each, language tiers, the four document passes, foreign-language limits, worked examples.
• README Step 4 is restructured.
• tests/test_categorization_walkthrough.py follows a line through every step and pins the known limits.

Warning

No container images were published for this tag (its Docker run stopped at the test step). Use v1.12.1-beta: the same code, with images.

Full Changelog: v1.11.0-beta...v1.12.0-beta