Skip to content

feat: add batch document processing command#26

Open
wydrox wants to merge 1 commit intomainfrom
feat/batch-document-processing
Open

feat: add batch document processing command#26
wydrox wants to merge 1 commit intomainfrom
feat/batch-document-processing

Conversation

@wydrox
Copy link
Contributor

@wydrox wydrox commented Mar 26, 2026

Summary

  • Adds ppmlx process <dir> CLI command for batch LLM processing of document directories
  • New ppmlx/processor.py module with file discovery, task definitions (summarize, translate, extract_entities, classify, custom prompts), checkpoint/resume, and concurrent batch processing via httpx
  • Rich progress bars, --dry-run preview, --pipe for stdout output, --resume for failure recovery via checkpoint file, configurable --parallel workers

Test plan

  • 46 new tests in tests/test_processor.py covering file discovery, task definitions, checkpoint save/load/resume, single-file processing (success, connection error, timeout, API error, unreadable file), batch processing (sequential, parallel, resume, output files, progress callback, checkpoint updates), CLI integration (help, invalid args, dry-run, custom task)
  • Full test suite (221 tests) passes with no regressions
  • End-to-end: ppmlx process ./docs --task summarize --model qwen3.5 --dry-run shows file listing

🤖 Generated with Claude Code

Process entire directories of documents with LLM tasks (summarize,
translate, extract_entities, classify, or custom prompts). Calls the
local ppmlx API via httpx with Rich progress bars, concurrent workers,
checkpoint-based resume, dry-run preview, and stdout piping support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector
Copy link

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant