Add parallel file conversion support with ThreadPoolExecutor

[samuelduchesne]

Summary

This PR adds support for parallel file conversion in the EnergyPlus documentation build process. File conversions are now executed concurrently using a ThreadPoolExecutor when multiple workers are specified, improving build performance for large documentation sets.

Key Changes

• Parallel conversion infrastructure: Introduced _convert_files() function that uses ThreadPoolExecutor to run file conversions in parallel when max_workers > 1. The Pandoc subprocess calls are I/O-bound, making threads well-suited for this workload.

• Task collection refactoring: Extracted _collect_tasks() function to separate task building from execution, enabling better separation of concerns and cleaner parallel execution logic.

• Refactored convert_doc_set(): Split into two phases:

  • Phase 1: Parallel file conversions (when applicable)
  • Phase 2: Sequential result logging and TOC generation (must happen after files are written)

• CLI enhancements:

  • Added --max-workers argument to scripts/convert.py with default value of CPU count
  • Added --file-workers argument to scripts/convert_all.py to control per-version parallelism
  • Updated Makefile to support MAX_WORKERS environment variable

• API updates: Added max_workers keyword-only parameter to convert_doc_set() and convert_version() functions with sensible defaults.

Implementation Details

• The parallel execution only activates when max_workers > 1 AND there are multiple tasks, otherwise falls back to sequential processing
• TOC generation and result logging remain sequential to ensure proper file ordering and consistency
• Error handling and logging use file_result.source instead of tex_path for consistency across parallel and sequential paths
• The convert_all.py script can now control both version-level parallelism (multiple versions) and file-level parallelism (files within a version)

https://claude.ai/code/session_01XAaaYCg6rvfpWNsmWRxZen

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67950de69d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reduce default file-worker fanout in convert_all

--file-workers now defaults to os.cpu_count(), but convert_all already runs up to --max-workers processes (also defaulting to CPU count), so the default invocation can fan out to roughly cpu_count^2 concurrent file conversions and Pandoc subprocesses; on typical CI/dev machines this can cause heavy oversubscription, memory pressure, and build failures/timeouts instead of speeding up conversion. A safer default for per-version file workers (for example 1, or a value derived from max_workers) avoids breaking the default workflow.

Useful? React with 👍 / 👎.