Skip to content

Conversation

@CybotTM
Copy link
Contributor

@CybotTM CybotTM commented Jan 23, 2026

Summary

Adds core utilities for parallel documentation processing using pcntl_fork.

New Classes

Class Purpose
CpuDetector Cross-platform CPU core detection (Linux, macOS, BSD)
ProcessManager Forked process management with timeout and cleanup

Features

CpuDetector:

  • Detects cores via /proc/cpuinfo (Linux)
  • Falls back to nproc command (GNU/Linux)
  • Falls back to sysctl -n hw.ncpu (macOS/BSD)
  • Configurable max workers cap (default 8)

ProcessManager:

  • Non-blocking wait with configurable timeout (default 300s)
  • Automatic SIGKILL for stuck processes after timeout
  • Secure temp file creation with 0600 permissions
  • Signal handlers for cleanup on SIGTERM/SIGINT
  • Child process temp file tracking to prevent cleanup conflicts

Usage Example

// Detect optimal worker count
$workers = CpuDetector::detectCores(maxWorkers: 8);

// Fork and track children
$childPids = [];
for ($i = 0; $i < $workers; $i++) {
    $pid = pcntl_fork();
    if ($pid === 0) {
        // Child: clear inherited temp tracking
        ProcessManager::clearTempFileTracking();
        // ... do work ...
        exit(0);
    }
    $childPids[$i] = $pid;
}

// Wait with timeout
$result = ProcessManager::waitForChildrenWithTimeout($childPids);
// $result['successes'] = worker IDs that completed
// $result['failures'] = worker ID => error message

Test Coverage

  • CpuDetector: 4 tests covering core detection and max workers

Note: This is a WIP PR. The utilities are complete but integration with a parallel renderer is planned for a follow-up PR.

Test Plan

  • All existing tests pass
  • New unit tests for CpuDetector (4 tests)
  • Code style checks pass

Adds core utilities for parallel documentation processing:

- CpuDetector: Cross-platform CPU core detection (Linux, macOS, BSD)
- ProcessManager: Forked process management with timeout and cleanup

Features:
- Non-blocking wait with configurable timeout (default 300s)
- Automatic SIGKILL for stuck processes after timeout
- Secure temp file creation with 0600 permissions
- Signal handlers for cleanup on SIGTERM/SIGINT
- Child process temp file tracking to prevent cleanup conflicts

These utilities provide the foundation for parallel compilation and
rendering using pcntl_fork.

Includes unit tests for CpuDetector.
@CybotTM CybotTM marked this pull request as draft January 23, 2026 15:46
@CybotTM CybotTM changed the title [WIP] perf: Add parallel processing infrastructure perf: Add parallel processing infrastructure Jan 23, 2026
@CybotTM CybotTM marked this pull request as ready for review January 23, 2026 16:30
@jaapio
Copy link
Member

jaapio commented Jan 23, 2026

I'm sorry, I think this will add a lot of extra complexity to this project. I had this idea in mind for quit a while, and I do understand how this would drastically improve the speed of this project. But before we go this path I would love to learn more about how you think the new method would make sure we do not end up with race conditions. How do different processes share results etc.

This whole topic is very new to me, I'm open to learn new things but as I will be the one maintaining this, I need to have full understanding of this topic before we go this route.

@CybotTM
Copy link
Contributor Author

CybotTM commented Jan 23, 2026

Initially I thought I will provide the integration part with another PR, but does not make much sense, right?

Note: This is a WIP PR. The utilities are complete but integration with a parallel renderer is planned for a follow-up PR.

So I will add it here as additional commits.
And the bigger picture is still: https://cybottm.github.io/render-guides/ - render complete TYPO3 Core Changelog in under 1 minute.

@CybotTM CybotTM marked this pull request as draft January 24, 2026 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants