Skip to content

Improve metagen multiprocessing pool lifecycle#69

Merged
camrossi merged 1 commit intodatacenter:mainfrom
asingiri:asingiri-patch-1
Apr 8, 2026
Merged

Improve metagen multiprocessing pool lifecycle#69
camrossi merged 1 commit intodatacenter:mainfrom
asingiri:asingiri-patch-1

Conversation

@asingiri
Copy link
Copy Markdown
Contributor

@asingiri asingiri commented Apr 1, 2026

Summary bullets:

  • refactor metadata generation to use a managed multiprocessing pool context
  • cap worker process count to 6 to reduce host contention while preserving parallelism
  • set maxtasksperchild and materialize pool output before writing aci-meta.json

@asingiri asingiri requested a review from camrossi April 1, 2026 05:47
@camrossi camrossi merged commit 99395b8 into datacenter:main Apr 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants