perf(indexing): run independent capability groups concurrently - #143
Open
ik020 wants to merge 2 commits into
Open
perf(indexing): run independent capability groups concurrently#143ik020 wants to merge 2 commits into
ik020 wants to merge 2 commits into
Conversation
Parallelizes _run_enabled_modalities using ThreadPoolExecutor so independent capability groups (e.g. scene, speech) index concurrently instead of sequentially. Adds a lock to ManifestStore to make its read-modify-write methods safe under concurrent calls, and replaces the single shared 'stage' variable with a per-group tracked dict to avoid one group's status clobbering another's. GPU/CPU resource coordination between groups (ResourceScheduler) is left out of scope for this change and can be addressed as a follow-up if benchmarking shows contention is a real problem in practice. Benchmark (5 runs, simulated 0.40s/0.55s group latencies): sequential (theoretical): 0.950s concurrent (measured): 0.565s mean, stdev 0.003s speedup: 1.68x (theoretical ceiling 1.73x) Refs grayhatdevelopers#96
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Closes #96
Summary
Parallelizes
_run_enabled_modalitiesso independent capability groups (for example,sceneandspeech) can run concurrently instead of sequentially, usingThreadPoolExecutor.The implementation also tracks active stages per capability group so concurrent workers cannot overwrite a single shared stage value.
This PR adds regression coverage for concurrent execution and manifest stage preservation.
GPU/CPU resource coordination via
ResourceScheduleris intentionally left out of scope. It is not currently wired into the relevant capability indexers, so resource contention should be measured before introducing additional scheduling logic.No public interfaces or capability contracts are changed. The change is limited to the internal execution of independent capability groups.
Validation
PYTHONPATH=src python -m pytest tests/test_runner.py -vgit diff --checktest_concurrent_capability_groups_do_not_drop_stagesscene+speechindexing repeatedly and verifies that both stages are preserved in the manifest.test_capability_groups_actually_run_concurrentlythreading.Eventsynchronization to verify that the two capability groups actually overlap in execution rather than merely being dispatched independently.Benchmark
Added
benchmark_concurrency.pyto measure the orchestration behavior using simulated capability latency.With simulated latencies of 0.40s for
sceneand 0.55s forspeech:These numbers use artificial delays and do not represent real model-inference performance. The benchmark is intended to demonstrate the concurrency behavior and orchestration overhead. Real-world speedup will depend on model execution, CPU/GPU resources, and workload characteristics.