Apply plans against Neutron with tagging, concurrency, and metrics#9
Merged
Conversation
Introduce the leaf metrics package that the Neutron wrappers feed: per-call Samples (type, op, duration, success, HTTP status, error kind) and time-to-ready Readiness records, accumulated by a mutex-guarded Collector that is safe for the executor's concurrent workers. Aggregate computes overall and per-type counts, nearest-rank latency percentiles, throughput over wall-clock, an error breakdown by kind, and per-type readiness stats. The pure percentile math is unit-tested against a pinned input, alongside the empty-collector edge case. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
Add the ports-and-adapters seam to OpenStack: one create wrapper per Phase 1 resource kind (address scopes, subnet pools, networks, subnets, routers and interfaces, security groups and rules, ports), each applying a deterministic ostester-<id>-<logical> name and the ostester:run=<id> and ostester:type=<kind> tags. A cross-kind Get and Delete support readiness polling and later cleanup, and WaitForReady measures time-to-ready for status-bearing kinds with capped backoff. Every call is timed into a metrics.Collector. Errors are classified for the executor: IsRetryable covers 5xx/429/non-quota-409, and quota rejections are wrapped with the ErrQuota sentinel (keeping the response error in the chain) so the run can fail fast. Pure helpers and the classifier are unit-tested; an integration test mirrors internal/config, creating one of each kind, asserting the run tag, and tearing down. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
Apply walks the plan as ordered stages — address scopes, subnet pools, networks, routers, security groups, subnets, rules, router interfaces, ports — resolving each kind's references from the ids the prior stages produced. Within a stage, independent resources are created through a bounded worker pool capped at --concurrency, so a large plan cannot spawn an unbounded number of goroutines. Each create is retried on transient 5xx/429/409 errors with exponential backoff and bounded by the per-operation timeout; a quota error fails the run fast via errors.Is(ErrQuota) without retry, and context cancellation returns ctx.Err(). Created resources become ready before the next stage through WaitForReady. A fake Neutron drives unit tests for dependency ordering, the concurrency bound, transient retry, fail-fast quota, and cancellation; the suite is race-clean. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
Implement the non-dry-run path of "neutron apply": generate a run id, authenticate, build a neutron.Client bound to that id and a metrics collector, and run executor.Apply to create the full tagged topology in dependency order. The collected timing aggregate is printed even on partial failure so a run is never silent, and SIGINT/SIGTERM cancel the run cleanly through the context. The cloud client is constructed only after the dry-run early return, so --dry-run still makes zero API calls. The not-implemented test is replaced with one proving the non-dry-run path attempts execution and fails hermetically when no cloud is configured; the neutron namespace doc comment now reflects apply being implemented. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
Mark the Neutron resource wrappers, the dependency-ordered concurrent executor, and metrics collection with state polling as done, and refine the status block to note that run records, reporting, tag-based cleanup, and the quota pre-check remain. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
Gate the wrappers integration test on OS_CLOUD and fail on a client error, mirroring internal/config's integration test exactly, instead of skipping on any client-creation error. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Christian Berendt <berendt@b42labs.com>
berendt
marked this pull request as ready for review
June 24, 2026 10:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4
Implements the apply engine that turns a generated plan into a real Neutron topology and records how long every operation takes. Built bottom-up across six commits: a leaf metrics package, the gophercloud wrappers that feed it, the dependency-ordered executor on top, then the CLI wiring and docs.
Change set (commit order)
ab841d0Add internal/metrics samples, collector, and aggregationThe leaf package the wrappers feed. Per-call
Samples (type, op, duration, success, HTTP status, error kind) and time-to-readyReadinessrecords accumulate in a mutex-guardedCollectorthat is safe for the executor's concurrent workers.Aggregatecomputes overall and per-type counts, nearest-rank latency percentiles, throughput over wall-clock, an error breakdown by kind, and per-type readiness stats. The percentile math is unit-tested against a pinned input plus the empty-collector edge case.05eda5fAdd internal/neutron gophercloud wrappers with taggingThe ports-and-adapters seam to OpenStack: one create wrapper per Phase 1 resource kind (address scopes, subnet pools, networks, subnets, routers + interfaces, security groups + rules, ports), each applying a deterministic name and run tags. A cross-kind
Get/Deleteand aWaitForReady(capped backoff) support readiness polling and later cleanup. Every call is timed into ametrics.Collector. Errors are classified for the executor:IsRetryablecovers 5xx/429/non-quota-409, and quota rejections are wrapped with theErrQuotasentinel (preserving the response error in the chain) so the run can fail fast. Pure helpers and the classifier are unit-tested; an integration test mirrorsinternal/config.bec5504Add dependency-ordered concurrent executor with retryApplywalks the plan as ordered stages — address scopes, subnet pools, networks, routers, security groups, subnets, rules, router interfaces, ports — resolving each kind's references from the ids prior stages produced. Within a stage, independent resources are created through a bounded worker pool capped at--concurrency, so a large plan cannot spawn unbounded goroutines. Each create retries on transient 5xx/429/409 with exponential backoff, bounded by the per-operation timeout; a quota error fails fast viaerrors.Is(ErrQuota)without retry, and context cancellation returnsctx.Err(). Created resources reach ready (viaWaitForReady) before the next stage. A fake Neutron drives race-clean unit tests for ordering, the concurrency bound, transient retry, fail-fast quota, and cancellation.d9b0b8aWire neutron apply to run executor and print metricsImplements the non-dry-run path of
neutron apply: generate a run id, authenticate, build aneutron.Clientbound to that id and a collector, and runexecutor.Apply. The timing aggregate is printed even on partial failure so a run is never silent, and SIGINT/SIGTERM cancel cleanly through the context. The cloud client is constructed only after the dry-run early return, so--dry-runstill makes zero API calls. The not-implemented test is replaced with one proving the non-dry-run path attempts execution and fails hermetically when no cloud is configured.2e4a2f6Update README status for apply executor and metricsMarks the resource wrappers, the executor, and metrics collection with state polling as done; notes that run records, reporting, tag-based cleanup, and the quota pre-check remain.
965a688Align neutron integration-test skip with config testGates the wrappers integration test on
OS_CLOUDand fails on a client error, mirroringinternal/config's integration test exactly, instead of skipping on any client-creation error.Divergences from the issue
ostester-<id>-<type>-NNNN. The implementation usesostester-<id>-<logical>, where<logical>is the plan's logical name — which already encodes type and index deterministically — rather than re-deriving anNNNNcounter the plan already provides. Net naming is still deterministic and run-scoped.ostester:run=<id>on every created resource. The implementation applies that tag plus an additionalostester:type=<kind>tag, enabling type-scoped queries and cleanup later.Not in this slice (called out as remaining in the README)
Run records, reporting, tag-based cleanup, and the quota pre-check are deliberately out of scope here and tracked as remaining work.
Implemented by planwerk-review a24f4d9 with Claude:claude-opus-4-8