Skip to content

Apply plans against Neutron with tagging, concurrency, and metrics#9

Merged
berendt merged 6 commits into
mainfrom
implement/issue-4-apply-executor
Jun 24, 2026
Merged

Apply plans against Neutron with tagging, concurrency, and metrics#9
berendt merged 6 commits into
mainfrom
implement/issue-4-apply-executor

Conversation

@berendt

@berendt berendt commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Closes #4

Implements the apply engine that turns a generated plan into a real Neutron topology and records how long every operation takes. Built bottom-up across six commits: a leaf metrics package, the gophercloud wrappers that feed it, the dependency-ordered executor on top, then the CLI wiring and docs.

Change set (commit order)

ab841d0 Add internal/metrics samples, collector, and aggregation
The leaf package the wrappers feed. Per-call Samples (type, op, duration, success, HTTP status, error kind) and time-to-ready Readiness records accumulate in a mutex-guarded Collector that is safe for the executor's concurrent workers. Aggregate computes overall and per-type counts, nearest-rank latency percentiles, throughput over wall-clock, an error breakdown by kind, and per-type readiness stats. The percentile math is unit-tested against a pinned input plus the empty-collector edge case.

05eda5f Add internal/neutron gophercloud wrappers with tagging
The ports-and-adapters seam to OpenStack: one create wrapper per Phase 1 resource kind (address scopes, subnet pools, networks, subnets, routers + interfaces, security groups + rules, ports), each applying a deterministic name and run tags. A cross-kind Get/Delete and a WaitForReady (capped backoff) support readiness polling and later cleanup. Every call is timed into a metrics.Collector. Errors are classified for the executor: IsRetryable covers 5xx/429/non-quota-409, and quota rejections are wrapped with the ErrQuota sentinel (preserving the response error in the chain) so the run can fail fast. Pure helpers and the classifier are unit-tested; an integration test mirrors internal/config.

bec5504 Add dependency-ordered concurrent executor with retry
Apply walks the plan as ordered stages — address scopes, subnet pools, networks, routers, security groups, subnets, rules, router interfaces, ports — resolving each kind's references from the ids prior stages produced. Within a stage, independent resources are created through a bounded worker pool capped at --concurrency, so a large plan cannot spawn unbounded goroutines. Each create retries on transient 5xx/429/409 with exponential backoff, bounded by the per-operation timeout; a quota error fails fast via errors.Is(ErrQuota) without retry, and context cancellation returns ctx.Err(). Created resources reach ready (via WaitForReady) before the next stage. A fake Neutron drives race-clean unit tests for ordering, the concurrency bound, transient retry, fail-fast quota, and cancellation.

d9b0b8a Wire neutron apply to run executor and print metrics
Implements the non-dry-run path of neutron apply: generate a run id, authenticate, build a neutron.Client bound to that id and a collector, and run executor.Apply. The timing aggregate is printed even on partial failure so a run is never silent, and SIGINT/SIGTERM cancel cleanly through the context. The cloud client is constructed only after the dry-run early return, so --dry-run still makes zero API calls. The not-implemented test is replaced with one proving the non-dry-run path attempts execution and fails hermetically when no cloud is configured.

2e4a2f6 Update README status for apply executor and metrics
Marks the resource wrappers, the executor, and metrics collection with state polling as done; notes that run records, reporting, tag-based cleanup, and the quota pre-check remain.

965a688 Align neutron integration-test skip with config test
Gates the wrappers integration test on OS_CLOUD and fails on a client error, mirroring internal/config's integration test exactly, instead of skipping on any client-creation error.

Divergences from the issue

  • Resource naming. The issue specified ostester-<id>-<type>-NNNN. The implementation uses ostester-<id>-<logical>, where <logical> is the plan's logical name — which already encodes type and index deterministically — rather than re-deriving an NNNN counter the plan already provides. Net naming is still deterministic and run-scoped.
  • Tagging. The issue called for ostester:run=<id> on every created resource. The implementation applies that tag plus an additional ostester:type=<kind> tag, enabling type-scoped queries and cleanup later.

Not in this slice (called out as remaining in the README)

Run records, reporting, tag-based cleanup, and the quota pre-check are deliberately out of scope here and tracked as remaining work.


Implemented by planwerk-review a24f4d9 with Claude:claude-opus-4-8

berendt added 6 commits June 24, 2026 11:46
Introduce the leaf metrics package that the Neutron wrappers feed:
per-call Samples (type, op, duration, success, HTTP status, error
kind) and time-to-ready Readiness records, accumulated by a
mutex-guarded Collector that is safe for the executor's concurrent
workers.

Aggregate computes overall and per-type counts, nearest-rank latency
percentiles, throughput over wall-clock, an error breakdown by kind,
and per-type readiness stats. The pure percentile math is unit-tested
against a pinned input, alongside the empty-collector edge case.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Add the ports-and-adapters seam to OpenStack: one create wrapper per
Phase 1 resource kind (address scopes, subnet pools, networks, subnets,
routers and interfaces, security groups and rules, ports), each applying
a deterministic ostester-<id>-<logical> name and the ostester:run=<id>
and ostester:type=<kind> tags. A cross-kind Get and Delete support
readiness polling and later cleanup, and WaitForReady measures
time-to-ready for status-bearing kinds with capped backoff.

Every call is timed into a metrics.Collector. Errors are classified for
the executor: IsRetryable covers 5xx/429/non-quota-409, and quota
rejections are wrapped with the ErrQuota sentinel (keeping the response
error in the chain) so the run can fail fast. Pure helpers and the
classifier are unit-tested; an integration test mirrors internal/config,
creating one of each kind, asserting the run tag, and tearing down.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Apply walks the plan as ordered stages — address scopes, subnet pools,
networks, routers, security groups, subnets, rules, router interfaces,
ports — resolving each kind's references from the ids the prior stages
produced. Within a stage, independent resources are created through a
bounded worker pool capped at --concurrency, so a large plan cannot
spawn an unbounded number of goroutines.

Each create is retried on transient 5xx/429/409 errors with exponential
backoff and bounded by the per-operation timeout; a quota error fails the
run fast via errors.Is(ErrQuota) without retry, and context cancellation
returns ctx.Err(). Created resources become ready before the next stage
through WaitForReady. A fake Neutron drives unit tests for dependency
ordering, the concurrency bound, transient retry, fail-fast quota, and
cancellation; the suite is race-clean.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Implement the non-dry-run path of "neutron apply": generate a run id,
authenticate, build a neutron.Client bound to that id and a metrics
collector, and run executor.Apply to create the full tagged topology in
dependency order. The collected timing aggregate is printed even on
partial failure so a run is never silent, and SIGINT/SIGTERM cancel the
run cleanly through the context.

The cloud client is constructed only after the dry-run early return, so
--dry-run still makes zero API calls. The not-implemented test is
replaced with one proving the non-dry-run path attempts execution and
fails hermetically when no cloud is configured; the neutron namespace
doc comment now reflects apply being implemented.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Mark the Neutron resource wrappers, the dependency-ordered concurrent
executor, and metrics collection with state polling as done, and refine
the status block to note that run records, reporting, tag-based cleanup,
and the quota pre-check remain.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Gate the wrappers integration test on OS_CLOUD and fail on a client
error, mirroring internal/config's integration test exactly, instead of
skipping on any client-creation error.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
@berendt
berendt marked this pull request as ready for review June 24, 2026 10:18
@berendt
berendt merged commit ef89889 into main Jun 24, 2026
2 checks passed
@berendt
berendt deleted the implement/issue-4-apply-executor branch June 24, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Apply plans against Neutron with tagging, concurrency, and metrics

1 participant