Skip to content

Run monitor continuously by default (--interval optional, 0 = back-to-back)#28

Merged
berendt merged 3 commits into
mainfrom
implement/issue-27-continuous-monitor
Jul 2, 2026
Merged

Run monitor continuously by default (--interval optional, 0 = back-to-back)#28
berendt merged 3 commits into
mainfrom
implement/issue-27-continuous-monitor

Conversation

@berendt

@berendt berendt commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Closes #27

What changed

Monitor mode (neutron monitor) previously required --interval and rejected a zero or non-positive value. This branch makes --interval optional with a default of 0, which now means continuous: each iteration starts the moment the previous one finishes. Paced runs (--interval > 0) are unchanged.

The change lands in three commits:

275de60 — Run monitor continuously by default (--interval 0 = back-to-back)

  • cmd/openstack-tester/monitor.go: drop MarkFlagRequired("interval") so the flag is optional; its default was already 0. The flag usage string now documents 0 = continuous.
  • monitorConfig.validate() now rejects only a negative interval (< 0) instead of a non-positive one (<= 0); the error message changes from "must be positive" to "must not be negative". A zero interval is valid and means back-to-back.
  • Startup log: renders interval=continuous when the interval is 0, so the line stays self-explanatory; the value is otherwise cfg.interval.String().
  • runMonitorLoop: adds a minFailureBackoff (5s) floor on the post-failure wait. In continuous mode (interval and error-wait both 0), a fast-failing dependency — e.g. an unreachable Keystone that refuses auth in ~1ms — would otherwise hot-loop, pinning a CPU and flooding logs; the floor makes it back off. --error-wait remains the explicit brake.
  • Command/doc-comment wording updated to say "continuously or on a cadence".

018c967 — Default testbed-monitor to a continuous MONITOR_INTERVAL of 0

  • Makefile: MONITOR_INTERVAL default flips from 5m to 0, so make testbed-monitor runs iterations back-to-back and feeds the local OTEL stack a steady stream of data. MONITOR_INTERVAL=5m restores paced behaviour. The cadence echo annotates both zeros (interval 0 (0 = continuous)) so it stays readable.

7cffd7d — Document the continuous monitor default in README and dashboards

  • README.md: reword the command synopsis (--interval now optional), the cadence semantics, and the testbed-monitor walkthrough to lead with the new default; note that --error-wait is the brake in continuous mode.
  • contrib/otel/dashboards/*.json: the three sampling caveats no longer claim a 5m default, since metrics now flow steadily under the continuous default and only move per-iteration on a paced run.

Tests

cmd/openstack-tester/monitor_test.go gains coverage for the new semantics (continuous pacing, validate() accepting 0 and rejecting negatives, and the failure-backoff floor), all driven on the injected fake clock — no cloud required.

Divergence from the issue

  • The issue asked only to make --interval optional with 0 = back-to-back. The implementation additionally introduces a fixed 5s minFailureBackoff floor for failed iterations, to prevent a hot-loop when both --interval and --error-wait are 0. This is a safety guard implied by "continuous by default" rather than a literal ask.
  • Note for reviewers: commit 275de60's message says "the loop and clock need no changes", but that same commit does add the minFailureBackoff floor to runMonitorLoop (documented in the function's doc comment). The message narrates the primary mechanism; the guard is the one loop change that shipped alongside it.

Implemented by planwerk-agent 6b3f3ea with Claude:claude-opus-4-8

berendt added 3 commits July 2, 2026 19:12
Make --interval optional with a default of 0, which now means
continuous: the next iteration starts the moment the previous one
finishes. validate() rejects only negative intervals; the loop and
clock need no changes because runMonitorLoop already clamps the delay
to 0 and RealClock.Sleep treats a non-positive delay as a
cancellation-observing no-op. --error-wait still inserts its pause
after a failed iteration, so it remains the brake in continuous mode.

The startup log renders interval=continuous when zero so the line
stays self-explanatory; paced runs (--interval > 0) are byte-identical
to today.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
MONITOR_INTERVAL now defaults to 0, so `make testbed-monitor` runs
iterations back-to-back and drives the local OTEL stack with a steady
stream of data; MONITOR_INTERVAL=5m restores the paced behaviour. The
cadence echo annotates both zeros so the line stays self-explanatory.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
Reword the command synopsis (--interval now optional), the cadence
semantics, and the testbed-monitor walkthrough so they lead with the
new default: an omitted or 0 interval runs iterations back-to-back
continuously, and --error-wait is the brake in that mode. The three
Grafana dashboard sampling caveats no longer claim a 5m default —
metrics flow steadily under the continuous default and only move
per-iteration on a paced run.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Christian Berendt <berendt@b42labs.com>
@berendt
berendt marked this pull request as ready for review July 2, 2026 17:25
@berendt
berendt merged commit 1729e1e into main Jul 2, 2026
2 checks passed
@berendt
berendt deleted the implement/issue-27-continuous-monitor branch July 2, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Monitor mode: run continuously by default (--interval optional, 0 = back-to-back iterations)

1 participant