Skip to content

feat!: multi-source VictoriaLogs (v2.0.0 part 1, #34)#36

Merged
fxthiry merged 1 commit intorelease/2.0.0from
fix/multi-source-vl-core
Apr 16, 2026
Merged

feat!: multi-source VictoriaLogs (v2.0.0 part 1, #34)#36
fxthiry merged 1 commit intorelease/2.0.0from
fix/multi-source-vl-core

Conversation

@fxthiry
Copy link
Copy Markdown
Owner

@fxthiry fxthiry commented Apr 16, 2026

Implements #34 — multi-source VictoriaLogs support. Part 1 of 2 for v2.0.0. Part 2 (observability + guardrails: metrics vl_source label, valerter_vl_source_up gauge, max_streams cap, reconnect jitter) ships as a follow-up PR before the v2.0.0 tag.

What ships

  • victorialogs: is now a BTreeMap<String, VlSourceConfig>. Each source carries its own URL, basic_auth, TLS, headers.
  • Rules gain optional vl_sources: [name, ...]. Absent fans out to all sources. Engine spawns one task per (rule, source) with per-source cancellation and reconnect isolation.
  • {{ vl_source }} injected into every render context where rule_name already flowed (layer 1 templates, throttle.key, layer 2 notifier templates). Synthetic wins on collision.
  • Default throttle key becomes {rule}-{source}:global (per-source bucket isolation by default).
  • Source name restricted to ^[a-zA-Z0-9_]+$ to keep the default key separator unambiguous.
  • Legacy single-URL config rejected at load with a migration-oriented error.
  • AlertPayload.vl_source propagated end-to-end. Mattermost footer + default webhook payload extended (see Breaking changes in CHANGELOG).

Validation

  • cargo test — 540 tests pass (468 lib + 72 integration, +17 lib +5 integration vs baseline)
  • cargo clippy --all-targets --all-features -- -D warnings — clean
  • cargo fmt --check — clean
  • cargo run --bin valerter -- --validate -c config/config.example.yaml — exit 0, prints VictoriaLogs sources: 1 [default=...]
  • Manual live regression: 2 distinct VictoriaLogs backends (one remote, one local docker container), 3 rules covering the routing matrix (single-source pinned, single-source pinned to other, fanout). 4 tasks spawned, 4 Telegram messages received with {{ vl_source }} correctly rendered to the originating source name per task. Zero error, per-source isolation confirmed.

Out of scope (part 2)

The following ships as a separate follow-up PR before the v2.0.0 tag:

  • Prometheus vl_source label on existing counters (breaking for dashboards)
  • valerter_vl_source_up{source} gauge
  • defaults.max_streams cap with load-time enforcement
  • Per-task reconnect jitter
  • /metrics snapshot test

`victorialogs:` becomes a `BTreeMap<String, VlSourceConfig>` indexed by
source name. Rules gain an optional `vl_sources: [name, ...]` field;
absent means fan out to every configured source. The engine spawns one
task per (rule, source) pair with per-source cancellation and reconnect
isolation, so a single unhealthy source does not stop alerts on the
others.

Synthetic `vl_source` is injected into every render context where
`rule_name` already flowed: layer 1 templates (title, body,
email_body_html), throttle.key, and notifier-level layer 2 contexts.
The default throttle key becomes `{rule}-{source}:global` so multi-source
deployments get isolated buckets per source out of the box.

Source names are restricted to `^[a-zA-Z0-9_]+$` to keep the default
throttle key separator (`-`) unambiguous. Rules reject duplicate
`vl_sources` entries at load. The legacy single-URL config shape is
rejected with an actionable migration error.

`AlertPayload.vl_source` propagates end-to-end. The Mattermost footer
gains a source segment and the default webhook payload exposes
`vl_source` as a top-level JSON field. See CHANGELOG v2.0.0 Breaking
section for the full migration.

Tests consume the fixtures corpus (PR #35). The new
`tests/multi_source_integration.rs` exercises the full routing matrix
across two wiremock VL backends, including a negative-evidence assertion
that a pinned rule does not leak to the unpinned source.

Observability and guardrails (metrics `vl_source` label,
`valerter_vl_source_up` gauge, `max_streams` cap, reconnect jitter) ship
as a separate spec part 2 before the v2.0.0 tag.

Cargo.toml not bumped here; version bump lives in the release PR.
@fxthiry fxthiry merged commit 693c4cf into release/2.0.0 Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant