Skip to content

telemetry/gnmi-writer: store interface packet-class counters#3731

Merged
snormore merged 2 commits into
mainfrom
snor/gnmi-interface-packet-counters
May 20, 2026
Merged

telemetry/gnmi-writer: store interface packet-class counters#3731
snormore merged 2 commits into
mainfrom
snor/gnmi-interface-packet-counters

Conversation

@snormore
Copy link
Copy Markdown
Contributor

@snormore snormore commented May 20, 2026

Summary of Changes

  • Persist per-interface unicast, multicast, and broadcast packet counters (in/out) plus FCS errors to the interface_state table — these already stream in the gNMI interface-state subtree alongside the octet/packet counters we keep, so no upstream subscription change is needed.
  • Brings the ClickHouse interface_state table closer to parity with the interface counters already collected in InfluxDB.
  • Add the seven new columns to interface_state via a ClickHouse migration (ADD COLUMN is metadata-only; existing rows read back 0) and recreate the interface_state_latest view so SELECT * surfaces them.
  • Unblocks downstream rollups (e.g. device_interface_rollup_5m) that consume in_fcs_errors and multicast PPS metrics.

Diff Breakdown

Category Files Lines (+/-) Net
Tests 2 +96 / -2 +94
Scaffolding 2 +63 / -0 +63
Core logic 1 +21 / -0 +21
Total 5 +180 / -2 +178

Small, mechanical change: the bulk is tests and a schema migration; the actual logic is seven nil-checked field copies in the extractor.

Key files (click to expand)
  • telemetry/migrations/20260520000000_interface_state_packet_counters.sql — new migration adding the 7 UInt64 columns and recreating the _latest view (with full down-migration)
  • telemetry/gnmi-writer/internal/gnmi/processor_test.go — unit test asserting all 16 counters map to the correct record fields (distinct values catch column swaps)
  • telemetry/gnmi-writer/internal/gnmi/processor_integration_test.go — integration test asserting the 7 new columns round-trip through ClickHouse on an active interface with real traffic
  • telemetry/gnmi-writer/internal/gnmi/extractors.go — populate the new counter fields from iface.State.Counters.*
  • telemetry/gnmi-writer/internal/gnmi/records.go — add the 7 fields to InterfaceStateRecord with ch: column tags

Testing Verification

  • Unit test (TestExtractInterfaceState_Counters): builds an oc.Device with distinct non-zero values for all 16 counters and asserts the full extracted InterfaceStateRecord, guarding against field-assignment mistakes.
  • Integration test (verifyInterfaceState): validates the gNMI fixture → extractor → ClickHouse INSERT → query-back path for the new columns against a real ClickHouse container, asserting non-zero values on Switch1/11/2 — this exercises the ch: tag and migration column-name mapping end to end.
  • Full TestIntegration suite passes against a containerized ClickHouse with all migrations applied.

snormore added 2 commits May 20, 2026 07:52
Collect per-interface unicast/multicast/broadcast packet counters and
FCS errors from the gNMI interface state subtree, which already streams
them alongside the octet/packet counters we keep. Add the seven columns
to interface_state via migration and populate them in the extractor.
@snormore snormore merged commit 5edad87 into main May 20, 2026
33 checks passed
@snormore snormore deleted the snor/gnmi-interface-packet-counters branch May 20, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants