Skip to content

Releases: network-analytics/NetCalyx

v0.3.0

Choose a tag to compare

@riccardo-negri riccardo-negri released this 07 Sep 15:12

This release centers on YANG-Push robustness and caching: a new global YANG module cache with in-flight de-duplication, canonical XPath filter normalization, a stable order-independent content-id, and an upgrade to yang-rs-5 (libyang 5.4.9) enabling anydata nodes validation. It also ships the Kafka YANG consumer as its own released binary and expands OTEL metrics across the collector and yang-push.

✨ Highlights

YANG-Push & NETCONF

  • Global YANG module cache shared across SSH sessions — skips redundant get-schema RPCs per (name, revision), with single-flight de-duplication to kill the startup thundering herd (coalesced/hit/miss/size OTEL counters).
  • Stable content-id: module/feature/submodule hashing is now canonical (sorted, length-prefixed, kind-tagged), so semantically identical module sets no longer produce duplicate cache directories.
  • XPath target-filter normalization for both NETCONF- and JSON-sourced filters, with a new normalize_path engine, prefix/binding validation, and warnings on non-canonical or unresolvable targets.
  • yang-rs-5 (libyang 5.4.9) upgrade for the validation actor, with (configurable) ANYDATA_STRICT parsing enabling validation of nested anydata nodes.
  • Other fixes such as deferred subscription updates while a schema fetch is in-flight and more accurate content-id logging and datastore-scoped prefix module lookup.

Collector & observability

  • New schema cache and registry metrics; cache_actor pending gauge replaced with a duration histogram.
  • OTEL metrics reader isolated on a dedicated runtime; validation-actor metrics improved with enum-based attributes.

Packaging & CLI

  • Kafka YANG consumer extracted into a standalone netcalyx-yang-consumer crate, now built and released as a binary (Linux + macOS).
  • Consolidated --help/--version across all binaries.

Other

  • bgp-pkt: Display for ExtendedCommunityIpv6, derive Hash.
  • Fixes for empty-container handling in copy_buffer, IPFIX enum name sanitization, and example logging to stderr.

⚠️ Breaking changes

  • YANG-Push cache invalidation — the subscription-info format changed (no peer port anymore) and the content-id computation changed (now order-independent). Old on-disk yang-push cache directories are not supported and cannot be loaded: you need to clear the old cache directory before restarting the collector.
  • Metrics renamed/changed — the cache_actor pending gauge is replaced by a duration histogram, and several counters/attributes changed. Update dashboards and alerts.
  • API rename — YangSchemaFormat → GetSchemaFormat (per RFC 6022) for consumers of netconf-proto.
  • Binary relocation — the Kafka YANG consumer moved out of the collector into the standalone netcalyx-yang-consumer crate/binary.

What's Changed

  • ci(commitlint): limit commit message lines to 72 characters by @ustorbeck in #26
  • fix(bgp-speaker): use paused clock in hold-timer tests by @ustorbeck in #27
  • fix(examples): keep example stdout clean and stop panicking on JSON serialization errors by @rodonile in #33
  • feat(bgp-pkt): add Display for ExtendedCommunityIpv6 and derive Hash for path attributes by @rodonile in #32
  • fix(netconf-proto): handle empty containers in copy_buffer_till functions by @riccardo-negri in #28
  • feat(yang-push): migrate to yang-rs-5 / libyang 5.4.9, revamp OTEL metrics, refresh example YANG modules by @rodonile in #23
  • feat(yang-push): separate transport session info from subscription identity by @rodonile in #29
  • fix(ipfix-code-generator): drop invalid identifier chars in registry-derived enum names by @rodonile in #41
  • fix(yang-push): defer subscription updates while a schema fetch is in-flight by @rodonile in #35
  • fix(collector): subscribe to flow-recv actor only when needed for flow-options handler actor by @rodonile in #38
  • feat(collector): add schema cache and registry metrics to kafka yang publisher by @rodonile in #34
  • refactor(yang-consumer): extract kafka-yang-consumer example into standalone crate by @rodonile in #39
  • feat(yang-push): make anydata_strict validation configurable by @rodonile in #42
  • build(deps): bump schema_registry_converter from 4.9.0 to 4.10.0 by @dependabot[bot] in #47
  • feat(cli): consolidate --help and --version output across all binaries by @rodonile in #40
  • feat(netconf-proto): add global YANG module cache by @riccardo-negri in #22
  • fix(yang-push): resolve xpath-filter target modules by prefix by @rodonile in #43
  • fix(netconf-proto): make YANG library content-id calculation independent on the module order by @rodonile in #44
  • fix(collector): switch PeriodicReader on async runtime to enable tonic metrics logs by @rodonile in #50
  • style: reflow comments to match rustfmt comment_width by @ustorbeck in #51
  • feat(yang-push): normalize target xpath filters and add failed xpath resolution check with libyang by @rodonile in #45
  • chore: release v0.3.0 by @riccardo-negri in #52

Full Changelog: v0.2.0...v0.3.0

v0.2.0

Choose a tag to compare

@riccardo-negri riccardo-negri released this 04 Aug 07:42

This release is focused on YANG-Push and NETCONF reliability, with a notable upgrade to the UDP-Notif codec, improved observability via OpenTelemetry, and foundational CI tooling.

✨ Highlights

  • YANG-Push is more stable. A deadlock/livelock in the validation actor during cache drain is fixed, and NETCONF SSH connections no longer rely on a hardcoded source IP: the system-chosen address is used instead. These two fixes together significantly improve reliability in multi-interface deployments.
  • NETCONF protocol correctness. Short buffered chunk messages are now decoded correctly, and YANG revision selection consistently picks the latest revision regardless of how capabilities are ordered in the capability list. FTPS URL scheme support is also added.
  • UDP-Notif codec overhaul. The codec now supports configurable reassembly limits, is aligned with the latest draft spec, and ships with new OpenTelemetry metrics for deeper visibility into the reassembly pipeline.

What's Changed

  • fix(yang-push): use system-chosen source IP for NETCONF SSH by @riccardo-negri in #5
  • yang-push: fix clippy::result_large_err in the YANG Library fetcher by @ustorbeck in #8
  • fix(yang-push): stop validation actor deadlock/livelock on cache drain by @riccardo-negri in #6
  • chore(pcap-tests): regenerate json for preserve_order by @rodonile in #9
  • build(deps): bump actions/checkout from 6 to 7 by @dependabot[bot] in #1
  • netconf: add support for capability UrlScheme::Ftps by @ustorbeck in #11
  • build(deps): bump quinn-proto from 0.11.14 to 0.11.16 by @dependabot[bot] in #10
  • ci(codecov): disable upload to codecov.io by @riccardo-negri in #12
  • style: unify "YANG-Push" spelling by @ustorbeck in #15
  • feat(yang-push): add an otel counter for messages failed to send upst… by @ustorbeck in #16
  • fix(yang-push): rename opentelemetry counters back to "decoded" by @ustorbeck in #17
  • fix: add the forgotten copyright headers by @ustorbeck in #19
  • chore(README): fix github badge and add one for CI status by @rodonile in #14
  • ci: add commitlint job by @riccardo-negri in #13
  • ci(license): add license header check for .rs files by @riccardo-negri in #20
  • fix(netconf-proto): decode short buffered chunk messages by @ustorbeck in #21
  • feat(udp-notif): improve codec with configurable reassembly limits, align to draft, and add OTel metrics by @rodonile in #18
  • fix(netconf-proto): correctly select latest YANG revision regardless of order by @rodonile in #24
  • chore: release v0.2.0 by @riccardo-negri in #25

Full Changelog: v0.1.0...v0.2.0

v0.1.0

Choose a tag to compare

@riccardo-negri riccardo-negri released this 27 Jul 10:12

This release establishes the NetCalyx identity with no functional changes from the NetGauze baseline it was forked from.