Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
116 lines (100 sloc) 3.84 KB
INTRODUCTION.
pmacct strategy towards Streaming Telemetry is currently in transition
and encompasses two daemons: 1) pmtelemetryd, the original effort written
in C, and documented in "Quickstart guide to setup Streaming Telemetry
collection" in QUICKSTART; 2) pmgrpcd.py, written in Python and evolving,
and documented in telemetry/decoders/v3/README.pmgrpcd. Following is the
support matrix for Streaming Telemetry collection:
Input transports:
* TCP (pmtelemetryd)
* UDP (pmtelemetryd)
* gRPC (pmgrpcd.py)
* gNMI (pmgrcpd.py [WiP])
Input encodings:
* JSON (pmtelemetryd)
* Protobuf (pmgrpcd.py)
Output methods:
* Kafka (pmtelemetryd, pmgrpcd.py)
* files (pmtelemetryd)
Output encodings:
* JSON (pmtelemetryd, pmgrpcd.py [WiP])
* Avro (pmgrpcd.py)
Examples of supported collection scenarios:
* TCP/JSON -> pmtelemetryd -> Kafka/JSON
* gRPC/Protobuf -> pmgrpcd.py -> Kafka/Avro
* gRPC/Protobuf -> pmgrpcd.py -> pmtelemetryd -> files/JSON (-> decodeGPB.py)
HISTORICAL NOTES.
pmtelemetryd, the pmacct Streaming Telemetry daemon, does natively support
TCP and UDP transports and JSON encoding. In terms of encoding, GPB (Google
Protocol Buffers, or Protobuf) is only base64'd on output for de-marshalling
upon post-processing (utils/decodeGPB.py can help); also GPB encoding is
mainly coupled with gRPC as a transport resulting in limited (or buggy)
support for C language. As a result of that two efforts are currently
pursued: 1) write decoders to hold gRPC sessions and pre-process GPB
encoding and pass de-serialized data to pmtelemetryd; decoders can be
written in any language and pass data onto pmtelemetryd via a ZeroMQ queue;
sample decoders can be found in telemetry/decoders/v1 and are written in
Python; 2) write a slightly more autonomous tool in Python 3, pmgrpcd.py,
to hold gRPC sessions, process GPB encoding and write JSON or Avro to Kafka
(that is, bypassing pmtelemetryd); a sample of such tool can be found in
telemetry/decoders/v3 along with its own documentation.
In the ROUTER CONFIGS section below there are snippets of some routers
configuration in order to export Streaming Telemetry data. Quickstart
guides for collection are in telemetry/decoders/v1/README.grpc_dialout
and telemetry/decoders/v3/README.pmgrpcd
Further reading and more information, including a reference architecture
diagram, is available on the pmacct GitHub wiki at the following page:
https://github.com/pmacct/pmacct/wiki/Streaming-Telemetry
ROUTER CONFIGS.
IOS XR configuration example, gRPC JSON encoding suitable (at time of
this writing) for Openconfig YANG models, ie. openconfig-interface and
openconfig-platform:
telemetry model-driven
destination-group COLLECTOR
address-family ipv4 X.X.X.X port 10000
encoding json
protocol grpc no-tls
!
!
sensor-group SENSOR
sensor-path openconfig-platform:components
sensor-path openconfig-interfaces:interfaces
!
subscription SUBSCRIPTION
sensor-group-id SENSOR sample-interval 30000
destination-id COLLECTOR
source-interface Loopback0
!
!
IOS XR configuration example, gRPC GPB key-value encoding suitable for
for Cisco YANG models, ie. Cisco-IOS-XR-infra-statsd-oper:
telemetry model-driven
destination-group COLLECTOR
address-family ipv4 X.X.X.X port 10000
encoding compact-gpb
protocol grpc no-tls
!
!
sensor-group SENSOR
sensor-path Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters
!
subscription SUBSCRIPTION
sensor-group-id SENSOR sample-interval 30000
destination-id COLLECTOR
source-interface Loopback0
!
!
Huawei VRP configuration example, gRPC compact GPB encoding:
telemetry
#
sensor-group SENSOR
sensor-path huawei-ifm:ifm/interfaces/interface/ifStatistics
#
destination-group COLLECTOR
ipv4-address X.X.X.X port 10000 vpn-instance MANAGEMENT_VRF protocol grpc no-tls
#
subscription SUBSCRIPTION
sensor-group SENSOR
destination-group COLLECTOR
#
return
You can’t perform that action at this time.