feature/rfcs

RFCs/2021-04-16-75-taps.md

@@ -0,0 +1,55 @@
+# pktvisor Taps
+
+## Summary
+
+Taps are named, host specific connection specifications for the raw input streams accessed by pktvisord. They represent
+configuration data only; they do not cause any processing to take place in pktvisord. They should be referenced by
+Collection Policies (#76) by name.
+
+The goal of Taps is to abstract away host level details such as ethernet interface or dnstap socket location so that
+collection policies can apply to a broad set of pktvisor agents without worrying about these details. It also provides
+for an easy way to integrate with existing configuration management stacks which will hold the source of truth for this
+type of information.
+
+Taps may be configured on the command line at agent start up, and may optionally be configured by the Admin API if
+available.
+
+`taps.yaml`
+
+```yaml
+version: "1.0"
+
+visor:
+  # each tap has input module specific configuration options
+  taps:
+    # a pcap tap which uses eth0 and is referenced by the identifier "anycast"
+    anycast:
+      type: pcap
+      config:
+        iface: eth0
+    # an sflow tap which listens on the given IP and port, referenced by the identifier "pop_switch"
+    pop_switch:
+      type: sflow
+      config:
+        port: 6343
+        bind: 192.168.1.1
+    # a dnstap tap which gets its stream from the given socket, named "trex_tap"
+    trex_tap:
+      type: dnstap
+      config:
+        socket: /var/dns.sock
+```
+
+## REST API
+
+CRUD on Taps for a running pktvisord instance is possible if the Admin API is active.
+
+`/api/v1/taps`
+
+`/api/v1/taps/:id:`
+
+## Standalone Command Line Example
+
+```shell
+$ pktvisord --config taps.yaml
+```

RFCs/2021-04-16-76-collection-policies.md

@@ -0,0 +1,84 @@
+### pktvisor Collection Policies
+
+## Summary
+
+Collection policies direct pktvisor to use Taps (#75) to create an instance of an input stream (possibly with a filter),
+and attach handlers to it. Processing takes place, and the data is exposed for sinks to collect. These policies may be
+given directly to pktvisor via command line or through the Admin API if available.
+
+`collection-policy-anycast.yaml`
+
+```yaml
+version: "1.0"
+
+visor:
+  collection:
+    # policy name and description
+    anycast_dns:
+      description: "base anycast DNS policy"
+      # input stream to create based on the given tap and optional filter config
+      input:
+        # this must reference a tap name, or application of the policy will fail
+        tap: anycast
+        # this must match the type of the matching tap name. or application of the policy will fail
+        type: pcap
+        filter:
+          bpf: "port 53"
+      # stream handlers to attach to this input stream
+      # these decide exactly which data to summarize and expose for collection
+      handlers:
+        # default configuration for the stream handlers
+        config:
+          periods: 5
+          max_deep_sample: 50
+        modules:
+          # the keys at this level are unique identifiers
+          default_net:
+            type: net
+          udp_traffic:
+            type: net
+            config:
+              protocols: [ udp ]
+            metrics:
+              enable:
+                - top_ips
+          default_dns:
+            type: dns
+            config:
+              max_deep_sample: 75
+            # time window analyzers
+            analyzers:
+              modules:
+                nx_attack:
+                  type: dns_random_label
+          special_domain:
+            type: dns
+            # specify that the stream handler module requires >= specific version to be successfully applied 
+            require_version: "1.0"
+            config:
+              # must match the available configuration options for this version of this stream handler
+              qname_suffix: .mydomain.com
+            metrics:
+              disable:
+                - top_qtypes
+                - top_udp_ports
+```
+
+## REST API
+
+CRUD on Collection Policies for a running pktvisord instance is possible if the Admin API is active.
+
+`/api/v1/collection`
+
+`/api/v1/collection/:id:`
+
+## Standalone Command Line Example
+
+```shell
+$ pktvisord --config taps.yaml --config collection-policy-anycast.yaml
+```
+
+They may also be combined into a single YAML file (the schemas will merge) and passed in with one `--config` option.
+
+The REST API (or prometheus output, pktvisor-cli, etc) should then be used to collect the results manually.
+

docs/CONTROL_PLANE.md → RFCs/2021-04-16-77-module-reflection.md

@@ -1,10 +1,6 @@
-# Control Plane
+# Module Reflection
 
-**_Draft_**
-
-pktvisord exposes a control plane over REST API.
-
-## Discovery
+## Summary
 
 pktvisord exposes a method for discovering the available modules, their configurable properties, and their associated
 metrics schema.