From 9c13bd4654d3a7e07a4e74c8639060c6266669e1 Mon Sep 17 00:00:00 2001 From: Dimitri Nicolopoulos Date: Fri, 17 May 2024 15:21:48 -0700 Subject: [PATCH] [EV-4885] Improve flow log aggregation level doc --- .../visibility/elastic/flow/aggregation.mdx | 477 ++---------------- .../visibility/elastic/flow/aggregation.mdx | 475 ++--------------- .../visibility/elastic/flow/aggregation.mdx | 475 ++--------------- .../visibility/elastic/flow/aggregation.mdx | 475 ++--------------- 4 files changed, 206 insertions(+), 1696 deletions(-) diff --git a/calico-cloud/visibility/elastic/flow/aggregation.mdx b/calico-cloud/visibility/elastic/flow/aggregation.mdx index 7f0b1195d0..5b9417f659 100644 --- a/calico-cloud/visibility/elastic/flow/aggregation.mdx +++ b/calico-cloud/visibility/elastic/flow/aggregation.mdx @@ -10,96 +10,63 @@ Configure flow log aggregation level to reduce log volume and costs. ## Value -Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of aggregation is suitable for your implementation. +Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce +flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of +aggregation is suitable for your implementation. ## Concepts -### Aggregation: volume and cost versus visibility +### Volume and cost versus visibility -{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). - -### Aggregation flow log example - -The following is a sample flow log entry using aggregation by pod prefix for allowed traffic. For flow log fields and filter parameters, see [Filter flow logs](filtering.mdx). - -``` - { - "start_time": 1597086849, - "end_time": 1597087167, - "source_ip": null, - "source_name": "-", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": null, - "dest_name": "-", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52519, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 40, - "packets_in": 282, - "packets_out": 239, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597087167000 - } -``` - -- `1597086849` to `1597087167` is the 5 minute aggregation interval. -- Workload endpoints with a similar prefix `access-6b687c8dcb-*` in the `policy-demo` namespace connected to a workload-endpoints/pods with prefix `nginx-86c57db685-*` exposing a service on port `80`. -- The aggregated source workload endpoints had the labels `app: nginx` and `pod-template-hash: 6b687c8dcb` and the aggregated destination workload endpoint had the labels `app: nginx` and `pod-template-hash: 86c57db685`. -- It was an incoming connection reported by the "Destination" node, and a policy "Allowed" the connection. -- There were 282 incoming packets and 239 outgoing packets. -- Within the aggregation time interval, there were 7 flows aggregated, with 47 new flows started, and 40 flows completed. - -When viewing traffic flows, note that **null values** for `source_ip` and `dest_ip` means **pod prefix aggregation is enabled**. - -``` -... -"source_ip": null, -... -}, -"dest_ip": null, - -``` +{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The +level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all +depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). ### Aggregation types and levels -The following table summarizes the aggregation levels by flow log traffic. - -| Flow log aggregation by... | Available for... | Aggregates all flows that share... | -| -------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Pod prefix | Allowed and denied traffic (default is allowed) | `FlowLogsFileAggregationKindForAllowed`
**0**, No aggregation
**1**, A source port on each node.
**2, (default)** Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | -| Source port | Allowed and denied traffic | `FlowLogsFileAggregationKindForAllowed`
`FlowLogsFileAggregationKindForDenied`
**0**, No aggregation
**1, (default)** Source port on each node.
**2**, Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | +For allowed flows, the default aggregation level is 2, `AnyProcessInSamePodPrefix` and for denied flows the default aggregation level is 1, +`AnyProcessInSameSourcePod`. + +The following table summarizes the aggregation levels by flow log traffic: + +| **Level** | **Name** | **Description** | +|-----------|-------------------------------------|-------------------------------------------------------------------| +| 0 | | No aggregation | +| 1 | AnyProcessInSameSourcePod | Identity fields below source pod level are masked out. It means that if multiple processes or containers, within the same source pod, perform the same operation, the events are aggregated. | +| 2 | AnyProcessInSameSourcePodPrefix | Identity fields below source pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation, the events are aggregated. | +| 3 | AnyProcessInSamePodPrefix | Identity fields below source and destination pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation towards pods with the same prefix, the events are aggregated. | + +### Understanding aggregation level differences + +Here are examples of pod-to-pod flows, highlighting the differences between flow logs at various aggregation levels. + +The source port is usually ephemeral and does not convey useful information. By suppressing the source port, `AnyProcessInSameSourcePod` aggregation +type minimizes the flow logs generated for traffic coming from different containers within the same pod and going to the same destination endpoint +and port. The two flows originating from `client-a` without aggregation are combined into one. + +In Kubernetes, ReplicaSets and StatefulSets can automatically create names for pods. For example, the pods `nginx-1` and `nginx-2` are created by the +ReplicaSet nginx. The ReplicaSet name is considered a pod-prefix and is used to aggregate flow log entries (indicated with an asterisk * at the end +of the name). Flow logs originating from pods with the same prefix will be aggregated as long as the traffic is on the same protocol, and destined +towards the same IP, and destination port. The three flow logs without aggregation originating from `client-a` and `client-b` are combined into a +single flow log. This aggregation level is called `AnyProcessInSameSourcePodPrefix`. + +Finally, with `AnyProcessInSamePodPrefix` we combine source and destination pods that are part of the same ReplicaSets. With level 3, the flow logs +are aggregated by the destination port and protocol, as long as they originate from pods with the same pod-prefix and destined for pods of the same +pod-prefix. All logs previously distinct, are aggregated into a single flow log (see the last row). + +| | | **Src Traffic** | | | **Dst Traffic** | | | **Packet counts** | | +|--------------------------|-----------|----------|---------|----------|----------|---------|----------|------------|-------------| +| **Aggr lvl** | **Flows** | **Name** | **IP** | **Port** | **Name** | **IP** | **Port** | **Pkt in** | **Pkt out** | +| 0 (no aggr) | 4 | client-a | 1.1.1.1 | 45556 | nginx-1 | 2.2.2.2 | 80 | 1 | 2 | +| | | client-b | 1.1.2.2 | 45666 | nginx-2 | 2.2.3.3 | 80 | 2 | 2 | +| | | client-a | 1.1.1.1 | 65533 | nginx-1 | 2.2.2.2 | 80 | 1 | 3 | +| | | client-c | 1.1.1.2 | 65534 | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 1 (src port) | 3 | client-a | 1.1.1.1 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 5 | +| | | client-b | 1.1.2.2 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 2 | +| | | client-c | 1.1.3.3 | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 2 (src pod-prefix) | 2 | client-* | - | - | nginx-1 | 2.2.2.2 | 80 | 4 | 7 | +| | | client-* | - | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 3 (src/dest pod-prefix) | 1 | client-* | - | - | nginx-* | - | 80 | 7 | 11 | ## How to @@ -112,7 +79,7 @@ The following table summarizes the aggregation levels by flow log traffic. Use the following command: ```bash -kubectl get felixconfiguration.p -o yaml +kubectl get felixconfiguration -o yaml ``` ### Change default aggregation level @@ -138,346 +105,8 @@ In each aggregation interval, connections/connection attempts can be started or The underlying reason for this overlap is a dependency on Linux conntrack, which provides the lifetime of stats that {{prodname}} tracks across different protocols (TCP, ICMP, UDP). For example, for UDP and ICMP, {{prodname}} waits for a conntrack entry to timeout before it considers a “connection” closed, and this is usually greater than 10 seconds. -## Tutorial - -Here are examples of pod-to-pod flows for each aggregation type. - -### Aggregation by source port - -Source port aggregation is straightforward. When viewing `source IP`, `destination IP`, `protocol`, `source port`, `destination port`, the `source port` is usually ephemeral and does not convey useful information. By suppressing `source port`, this aggregation type minimizes the flows logs generated for traffic between the same source-destination endpoint, and same destination port. - -**Node 1** - -```json - { - "start_time": 1597164816, - "end_time": 1597165131, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 0, - "bytes_out": 3120, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 0, - "packets_out": 52, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597165131000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597164816, - "end_time": 1597165125, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "deny", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.deny-access-nginx|deny" - ] - }, - "bytes_in": 3120, - "bytes_out": 0, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 52, - "packets_out": 0, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597165125000 - } -``` - -### Aggregation by pod prefix - -In Kubernetes, pods that are part of ReplicaSets and StatefulSets, etc. can automatically create names for pods. For example, the pods `nginx-65899c769f-tdmw7` and `nginx-65899c769f-xyz123` are created by the ReplicaSet `nginx-65899c769f`. The ReplicaSet name is considered a **pod-prefix** and is used to aggregate flow log entries (shown with an asterisk `*` at the end of the name). For a destination pod to be aggregated, it must have the same "pod-prefix" and expose the same destination port. - -**Node 1** - -```json - { - "start_time": 1597166567, - "end_time": 1597166893, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 52311, - "bytes_out": 18236, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 235, - "packets_out": 282, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166893000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597166567, - "end_time": 1597166879, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52311, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 282, - "packets_out": 235, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166879000 - } -``` - -### No aggregation - -If you turn off aggregation, your log storage may be overwhelmed. Be sure to provision more storage if you do. - -**Node 1** - -```json - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 42106, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166383000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 43092, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166383000 - } -``` - ## Additional resources - [Archive logs to storage](../archive-storage.mdx) +- [Configure RBAC for Elasticsearch logs](../rbac-elasticsearch.mdx) +- [Configure data retention](../retention.mdx) diff --git a/calico-cloud_versioned_docs/version-19-1/visibility/elastic/flow/aggregation.mdx b/calico-cloud_versioned_docs/version-19-1/visibility/elastic/flow/aggregation.mdx index 7f0b1195d0..0fc5fb4458 100644 --- a/calico-cloud_versioned_docs/version-19-1/visibility/elastic/flow/aggregation.mdx +++ b/calico-cloud_versioned_docs/version-19-1/visibility/elastic/flow/aggregation.mdx @@ -10,96 +10,63 @@ Configure flow log aggregation level to reduce log volume and costs. ## Value -Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of aggregation is suitable for your implementation. +Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce +flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of +aggregation is suitable for your implementation. ## Concepts -### Aggregation: volume and cost versus visibility +### Volume and cost versus visibility -{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). - -### Aggregation flow log example - -The following is a sample flow log entry using aggregation by pod prefix for allowed traffic. For flow log fields and filter parameters, see [Filter flow logs](filtering.mdx). - -``` - { - "start_time": 1597086849, - "end_time": 1597087167, - "source_ip": null, - "source_name": "-", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": null, - "dest_name": "-", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52519, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 40, - "packets_in": 282, - "packets_out": 239, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597087167000 - } -``` - -- `1597086849` to `1597087167` is the 5 minute aggregation interval. -- Workload endpoints with a similar prefix `access-6b687c8dcb-*` in the `policy-demo` namespace connected to a workload-endpoints/pods with prefix `nginx-86c57db685-*` exposing a service on port `80`. -- The aggregated source workload endpoints had the labels `app: nginx` and `pod-template-hash: 6b687c8dcb` and the aggregated destination workload endpoint had the labels `app: nginx` and `pod-template-hash: 86c57db685`. -- It was an incoming connection reported by the "Destination" node, and a policy "Allowed" the connection. -- There were 282 incoming packets and 239 outgoing packets. -- Within the aggregation time interval, there were 7 flows aggregated, with 47 new flows started, and 40 flows completed. - -When viewing traffic flows, note that **null values** for `source_ip` and `dest_ip` means **pod prefix aggregation is enabled**. - -``` -... -"source_ip": null, -... -}, -"dest_ip": null, - -``` +{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The +level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all +depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). ### Aggregation types and levels -The following table summarizes the aggregation levels by flow log traffic. - -| Flow log aggregation by... | Available for... | Aggregates all flows that share... | -| -------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Pod prefix | Allowed and denied traffic (default is allowed) | `FlowLogsFileAggregationKindForAllowed`
**0**, No aggregation
**1**, A source port on each node.
**2, (default)** Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | -| Source port | Allowed and denied traffic | `FlowLogsFileAggregationKindForAllowed`
`FlowLogsFileAggregationKindForDenied`
**0**, No aggregation
**1, (default)** Source port on each node.
**2**, Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | +For allowed flows, the default aggregation level is 2, `AnyProcessInSamePodPrefix` and for denied flows the default aggregation level is 1, +`AnyProcessInSameSourcePod`. + +The following table summarizes the aggregation levels by flow log traffic: + +| **Level** | **Name** | **Description** | +|-----------|-------------------------------------|-------------------------------------------------------------------| +| 0 | | No aggregation | +| 1 | AnyProcessInSameSourcePod | Identity fields below source pod level are masked out. It means that if multiple processes or containers, within the same source pod, perform the same operation, the events are aggregated. | +| 2 | AnyProcessInSameSourcePodPrefix | Identity fields below source pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation, the events are aggregated. | +| 3 | AnyProcessInSamePodPrefix | Identity fields below source and destination pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation towards pods with the same prefix, the events are aggregated. | + +### Understanding aggregation level differences + +Here are examples of pod-to-pod flows, highlighting the differences between flow logs at various aggregation levels. + +The source port is usually ephemeral and does not convey useful information. By suppressing the source port, `AnyProcessInSameSourcePod` aggregation +type minimizes the flow logs generated for traffic coming from different containers within the same pod and going to the same destination endpoint +and port. The two flows originating from `client-a` without aggregation are combined into one. + +In Kubernetes, ReplicaSets and StatefulSets can automatically create names for pods. For example, the pods `nginx-1` and `nginx-2` are created by the +ReplicaSet nginx. The ReplicaSet name is considered a pod-prefix and is used to aggregate flow log entries (indicated with an asterisk * at the end +of the name). Flow logs originating from pods with the same prefix will be aggregated as long as the traffic is on the same protocol, and destined +towards the same IP, and destination port. The three flow logs without aggregation originating from `client-a` and `client-b` are combined into a +single flow log. This aggregation level is called `AnyProcessInSameSourcePodPrefix`. + +Finally, with `AnyProcessInSamePodPrefix` we combine source and destination pods that are part of the same ReplicaSets. With level 3, the flow logs +are aggregated by the destination port and protocol, as long as they originate from pods with the same pod-prefix and destined for pods of the same +pod-prefix. All logs previously distinct, are aggregated into a single flow log (see the last row). + +| | | **Src Traffic** | | | **Dst Traffic** | | | **Packet counts** | | +|--------------------------|-----------|----------|---------|----------|----------|---------|----------|------------|-------------| +| **Aggr lvl** | **Flows** | **Name** | **IP** | **Port** | **Name** | **IP** | **Port** | **Pkt in** | **Pkt out** | +| 0 (no aggr) | 4 | client-a | 1.1.1.1 | 45556 | nginx-1 | 2.2.2.2 | 80 | 1 | 2 | +| | | client-b | 1.1.2.2 | 45666 | nginx-2 | 2.2.3.3 | 80 | 2 | 2 | +| | | client-a | 1.1.1.1 | 65533 | nginx-1 | 2.2.2.2 | 80 | 1 | 3 | +| | | client-c | 1.1.1.2 | 65534 | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 1 (src port) | 3 | client-a | 1.1.1.1 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 5 | +| | | client-b | 1.1.2.2 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 2 | +| | | client-c | 1.1.3.3 | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 2 (src pod-prefix) | 2 | client-* | - | - | nginx-1 | 2.2.2.2 | 80 | 4 | 7 | +| | | client-* | - | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 3 (src/dest pod-prefix) | 1 | client-* | - | - | nginx-* | - | 80 | 7 | 11 | ## How to @@ -112,7 +79,7 @@ The following table summarizes the aggregation levels by flow log traffic. Use the following command: ```bash -kubectl get felixconfiguration.p -o yaml +kubectl get felixconfiguration -o yaml ``` ### Change default aggregation level @@ -138,346 +105,6 @@ In each aggregation interval, connections/connection attempts can be started or The underlying reason for this overlap is a dependency on Linux conntrack, which provides the lifetime of stats that {{prodname}} tracks across different protocols (TCP, ICMP, UDP). For example, for UDP and ICMP, {{prodname}} waits for a conntrack entry to timeout before it considers a “connection” closed, and this is usually greater than 10 seconds. -## Tutorial - -Here are examples of pod-to-pod flows for each aggregation type. - -### Aggregation by source port - -Source port aggregation is straightforward. When viewing `source IP`, `destination IP`, `protocol`, `source port`, `destination port`, the `source port` is usually ephemeral and does not convey useful information. By suppressing `source port`, this aggregation type minimizes the flows logs generated for traffic between the same source-destination endpoint, and same destination port. - -**Node 1** - -```json - { - "start_time": 1597164816, - "end_time": 1597165131, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 0, - "bytes_out": 3120, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 0, - "packets_out": 52, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597165131000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597164816, - "end_time": 1597165125, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "deny", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.deny-access-nginx|deny" - ] - }, - "bytes_in": 3120, - "bytes_out": 0, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 52, - "packets_out": 0, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597165125000 - } -``` - -### Aggregation by pod prefix - -In Kubernetes, pods that are part of ReplicaSets and StatefulSets, etc. can automatically create names for pods. For example, the pods `nginx-65899c769f-tdmw7` and `nginx-65899c769f-xyz123` are created by the ReplicaSet `nginx-65899c769f`. The ReplicaSet name is considered a **pod-prefix** and is used to aggregate flow log entries (shown with an asterisk `*` at the end of the name). For a destination pod to be aggregated, it must have the same "pod-prefix" and expose the same destination port. - -**Node 1** - -```json - { - "start_time": 1597166567, - "end_time": 1597166893, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 52311, - "bytes_out": 18236, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 235, - "packets_out": 282, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166893000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597166567, - "end_time": 1597166879, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52311, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 282, - "packets_out": 235, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166879000 - } -``` - -### No aggregation - -If you turn off aggregation, your log storage may be overwhelmed. Be sure to provision more storage if you do. - -**Node 1** - -```json - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 42106, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166383000 - } -``` - -**Node 2** - -```json - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 43092, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166383000 - } -``` - ## Additional resources - [Archive logs to storage](../archive-storage.mdx) diff --git a/calico-enterprise/visibility/elastic/flow/aggregation.mdx b/calico-enterprise/visibility/elastic/flow/aggregation.mdx index 69ade36b7b..aad75f90ab 100644 --- a/calico-enterprise/visibility/elastic/flow/aggregation.mdx +++ b/calico-enterprise/visibility/elastic/flow/aggregation.mdx @@ -10,96 +10,63 @@ Configure flow log aggregation level to reduce log volume and costs. ## Value -Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of aggregation is suitable for your implementation. +Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce +flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of +aggregation is suitable for your implementation. ## Concepts -### Aggregation: volume and cost versus visibility +### Volume and cost versus visibility -{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). - -### Aggregation flow log example - -The following is a sample flow log entry using aggregation by pod prefix for allowed traffic. For flow log fields and filter parameters, see [Filter flow logs](filtering.mdx). - -``` - { - "start_time": 1597086849, - "end_time": 1597087167, - "source_ip": null, - "source_name": "-", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": null, - "dest_name": "-", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52519, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 40, - "packets_in": 282, - "packets_out": 239, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597087167000 - } -``` - -- `1597086849` to `1597087167` is the 5 minute aggregation interval. -- Workload endpoints with a similar prefix `access-6b687c8dcb-*` in the `policy-demo` namespace connected to a workload-endpoints/pods with prefix `nginx-86c57db685-*` exposing a service on port `80`. -- The aggregated source workload endpoints had the labels `app: nginx` and `pod-template-hash: 6b687c8dcb` and the aggregated destination workload endpoint had the labels `app: nginx` and `pod-template-hash: 86c57db685`. -- It was an incoming connection reported by the "Destination" node, and a policy "Allowed" the connection. -- There were 282 incoming packets and 239 outgoing packets. -- Within the aggregation time interval, there were 7 flows aggregated, with 47 new flows started, and 40 flows completed. - -When viewing traffic flows, note that **null values** for `source_ip` and `dest_ip` means **pod prefix aggregation is enabled**. - -``` -... -"source_ip": null, -... -}, -"dest_ip": null, - -``` +{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The +level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all +depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). ### Aggregation types and levels -The following table summarizes the aggregation levels by flow log traffic. - -| Flow log aggregation by... | Available for... | Aggregates all flows that share... | -| -------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Pod prefix | Allowed and denied traffic (default is allowed) | `FlowLogsFileAggregationKindForAllowed`
**0**, No aggregation
**1**, A source port on each node.
**2, (default)** Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | -| Source port | Allowed and denied traffic | `FlowLogsFileAggregationKindForAllowed`
`FlowLogsFileAggregationKindForDenied`
**0**, No aggregation
**1, (default)** Source port on each node.
**2**, Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | +For allowed flows, the default aggregation level is 2, `AnyProcessInSamePodPrefix` and for denied flows the default aggregation level is 1, +`AnyProcessInSameSourcePod`. + +The following table summarizes the aggregation levels by flow log traffic: + +| **Level** | **Name** | **Description** | +|-----------|-------------------------------------|-------------------------------------------------------------------| +| 0 | | No aggregation | +| 1 | AnyProcessInSameSourcePod | Identity fields below source pod level are masked out. It means that if multiple processes or containers, within the same source pod, perform the same operation, the events are aggregated. | +| 2 | AnyProcessInSameSourcePodPrefix | Identity fields below source pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation, the events are aggregated. | +| 3 | AnyProcessInSamePodPrefix | Identity fields below source and destination pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation towards pods with the same prefix, the events are aggregated. | + +### Understanding aggregation level differences + +Here are examples of pod-to-pod flows, highlighting the differences between flow logs at various aggregation levels. + +The source port is usually ephemeral and does not convey useful information. By suppressing the source port, `AnyProcessInSameSourcePod` aggregation +type minimizes the flow logs generated for traffic coming from different containers within the same pod and going to the same destination endpoint +and port. The two flows originating from `client-a` without aggregation are combined into one. + +In Kubernetes, ReplicaSets and StatefulSets can automatically create names for pods. For example, the pods `nginx-1` and `nginx-2` are created by the +ReplicaSet nginx. The ReplicaSet name is considered a pod-prefix and is used to aggregate flow log entries (indicated with an asterisk * at the end +of the name). Flow logs originating from pods with the same prefix will be aggregated as long as the traffic is on the same protocol, and destined +towards the same IP, and destination port. The three flow logs without aggregation originating from `client-a` and `client-b` are combined into a +single flow log. This aggregation level is called `AnyProcessInSameSourcePodPrefix`. + +Finally, with `AnyProcessInSamePodPrefix` we combine source and destination pods that are part of the same ReplicaSets. With level 3, the flow logs +are aggregated by the destination port and protocol, as long as they originate from pods with the same pod-prefix and destined for pods of the same +pod-prefix. All logs previously distinct, are aggregated with into a single flow log (see the last row). + +| | | **Src Traffic** | | | **Dst Traffic** | | | **Packet counts** | | +|--------------------------|-----------|----------|---------|----------|----------|---------|----------|------------|-------------| +| **Aggr lvl** | **Flows** | **Name** | **IP** | **Port** | **Name** | **IP** | **Port** | **Pkt in** | **Pkt out** | +| 0 (no aggr) | 4 | client-a | 1.1.1.1 | 45556 | nginx-1 | 2.2.2.2 | 80 | 1 | 2 | +| | | client-b | 1.1.2.2 | 45666 | nginx-2 | 2.2.3.3 | 80 | 2 | 2 | +| | | client-a | 1.1.1.1 | 65533 | nginx-1 | 2.2.2.2 | 80 | 1 | 3 | +| | | client-c | 1.1.1.2 | 65534 | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 1 (src port) | 3 | client-a | 1.1.1.1 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 5 | +| | | client-b | 1.1.2.2 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 2 | +| | | client-c | 1.1.3.3 | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 2 (src pod-prefix) | 2 | client-* | - | - | nginx-1 | 2.2.2.2 | 80 | 4 | 7 | +| | | client-* | - | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 3 (src/dest pod-prefix) | 1 | client-* | - | - | nginx-* | - | 80 | 7 | 11 | ## How to @@ -112,7 +79,7 @@ The following table summarizes the aggregation levels by flow log traffic. Use the following command: ```bash -kubectl get felixconfiguration.p -o yaml +kubectl get felixconfiguration -o yaml ``` ### Change default aggregation level @@ -138,346 +105,6 @@ In each aggregation interval, connections/connection attempts can be started or The underlying reason for this overlap is a dependency on Linux conntrack, which provides the lifetime of stats that {{prodname}} tracks across different protocols (TCP, ICMP, UDP). For example, for UDP and ICMP, {{prodname}} waits for a conntrack entry to timeout before it considers a “connection” closed, and this is usually greater than 10 seconds. -## Tutorial - -Here are examples of pod-to-pod flows for each aggregation type. - -### Aggregation by source port - -Source port aggregation is straightforward. When viewing `source IP`, `destination IP`, `protocol`, `source port`, `destination port`, the `source port` is usually ephemeral and does not convey useful information. By suppressing `source port`, this aggregation type minimizes the flows logs generated for traffic between the same source-destination endpoint, and same destination port. - -**Node 1** - -``` - { - "start_time": 1597164816, - "end_time": 1597165131, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 0, - "bytes_out": 3120, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 0, - "packets_out": 52, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597165131000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597164816, - "end_time": 1597165125, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "deny", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.deny-access-nginx|deny" - ] - }, - "bytes_in": 3120, - "bytes_out": 0, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 52, - "packets_out": 0, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597165125000 - } -``` - -### Aggregation by pod prefix - -In Kubernetes, pods that are part of ReplicaSets and StatefulSets, etc. can automatically create names for pods. For example, the pods `nginx-65899c769f-tdmw7` and `nginx-65899c769f-xyz123` are created by the ReplicaSet `nginx-65899c769f`. The ReplicaSet name is considered a **pod-prefix** and is used to aggregate flow log entries (shown with an asterisk `*` at the end of the name). For a destination pod to be aggregated, it must have the same "pod-prefix" and expose the same destination port. - -**Node 1** - -``` - { - "start_time": 1597166567, - "end_time": 1597166893, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 52311, - "bytes_out": 18236, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 235, - "packets_out": 282, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166893000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597166567, - "end_time": 1597166879, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52311, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 282, - "packets_out": 235, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166879000 - } -``` - -### No aggregation - -If you turn off aggregation, your log storage may be overwhelmed. Be sure to provision more storage if you do. - -**Node 1** - -``` - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 42106, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166383000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 43092, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166383000 - } -``` - ## Additional resources - [Archive logs to storage](../archive-storage.mdx) diff --git a/calico-enterprise_versioned_docs/version-3.19-2/visibility/elastic/flow/aggregation.mdx b/calico-enterprise_versioned_docs/version-3.19-2/visibility/elastic/flow/aggregation.mdx index 69ade36b7b..5b9417f659 100644 --- a/calico-enterprise_versioned_docs/version-3.19-2/visibility/elastic/flow/aggregation.mdx +++ b/calico-enterprise_versioned_docs/version-3.19-2/visibility/elastic/flow/aggregation.mdx @@ -10,96 +10,63 @@ Configure flow log aggregation level to reduce log volume and costs. ## Value -Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of aggregation is suitable for your implementation. +Beyond using filtering to suppress flow logs, {{prodname}} provides controls to aggregate flow logs. Although aggressive aggregation levels reduce +flow volume and costs, it can also reduce visibility into specific metadata of allowed and denied traffic. Review this article to see which level of +aggregation is suitable for your implementation. ## Concepts -### Aggregation: volume and cost versus visibility +### Volume and cost versus visibility -{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). - -### Aggregation flow log example - -The following is a sample flow log entry using aggregation by pod prefix for allowed traffic. For flow log fields and filter parameters, see [Filter flow logs](filtering.mdx). - -``` - { - "start_time": 1597086849, - "end_time": 1597087167, - "source_ip": null, - "source_name": "-", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": null, - "dest_name": "-", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52519, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 40, - "packets_in": 282, - "packets_out": 239, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597087167000 - } -``` - -- `1597086849` to `1597087167` is the 5 minute aggregation interval. -- Workload endpoints with a similar prefix `access-6b687c8dcb-*` in the `policy-demo` namespace connected to a workload-endpoints/pods with prefix `nginx-86c57db685-*` exposing a service on port `80`. -- The aggregated source workload endpoints had the labels `app: nginx` and `pod-template-hash: 6b687c8dcb` and the aggregated destination workload endpoint had the labels `app: nginx` and `pod-template-hash: 86c57db685`. -- It was an incoming connection reported by the "Destination" node, and a policy "Allowed" the connection. -- There were 282 incoming packets and 239 outgoing packets. -- Within the aggregation time interval, there were 7 flows aggregated, with 47 new flows started, and 40 flows completed. - -When viewing traffic flows, note that **null values** for `source_ip` and `dest_ip` means **pod prefix aggregation is enabled**. - -``` -... -"source_ip": null, -... -}, -"dest_ip": null, - -``` +{{prodname}} enables flow log aggregation for pod/workload endpoints by default, and uses an aggressive aggregation level to reduce log volume. The +level assumes that most users do not need to see pod IP information (due to the ephemeral nature of pod IP address allocation). However, it all +depends on your deployment; we recommend reviewing aggregation levels to understand what information gets grouped (and thus suppressed from view). ### Aggregation types and levels -The following table summarizes the aggregation levels by flow log traffic. - -| Flow log aggregation by... | Available for... | Aggregates all flows that share... | -| -------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Pod prefix | Allowed and denied traffic (default is allowed) | `FlowLogsFileAggregationKindForAllowed`
**0**, No aggregation
**1**, A source port on each node.
**2, (default)** Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | -| Source port | Allowed and denied traffic | `FlowLogsFileAggregationKindForAllowed`
`FlowLogsFileAggregationKindForDenied`
**0**, No aggregation
**1, (default)** Source port on each node.
**2**, Source ports, or are from the same ReplicaSet.
**3**, Destination and source ports, and are from the same ReplicateSet. | +For allowed flows, the default aggregation level is 2, `AnyProcessInSamePodPrefix` and for denied flows the default aggregation level is 1, +`AnyProcessInSameSourcePod`. + +The following table summarizes the aggregation levels by flow log traffic: + +| **Level** | **Name** | **Description** | +|-----------|-------------------------------------|-------------------------------------------------------------------| +| 0 | | No aggregation | +| 1 | AnyProcessInSameSourcePod | Identity fields below source pod level are masked out. It means that if multiple processes or containers, within the same source pod, perform the same operation, the events are aggregated. | +| 2 | AnyProcessInSameSourcePodPrefix | Identity fields below source pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation, the events are aggregated. | +| 3 | AnyProcessInSamePodPrefix | Identity fields below source and destination pod-prefix level are masked out. It means that if multiple processes or containers, within pods with the same prefix, perform the same operation towards pods with the same prefix, the events are aggregated. | + +### Understanding aggregation level differences + +Here are examples of pod-to-pod flows, highlighting the differences between flow logs at various aggregation levels. + +The source port is usually ephemeral and does not convey useful information. By suppressing the source port, `AnyProcessInSameSourcePod` aggregation +type minimizes the flow logs generated for traffic coming from different containers within the same pod and going to the same destination endpoint +and port. The two flows originating from `client-a` without aggregation are combined into one. + +In Kubernetes, ReplicaSets and StatefulSets can automatically create names for pods. For example, the pods `nginx-1` and `nginx-2` are created by the +ReplicaSet nginx. The ReplicaSet name is considered a pod-prefix and is used to aggregate flow log entries (indicated with an asterisk * at the end +of the name). Flow logs originating from pods with the same prefix will be aggregated as long as the traffic is on the same protocol, and destined +towards the same IP, and destination port. The three flow logs without aggregation originating from `client-a` and `client-b` are combined into a +single flow log. This aggregation level is called `AnyProcessInSameSourcePodPrefix`. + +Finally, with `AnyProcessInSamePodPrefix` we combine source and destination pods that are part of the same ReplicaSets. With level 3, the flow logs +are aggregated by the destination port and protocol, as long as they originate from pods with the same pod-prefix and destined for pods of the same +pod-prefix. All logs previously distinct, are aggregated into a single flow log (see the last row). + +| | | **Src Traffic** | | | **Dst Traffic** | | | **Packet counts** | | +|--------------------------|-----------|----------|---------|----------|----------|---------|----------|------------|-------------| +| **Aggr lvl** | **Flows** | **Name** | **IP** | **Port** | **Name** | **IP** | **Port** | **Pkt in** | **Pkt out** | +| 0 (no aggr) | 4 | client-a | 1.1.1.1 | 45556 | nginx-1 | 2.2.2.2 | 80 | 1 | 2 | +| | | client-b | 1.1.2.2 | 45666 | nginx-2 | 2.2.3.3 | 80 | 2 | 2 | +| | | client-a | 1.1.1.1 | 65533 | nginx-1 | 2.2.2.2 | 80 | 1 | 3 | +| | | client-c | 1.1.1.2 | 65534 | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 1 (src port) | 3 | client-a | 1.1.1.1 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 5 | +| | | client-b | 1.1.2.2 | - | nginx-1 | 2.2.2.2 | 80 | 2 | 2 | +| | | client-c | 1.1.3.3 | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 2 (src pod-prefix) | 2 | client-* | - | - | nginx-1 | 2.2.2.2 | 80 | 4 | 7 | +| | | client-* | - | - | nginx-2 | 2.2.3.3 | 80 | 3 | 4 | +| 3 (src/dest pod-prefix) | 1 | client-* | - | - | nginx-* | - | 80 | 7 | 11 | ## How to @@ -112,7 +79,7 @@ The following table summarizes the aggregation levels by flow log traffic. Use the following command: ```bash -kubectl get felixconfiguration.p -o yaml +kubectl get felixconfiguration -o yaml ``` ### Change default aggregation level @@ -138,346 +105,6 @@ In each aggregation interval, connections/connection attempts can be started or The underlying reason for this overlap is a dependency on Linux conntrack, which provides the lifetime of stats that {{prodname}} tracks across different protocols (TCP, ICMP, UDP). For example, for UDP and ICMP, {{prodname}} waits for a conntrack entry to timeout before it considers a “connection” closed, and this is usually greater than 10 seconds. -## Tutorial - -Here are examples of pod-to-pod flows for each aggregation type. - -### Aggregation by source port - -Source port aggregation is straightforward. When viewing `source IP`, `destination IP`, `protocol`, `source port`, `destination port`, the `source port` is usually ephemeral and does not convey useful information. By suppressing `source port`, this aggregation type minimizes the flows logs generated for traffic between the same source-destination endpoint, and same destination port. - -**Node 1** - -``` - { - "start_time": 1597164816, - "end_time": 1597165131, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 0, - "bytes_out": 3120, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 0, - "packets_out": 52, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597165131000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597164816, - "end_time": 1597165125, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "deny", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.deny-access-nginx|deny" - ] - }, - "bytes_in": 3120, - "bytes_out": 0, - "num_flows": 50, - "num_flows_started": 50, - "num_flows_completed": 50, - "packets_in": 52, - "packets_out": 0, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597165125000 - } -``` - -### Aggregation by pod prefix - -In Kubernetes, pods that are part of ReplicaSets and StatefulSets, etc. can automatically create names for pods. For example, the pods `nginx-65899c769f-tdmw7` and `nginx-65899c769f-xyz123` are created by the ReplicaSet `nginx-65899c769f`. The ReplicaSet name is considered a **pod-prefix** and is used to aggregate flow log entries (shown with an asterisk `*` at the end of the name). For a destination pod to be aggregated, it must have the same "pod-prefix" and expose the same destination port. - -**Node 1** - -``` - { - "start_time": 1597166567, - "end_time": 1597166893, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "src", - "policies": { - "all_policies": [ - "0|__PROFILE__|__PROFILE__.kns.policy-demo|allow" - ] - }, - "bytes_in": 52311, - "bytes_out": 18236, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 235, - "packets_out": 282, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166893000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597166567, - "end_time": 1597166879, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": null, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.213.223", - "dest_name": "nginx-86c57db685-rkhnf", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 18236, - "bytes_out": 52311, - "num_flows": 47, - "num_flows_started": 47, - "num_flows_completed": 47, - "packets_in": 282, - "packets_out": 235, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166879000 - } -``` - -### No aggregation - -If you turn off aggregation, your log storage may be overwhelmed. Be sure to provision more storage if you do. - -**Node 1** - -``` - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 42106, - "source_type": "wep", - "source_labels": { - "labels": [ - "pod-template-hash=6b687c8dcb", - "app=access" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "pod-template-hash=86c57db685", - "app=nginx" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-1", - "@timestamp": 1597166383000 - } -``` - -**Node 2** - -``` - { - "start_time": 1597166083, - "end_time": 1597166383, - "source_ip": "192.168.47.9", - "source_name": "access-6b687c8dcb-zn5s2", - "source_name_aggr": "access-6b687c8dcb-*", - "source_namespace": "policy-demo", - "source_port": 43092, - "source_type": "wep", - "source_labels": { - "labels": [ - "app=access", - "pod-template-hash=6b687c8dcb" - ] - }, - "dest_ip": "192.168.138.79", - "dest_name": "nginx-86c57db685-h6792", - "dest_name_aggr": "nginx-86c57db685-*", - "dest_namespace": "policy-demo", - "dest_port": 80, - "dest_type": "wep", - "dest_labels": { - "labels": [ - "app=nginx", - "pod-template-hash=86c57db685" - ] - }, - "proto": "tcp", - "action": "allow", - "reporter": "dst", - "policies": { - "all_policies": [ - "0|default|policy-demo/default.access-nginx|allow" - ] - }, - "bytes_in": 388, - "bytes_out": 1113, - "num_flows": 1, - "num_flows_started": 1, - "num_flows_completed": 1, - "packets_in": 6, - "packets_out": 5, - "http_requests_allowed_in": 0, - "http_requests_denied_in": 0, - "original_source_ips": null, - "num_original_source_ips": 0, - "host": "bz-n8kf-kadm-node-2", - "@timestamp": 1597166383000 - } -``` - ## Additional resources - [Archive logs to storage](../archive-storage.mdx)