Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
// network_observability/configuring-operator.adoc

:_content-type: PROCEDURE
[id="network-observability-enriched-flows-kafka_{context}"]
[id="network-observability-enriched-flows_{context}"]
= Export enriched network flow data

You can send network flows to Kafka, so that they can be consumed by any processor or storage that supports Kafka input, such as Splunk, Elasticsearch, or Fluentd.
You can send network flows to Kafka, IPFIX, or both at the same time. Any processor or storage that supports Kafka or IPFIX input, such as Splunk, Elasticsearch, or Fluentd, can consume the enriched network flow data.

.Prerequisites
* Installed Kafka
* Your Kafka or IPFIX collector endpoint(s) are available from Network Observability `flowlogs-pipeline` pods.

.Procedure

Expand All @@ -26,14 +26,22 @@ metadata:
name: cluster
spec:
exporters:
- type: KAFKA
- type: KAFKA <3>
kafka:
address: "kafka-cluster-kafka-bootstrap.netobserv"
topic: netobserv-flows-export <1>
tls:
enable: false <2>
- type: IPFIX <3>
ipfix:
targetHost: "ipfix-collector.ipfix.svc.cluster.local"
targetPort: 4739
transport: tcp or udp <4>


----
<1> The Network Observability Operator exports all flows to the configured Kafka topic.
<2> You can encrypt all communications to and from Kafka with SSL/TLS or mTLS. When enabled, the Kafka CA certificate must be available as a ConfigMap or a Secret, both in the namespace where the `flowlogs-pipeline` processor component is deployed (default: netobserv). It must be referenced with `spec.exporters.tls.caCert`. When using mTLS, client secrets must be available in these namespaces as well (they can be generated for instance using the AMQ Streams User Operator) and referenced with `spec.exporters.tls.userCert`.
. After configuration, network flows data can be sent to an available output in a JSON format. For more information, see _Network flows format reference_
<3> You can export flows to IPFIX instead of or in conjunction with exporting flows to Kafka.
<4> You have the option to specify transport. The default value is `tcp` but you can also specify `udp`.
. After configuration, network flows data can be sent to an available output in a JSON format. For more information, see _Network flows format reference_.
2 changes: 1 addition & 1 deletion networking/network_observability/configuring-operator.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ include::modules/network-observability-flowcollector-view.adoc[leveloffset=+1]
For more information about conversation tracking, see xref:../../networking/network_observability/observing-network-traffic.adoc#network-observability-working-with-conversations_nw-observe-network-traffic[Working with conversations].

include::modules/network-observability-flowcollector-kafka-config.adoc[leveloffset=+1]
include::modules/network-observability-enriched-flows-kafka.adoc[leveloffset=+1]
include::modules/network-observability-enriched-flows.adoc[leveloffset=+1]

[role="_additional-resources"]
.Additional resources
Expand Down
2 changes: 1 addition & 1 deletion networking/network_observability/installing-operators.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ include::modules/network-observability-operator-install.adoc[leveloffset=+1]
.Additional resources
* For more information about Flow Collector specifications, see the xref:../../networking/network_observability/flowcollector-api.adoc#network-observability-flowcollector-api-specifications_network_observability[Flow Collector API Reference] and the xref:../../networking/network_observability/configuring-operator.adoc#network-observability-flowcollector-view_network_observability[Flow Collector sample resource].

* For more information about exporting flow data to Kafka for third party processing consumption, see xref:../../networking/network_observability/configuring-operator.adoc#network-observability-enriched-flows-kafka_network_observability[Export enriched network flow data].
* For more information about exporting flow data to Kafka or IPFIX for third party processing consumption, see xref:../../networking/network_observability/configuring-operator.adoc#network-observability-enriched-flows_network_observability[Export enriched network flow data].

include::modules/network-observability-operator-uninstall.adoc[leveloffset=+1]