Skip to content
Main Repository for bwNetFlow - Open Source Network Flow Analysis Suite
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs
README.md
_config.yml
overview.png
overview.svg

README.md

bwNetFlow - Open Source Network Flow Analysis Suite

bwNetFlow Overview

The bwNetFlow suite uses existing software and provides glue codes to allow for large scale network flow analysis.

bwNetFlow uses GoFlow and Apache Kafka to process network flow via NetFlow and Kafka.

We provide a set of tools working with Kafka as consumers and/or producers to establish a flow monitoring analysis pipeline. While these tools can be combined in any specific way, the core components and wiring in our use case is as follows:

  • GoFlow: Cloudflare's GoFlow receives NetFlow and produces protobuf messages in Kafka topic input
  • Enricher: reads from Kafka topic input, adds domain specific knowlege (customer, direction, device info, etc), and writes protobuf messages in Kafka topic enriched
  • Splitter: reads from Kafka topic enriched, writes into customer specific topics enriched-$cid for each enabled customer
  • Dashboard: reads from Kafka topic enriched, aggregates the flows to counters and writes these counter values to a Time Series Database (InfluxDB or Prometheus)

The tools work with Protobuf messages for representing NetFlow packets from GoFlow - yet with an extended protobuf message as soon as enriched by the enricher component.

Other Tools:

To develop Kafka consumers/producers with Go the kafkaconnector library abstracts most of the recurrent code fragments. To develop Kafka consumers/producers with C++ the cpp_kafkaconnector library abstracts most of the recurrent code fragments.

Deployment

For deploying the bwNetFlow suite we provide Ansible scripts or Docker / Docker-Compose description.

tbd

You can’t perform that action at this time.