FastNetMon Advanced 2.0.373

Changes:

• Added logic to print agent IP address when we have error with parsing sFlow samples
• Added logic to export other and internal traffic in usage telemetry
• Breaking change: sudo fcli show traffic_buffer will start returning packets in JSON instead of logging them to log file
• Added GCore support for scrubbing integration
• Update scrubbing integration to latest version to add gCore support
• Added full logic to implement Flow Spec based allow list
• Added logic to lock down Netflow v5, v9 and IPFIX to only specific list of routers: ipfix_devices_allow_list, netflow_v9_devices_allow_list, netflow_v5_devices_allow_list
• Added logic to populate input and output interfaces information in AF_PACKET mode by using internal linux network number which can be got via sudo ip link in first column
• Allowed usage of CGNAT for internal IP address for external licenses
• Deprecated CentOS 7

FastNetMon Advanced 2.0.372

Changes:

• Added support for IPv6 sudo sudo fcli show ip_asn command
• Addressed crash when ASN feed has malformed data
• Reworked print_ipv6_address to use Boost based functions to print IPv6 addresses as our own IPv6 compression broke up addresses like 0000::1/128 and printed them as :0001/128
• Moved away from lookup_tree_32bit_with_dynamically_allocated_payload_t to lookup_tree_32bit_with_payload_t which stores data in leafs and does not use intermediate pointers. Some speed up for ASN allocation is expected. As side affect we have improved sanity checks for input data
• Re-enabled usage stats for offline licenses
• Enabled sflow_track_sampling_rate by default
• Small improvement for performance of sFlow sampling rate tracking
• Addressed bug in Mikrotik API based integration which used /128 instead of /32
• Removed Debian 9, 10 and Ubuntu 16.04 as we deprecated them long time ago
• Implemented move constructor to address issue with gcc 13 and LLVM, GitHub #1013
• Added support for IPFIX fieds with IDs 85 and 86 to implement Juniper SRX support
• Fixed content type for Prometheus endpoint to be text/plain
• Reduced logging level for: Successfully loaded flexible threshold
• Added log level checking for Netflow v9 logic as it may be root cause of performance degradation
• Added log level checking for IPFIX logic as it may be root cause of performance degradation
• Suppressed error 'process_ipfix_sets returned error' when we have no template to continue processing for other flowsets
• Added logic to agent address matches in Flow Cpec filters
• Added input_interface_matches and output_interface_matches for Flow Spec based filter
• Added input_interfaces and output_interfaces to Flow Spec structure

FastNetMon Advanced 2.0.371

Changes:

• IPFIX plugin, added logic to gracefully handle padding on the end of data set. Previously it caused following error: Attempt to read data after end of flowset. Offset: 0 record length: 4 flowset_maximum_length: 2
• Reversed order of checks in IPFIX and Netflow v9 logic for forwarding status detection for Juniper reported by Samuel K. Lam
• Added sanity checks for IPFIX variable length encoding
• Added support for IPFIX enterprise fields used by Arista
• Implemented logic to read multiple options templates in IPFIX packets. Implemented logic to detect padding on the end of IPFIX options template set
• Reworked ipfix_data_set_header_t to generic ipfix_set_header_common_t to unify logic and reduce number of duplicated structures
• Fixed issues that ipfix_variable_length_elements_used was accidentally set to all templates in SET even if just one had it. Extracted logic to read IPFIX template into dedicated function, ongoing refactoring
• Rename ipfix_flowsets_with_anomaly_padding to ipfix_sets_with_anomaly_padding as part of getting rid non RFC compliant naming
• Moved away from endian-less conversions in IPFIX logic
• Added enterprise_bit and enterprise_number for IPFIX and Netflow v9 templates
• Additional sanity checks for IPFIX template logic. Added explicit length check for enterprise numbers
• Added explicit IPFIX structures length checking.
• Extracted IPFIX set processing to function process_ipfix_sets
• Added detailed logging to trace how we parse multiple data templates
• Added test for Cisco 315 with multiple flows exported
• Added logic to ensure that we read sampling rates correctly and added check that number of flows read correctly
• Added test coverage for Nokia which sends data and template templates in same packet
• Added logic to test when data and options template is carried in single packet
• Typos reported by Patrick Matthai
• Added information to show sequence of IPFIX packet for easier debugging
• Added FastNetMon daemon dependency on network-online.target to ensure that network is configured and Internet connection is available, more details: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

FastNetMon Advanced 2.0.370

Changes:

• Added sanity check in IPFIX code to avoid reading outside of our memory region
• Added sanity check in Netflow v9 code to avoid reading outside of our memory region
• Added safety check in IPFIX to avoid potential division by zero
• DoS: explicitly blocked zero length data templates for Netflow v9 as they have no sense
• DoS: explicitly blocked zero length options templates for Netflow v9 as they have no sense
• DoS: Added fix for FPE / division by zero in Netflow v9 logic when length of template is zero, CVE CVE-2024-56073
• Added explicit check about number of counter records in sFlow packet to reduce chances of DoS attack
• Added explicit check about number of flow records in sFlow packet to reduce chances of DoS attack
• Fixed DoS vulnerability in sFlow v5 plugin which crashed FastNetMon with specially crafted packet, CVE-2024-56072
• Added logic to correctly populate hostgroup for Flow Spec announces injected manually
• Moved current attack logic up in function to grant space for hsotgroup lookup
• Switched text/html to text/plain for Prometheus endpoint: https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md
• Fixed bug with traffic buffer size reporting for IPv6: IPv6 traffic buffer is too small to generate attack_traffic_samples correctly and IPv6 traffic buffer is too small to generate hostgroup_traffic_samples correctly
• Added Kafka support for traffic export via configuration options kafka_traffic_export, kafka_traffic_export_topicm kafka_traffic_export_format, kafka_traffic_export_brokers for Kafka traffic export'

FastNetMon Advanced 2.0.369

Changes:

• Fixed bug with endpoints total_traffic_counters_v4 and total_traffic_counters_v6 which did not work and required not needed parameters

FastNetMon Advanced 2.0.368

Changes:

• Complete multi user support for API via users_configuration and roles_configuration sections
• Improved log messages Traffic buffer size for IPv6 is too small to accommodate whole traffic calculation period. Please increase traffic buffer size to add more details
• Added logic to use source_asn and destination_asn for filtering for Flow Spec mode and Flow Spec white list
• Added source_asns and destination_asns to JSON representation of Flow Spec announces
• Added logic to return ipv4_nexthops for sudo fcli show flowspec
• Added new option for AF_PACKET to unpack GTPv1 tunnels: af_packet_extract_gtp_v1_tunnels
• Implemented API for sudo fcli show blackhole uuid
• Switched protocol version to lowercase in blackhole details
• Added command sudo fcli show blackhole UUID to show basic information about attack
• Behaviour change: removed legacy fields attack_type, initial_attack_power, peak_attack_power, attack_protocol, attack_direction for attack callbacks, MongoDB and POST callbacks
• Behaviour change: removed legacy fields in attack notifications average_incoming_traffic, average_incoming_traffic_bits, average_outgoing_traffic, average_outgoing_traffic_bits, average_incoming_pps, average_outgoing_pps, average_incoming_flows, average_outgoing_flows
• Removed legacy fields attack_direction and attack_power from attack details information
• Behaviour change: we reworked logic for attack direction and attack power in PPS arguments for legacy Bash callback scripts to use modern approach for calculations
• Added logic to calculate number of flows we keep in storage for TCP, UDP, ICMP and other protocols and exposed them via system counters
• Added new metrics for TCP, UDP, ICMP flow tracking structures
• Added new counters entries_flow_tracking_tcp_ipv4, entries_flow_tracking_udp_ipv4, entries_flow_tracking_icmp_ipv4
• Renamed counter entries_flow_tracking to entries_flow_tracking_ipv4
• Added support for new forwardingStatus 4 byte encoding which is used by Cisco ASR9006 with IOS XR 6.4.2

FastNetMon Advanced 2.0.367

Changes:

• Added new type string for system_counters and exposed FastNetMon version
• Added logic to store unban actions to attacks collection in MongoDB
• Fixed bug that clickhouse_metrics_push_period was used in Graphite plugin instead of correct graphite_push_period
• Change in BGP Flow Spec mitigation logic: DF (don't fragment) can be combined with other fragmentations flags. Previously if we had DF flag then we set only it and ignored all other fragmentation flags
• Added export for ip_dont_fragment, ip_more_fragments, ip_fragment_offset fields for packet dump
• Added logic to remove source and destination ports when flow spec rules has is-fragment flag. We do it for Arista due to their hardware limitations. It can be enabled using flag flow_spec_strip_ports_for_fragmented_traffic
• Deprecated field attack_protocol for JSON callbacks and explicitly set it to unknown value. Please use threshold names for attack protocol detection instead
• Deprecated fields peak_attack_power and initial_attack_power for JSON callback and set them to zeroes. Please use per protocol counters instead
• Manually set field attack_type for JSON based attack notifications to unknown as this logic was broken and we're deprecating it
• We stopped populating fields: average_incoming_traffic, average_incoming_traffic_bits, average_outgoing_traffic, average_outgoing_traffic_bits, average_incoming_pps, average_outgoing_pps, average_incoming_flows, average_outgoing_flows
• Removed deprecated attack detection fields from email alerts: Attack type, Initial attack power, Peak attack power, Attack direction, Attack protocol. These fields were calculated by very flawed logic and we replaced them by new fields which precisely reflect reality of attack.
• Added configuration options gobgp_as_path_host_ipv4, gobgp_as_path_subnet_ipv4, gobgp_as_path_host_ipv6, gobgp_as_path_subnet_ipv6
• Added bgp_as_path_host_ipv4, bgp_as_path_subnet_ipv4, bgp_as_path_host_ipv6, bgp_as_path_subnet_ipv6 for all hostgroups
• Improved logic to craft IPv4 BGP attributes
• We deprecated Ubuntu 16.04, Debian 9 and 10 and removed them from releases
• Fixed bug with too high values in sudo fcli show baseline_per_host global outgoing for outgoing traffic
• Adjusted Clickhouse datasource UID for password reset logic
• Changed Clickhouse datasource to new UUID format
• Added logic to set correct owner for /var/lib/grafana/plugins
• Added fcli flag NO_DATABASE_MODE which can be set to on to suppress connection attempts to MongoDB

FastNetMon Advanced 2.0.366

Changes:

• Adding option to pcap reader to load networks list from /tmp/networks_list_pcap.dat
• Added cron installation for Panel
• Added counter to track UDP packets for Netflow or IPFIX plugin which exceed 1500 bytes
• Switched Nginx signature to binary format
• Changed the way how we sign key for Ubuntu and Debian for Nginx
• Added explicit checks that repos work on Debian platforms
• Added logic to verify that MongoDB repo works fine on Ubuntu and Debian
• Added explicit check that FastNetMon and Grafana repos fro Debian and Ubuntu work fine before installing anything from them
• Added logic to check that FastNetMon repo works before installing from it
• Added logic to explicitly check that Nginx repo works
• Finished logic to set password in Grafana and for Nginx http auth in same time
• Moved to new logic which just replicates password of Grafana user on Nginx auth as old scheme was broken around Grafana 11 and we weren't able to pin point issue
• Added logic to pass IPv6 address in Netflow and IPFIX plugin. Also improve logic to use IPv6 address as is in pcap readers
• Deprecated visual stack for Ubuntu 16.04 because Clickhouse is not working on it
• Switched MongoDB to FerretDB for Ubuntu 16.04
• Switched to use FerretDB on Debian and Ubuntu and RedHat machines without AVX or when old CPU is forced via CLI
• Added per protocol counters for asn_counters_v4 and asn_counters_v6
• Added logic to respect unit passed to total_traffic_counters using GET query bandwidth_unit set to bps
• Added logic to pass unit to FNM internal API
• Added installer flag to install only FerretDB
• Added logic to start GoBGP daemon to eliminate errors like context deadline exceeded in Panel
• Added logic to export per protocol counters for network counters and per interface counters endpoints
• Added per protocol counters for networks and interfaces
• Unified networks_counters for IPv6 per network counters
• Ported ASN counters to new per protocol counters logic
• Added per protocol counters for ASNs and for networks for internal API
• Added optional capability to use TTL for Flow spec attack detection
• Added proxy_set_header for Grafana to address origin not allowed issue
• Introduced option to calculate speed in parallel. Please note that it performs best if you have ban disabled
• Added configuration options parallel_speed_calculation and parallel_speed_calculation_threads
• Implemented previously missing logic for flow_spec_ignore_do_not_fragment_flag
• Changed permissions for systemd unit files for CentOS family from 755 to 644 as RHEL 8 does not like it: Configuration file /usr/lib/systemd/system/fastnetmon.service is marked executable. Please remove executable permission bits. Proceeding anyway
• Added flows_per_second sorter for Partner integration
• Added FerretDB support for logic to import community configuration
• Added logic to use Nginx repository from Ubuntu 22.04 Jammy for Ubuntu 24.04 as we do not have official one yet
• Added Ubuntu 24.04 support to installer

FastNetMon Advanced 2.0.365

Changes:

• Multiple improvements for licensing logic

FastNetMon Advanced 2.0.364

Changes:

• Added BGP peering configuration options ipv4_unicast_add_path and ipv6_unicast_add_path to control add path logic with 8 routes for each prefix
• When you specify single value in bgp_next_hops_subnet_ipv4, bgp_next_hops_host_ipv4, bgp_next_hops_subnet_ipv6, bgp_next_hops_host_ipv6 for hostgroup it overrides default value in configuration
• Introduced gobgp_flow_spec_v4_redirect_target_as, gobgp_flow_spec_v4_redirect_target_community, gobgp_flow_spec_v6_redirect_target_as, gobgp_flow_spec_v6_redirect_target_community to control IPv6 Flow Spec redirect
• Added warning message to log when capacity of traffic buffer is not enough to accommodate generate_hostgroup_traffic_samples_delay or generate_attack_traffic_samples_delay