CEF (Common Event Format)
Common Event Format Interceptor for Apache Flume. Use this interceptor to parse and convert CEF payloads in Syslog to Flume Event headers.
Common Event Format commonly known as CEF (pronounced sef), is the defacto format of HP ArcSight Logging and Security monitoring product family (including Loggers and ESM), you can read more about CEF here: https://www.protect724.hpe.com/docs/DOC-1072. It's also widely adopted by a few security products making parsing it critical to interpreting the logs and making sense out of them. CEF is usually transmitted over Syslog protocol as a payload.
This parser is a available in form of an Apache Flume Interceptor (https://flume.apache.org/FlumeUserGuide.html#flume-interceptors) and is best used in conjunction with SyslogSources (TCP / UDP) however you can use it with any other Flume Source as long as the byte body of emitted from the Flume Source is CEF only.
Note: This parser currently doesn't validate events beyond CEF Prefix and Invalid extensions. It currently doesn't check dictionary validation or data type validation for dictionary items. This is partly because Flume headers are String valued therefore type conversion has no benefit and as well as for performance reasons.
This version of CEF Interceptor is complaint with Apache Flume 1.6.0
How to use?
Please download the published artifact from Maven Central, 0.0.11 or greater is current stable version.
<dependency> <groupId>com.srotya.flume</groupId> <artifactId>cef</artifactId> <version>0.0.11</version> </dependency>
The parser has no external dependencies so all you need to do is download the latest JAR from Maven Central to the lib folder of your Flume installation and configure the interceptor as shown below and start sending some CEF logs:
agent.sources = syslog agent.channels = memoryChannel agent.sinks = loggerSink agent.sources.syslog.type = syslogtcp agent.sources.syslog.port = 1514 agent.sources.syslog.interceptors = cef agent.sources.syslog.interceptors.cef.type = com.srotya.flume.cef.CEFInterceptor$Builder agent.sources.syslog.channels = memoryChannel agent.sinks.loggerSink.type = logger agent.sinks.loggerSink.channel = memoryChannel agent.channels.memoryChannel.type = memory agent.channels.memoryChannel.capacity = 100
Bugs and Issues?
Please report bugs via github issues.