Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Hadoop library to read packet capture (PCAP) files
Java
tag: 0.1

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
hadoop-pcap-lib
hadoop-pcap-serde
LICENSE
README.md
pom.xml

README.md

Hadoop PCAP library

License

This library is distributed under the LGPL. See: https://raw.github.com/RIPE-NCC/hadoop-pcap/master/LICENSE

Usage

Important Do not forget to add the library before trying the examples below:

ADD JAR hadoop-pcap-serde-0.1-jar-with-dependencies.jar;

DNS table on HDFS

CREATE EXTERNAL TABLE pcaps (ts bigint,
                             protocol string,
                             src string,
                             src_port int,
                             dst string,
                             dst_port int,
                             len int,
                             ttl int,
                             dns_queryid int,
                             dns_flags string,
                             dns_opcode string,
                             dns_rcode string,
                             dns_question string,
                             dns_answer array<string>,
                             dns_authority array<string>,
                             dns_additional array<string>)
ROW FORMAT SERDE 'net.ripe.hadoop.pcap.serde.PcapDeserializer'
STORED AS INPUTFORMAT 'net.ripe.hadoop.pcap.io.DnsPcapInputFormat'
          OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION 'hdfs:///pcaps/';

PCAP table on Amazon S3

CREATE EXTERNAL TABLE pcaps (ts bigint,
                             protocol string,
                             src string,
                             src_port int,
                             dst string,
                             dst_port int,
                             len int,
                             ttl int)
ROW FORMAT SERDE 'net.ripe.hadoop.pcap.serde.PcapDeserializer' 
STORED AS INPUTFORMAT 'net.ripe.hadoop.pcap.io.PcapInputFormat' 
          OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' 
LOCATION 's3n://pcaps/';
Something went wrong with that request. Please try again.