Skip to content

A multi-threading tool to sniff TCP flow statistics and embedded HTTP headers from PCAP file. Each TCP flow carrying HTTP is exported to text file in json format.

License

MIT, Unknown licenses found

Licenses found

MIT
LICENSE
Unknown
COPYING
Notifications You must be signed in to change notification settings

caesar0301/http-sniffer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

http-sniffer

Build Status

A multi-threading tool to sniff HTTP header records beyond TCP flow statistics.

MIT licensed.

Features

  • Support both offline PCAP file and live NIC sniffing
  • Perform multi-threading process, whereby the tool gets higher performance in face of big tarffic volume
  • Export statistics of TCP flows
  • Export HTTP request/response pairs if they are present in the TCP flow
  • Support JSON format output

Dependencis

The http-sniffer depends on other utilities:

  • libpcap to extract traffic packet,
  • json-c to support json parsing, and
  • scons to build the project.

A known issue about json-c is the unused params errors on linux platform, you can use an alternative to solve it. You may also need to export the library path where libjson-c.* locate, by

export LD_LIBRARY_PATH=/usr/local/lib

How to Use

Run scons in root folder to compile:

$ cd http-sniffer
$ scons

Get your live interface with ifconfig in terminal, e.g. en0, then

$ ./bin/http-sniffer -i en0

Or store output flows as json

$ ./bin/http-sniffer -i en0 -o output.json

Output

  • In brief CSV format:
[20120921 16:40:09]10.187.179.28:53196-->180.149.134.229:80 1335164797.208360 0.0 0.0 167 5/3 0/0 0 0
[20120921 16:40:09]10.187.179.28:53160-->58.63.234.206:80 1335164789.893109 0.0 0.0 21 4/2 0/0 0 0
[20120921 16:40:09]10.187.179.28:53161-->58.63.234.206:80 1335164789.893219 0.0 0.0 225 4/2 0/0 0 0
[20120921 16:40:09]10.187.179.28:53158-->58.63.234.198:80 1335164789.769004 0.0 0.0 118 4/2 0/0 0 0
[20120921 16:40:09]10.187.179.28:53164-->113.108.216.252:80 1335164790.179680 0.0 0.0 12 4/2 0/0 0 0
[20120921 16:40:09]10.187.179.28:53189-->180.149.134.221:80 1335164797.961918 0.0 0.0 111 3/1 0/0 0 0
  • In full JSON format: each line records one TCP flow with piggybacked HTTP messages, e.g.
{
   "time_local": "2012-04-23T13:50:09",
   "saddr": "192.168.1.4", 
   "daddr": "192.168.1.5", 
   "sport": 45753, 
   "dport": 80, 
   "time_syn": 1335160209.417475, 
   "time_first_byte": 1335160209.452336, 
   "time_last_byte": 1335160209.488276, 
   "rtt": 248, 
   "src_packets": 4, 
   "dst_packets": 4, 
   "src_bytes": 521, 
   "dst_bytes": 257, 
   "http_pair_count": 1, 
   "force_closed": 0, 
   "http_pairs": [ 
    { 
     "request": { 
      "time_first_byte": 1335160209.452336, 
      "time_last_byte": 1335160209.452336, 
      "bytes_transfered": 521, 
      "http_version": 1, 
      "method": 1, 
      "host": "s1.bdstatic.com", 
      "uri": "\/r\/www\/img\/i-1.0.0.png", 
      "referer": "http:\/\/www.baidu.com\/", 
      "user_agent": "Mozilla\/5.0", 
      "accept": "image\/png,image\/*;q=0.8,*\/*;q=0.5", 
      "accept_encoding": "gzip,deflate", 
      "accept_language": "en-us,en;q=0.5", 
      "accept_charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.7"}, 
     "response": {
      "time_first_byte": 1335160209.488260, 
      "time_last_byte": 1335160209.488260, 
      "bytes_transfered": 257, 
      "http_version": 1, 
      "status": 304, 
      "server": "JSP\/1.0.18", 
      "date": "Mon, 23 Apr 2012 06:02:23 GMT", 
      "expires": "Tue, 29 Mar 2022 09:34:06 GMT", 
      "etag": "\"25f-4a6ebc21c42c0\"", 
      "last_modified": "Thu, 30 Jun 2011 10:56:51 GMT"} 
    }] 
}

About Author

Xiaming Chen chen@xiaming.me

SJTU, Shanghai, China

2012-04-01

About

A multi-threading tool to sniff TCP flow statistics and embedded HTTP headers from PCAP file. Each TCP flow carrying HTTP is exported to text file in json format.

Resources

License

MIT, Unknown licenses found

Licenses found

MIT
LICENSE
Unknown
COPYING

Stars

Watchers

Forks

Packages

No packages published