Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LZ4 compression for pcap logs #3340

Closed
wants to merge 3 commits into from
Closed

Conversation

mfil
Copy link

@mfil mfil commented Apr 12, 2018

Link to redmine ticket:
https://redmine.openinfosecfoundation.org/issues/2456

Describe changes:

  • Add option to compress pcap log files with lz4 before writing to disk

Update to #3188, #3200, #3206, #3250, #3283, and #3285

  • fix bugs and formatting issues pointed out by victorjulien and jasonish, including the memory leaks
  • add lz4 file extension
  • rename lz4-compression-level option to lz4-level
  • squash fixup commits
  • exit with error when asked to use compression in sguil mode
  • in PcapLogInitCtx, exit in case of memory allocation errors, return in case of configuration errors

Max Fillinger added 3 commits April 12, 2018 17:15
Introduces the option 'outputs.pcap-log.compression' which can be set
to 'none' or 'lz4', plus options to set the compression level and to
enable checksums. SCFmemopen is used to make pcap_dump() write to a
buffer which is then compressed using liblz4.
Not sure about the Fedora case...
@mfil mfil requested review from norg and a team as code owners April 12, 2018 16:08
Copy link
Member

@jasonish jasonish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue addressed. I'm going to fixup the commit to configure.ac to make it correct for Fedora, plus a spelling mistake.

@jasonish
Copy link
Member

Rebased and correct configure instructions for Fedora: #3357

@mfil
Copy link
Author

mfil commented Apr 26, 2018

I can close this one now, right?

@victorjulien
Copy link
Member

Replaced by #3357

glongo added a commit to glongo/suricata that referenced this pull request Nov 19, 2019
Like bug OISF#3340, SIP TCP transactions are logged with
source and destination addresses swapped.
But in this case logging logic needed to be reworkerd,
because source and destination
addresses were not logged correctly with regard to the flow direction.
In this example below, there are 2 requests and 1 response but
source and destination addresses are not correct.

{"timestamp":"2019-11-17T07:43:57.799196-0600","flow_id":56712067097706,"pcap_cnt":5,"event_type":"sip","src_ip":"192.168.43.1","src_port":43168,"dest_ip":"192.168.43.100","dest_port":5060,"proto":"TCP",
 "sip":{"method":"REGISTER","uri":"sip:192.168.43.100;transport=TCP","version":"SIP/2.0","request_line":"REGISTER sip:192.168.43.100;transport=TCP SIP/2.0"}}

{"timestamp":"2019-11-17T07:43:57.802382-0600","flow_id":56712067097706,"pcap_cnt":7,"event_type":"sip","src_ip":"192.168.43.1","src_port":43168,"dest_ip":"192.168.43.100","dest_port":5060,"proto":"TCP",
 "sip":{"version":"SIP/2.0","code":"200","reason":"OK","response_line":"SIP/2.0 200 OK"}}

{"timestamp":"2019-11-17T07:43:57.806796-0600","flow_id":56712067097706,"pcap_cnt":9,"event_type":"sip","src_ip":"192.168.43.1","src_port":43168,"dest_ip":"192.168.43.100","dest_port":5060,"proto":"TCP",
 "sip":{"method":"REGISTER","uri":"sip:192.168.43.100;transport=TCP","version":"SIP/2.0","request_line":"REGISTER sip:192.168.43.100;transport=TCP SIP/2.0"}}
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Nov 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants