Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zeek module addon #14150

Merged
merged 15 commits into from
Oct 25, 2019
Merged

Zeek module addon #14150

merged 15 commits into from
Oct 25, 2019

Conversation

alakahakai
Copy link

@alakahakai alakahakai commented Oct 19, 2019

Took over PR #13683 and finish it up.

  • Added 30+ filesets to the Zeek module

EDIT by @cwurm

These filesets have been added (in addition to the ones that were already present: connection, dns, http, files, ssl, notice):

  1. capture_loss
  2. dce_rpc
  3. dhcp
  4. dnp3
  5. dpd
  6. ftp
  7. http
  8. irc
  9. kerberos
  10. modbus
  11. mysql
  12. ntlm
  13. ocsp
  14. pe
  15. radius
  16. rdp
  17. rfb
  18. signatures
  19. sip
  20. smb_cmd
  21. smb_files
  22. smb_mapping
  23. smtp
  24. snmp
  25. socks
  26. ssh
  27. stats
  28. syslog
  29. traceroute
  30. tunnel
  31. weird
  32. x509

@alakahakai alakahakai requested a review from a team as a code owner October 19, 2019 17:32
@alakahakai alakahakai added module Filebeat Filebeat in progress Pull request is currently in progress. labels Oct 19, 2019
@alakahakai alakahakai added review and removed in progress Pull request is currently in progress. labels Oct 19, 2019
@alakahakai alakahakai requested a review from a team October 19, 2019 21:39
@alakahakai
Copy link
Author

jenkins, test this

@alakahakai
Copy link
Author

There are some unrelated build test failures on iis, nginx, etc.

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left a few comments with doubts and some small mistakes.

x-pack/filebeat/module/zeek/dhcp/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/ftp/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/irc/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/modbus/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/sip/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/sip/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/smb_cmd/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/snmp/_meta/fields.yml Outdated Show resolved Hide resolved
x-pack/filebeat/module/zeek/stats/_meta/fields.yml Outdated Show resolved Hide resolved
@adriansr
Copy link
Contributor

jenkins, test this

@alakahakai alakahakai merged commit c275fbc into elastic:master Oct 25, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

andrewkroh pushed a commit to garanews/beats that referenced this pull request Nov 18, 2019
Enrich the elastic#14150 supporting intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>
andrewkroh pushed a commit that referenced this pull request Nov 19, 2019
* Added support for intel.log zeek module

Enrich the #14150 supporting intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Update fields.yml

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* intel.log

example intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* added default_field: false

added default_field: false

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Generate expected zeek/intel output event

* Add changelog entry

* Update field docs

* Misc improvements

Expand dots in zeek.intel.seen
Parse ts value without dropping millisecond value
Add event.ingested timestamp
Convert ingest node pipeline to YAML
Save JSON message in event.original

* Updates to zeek.intel.seen

Expand dots of all seen.* fields
Change name of zeek.intel.seen.fa_file to zeek.intel.seen.f as documented by Zeek.

* Update field docs
andrewkroh pushed a commit to andrewkroh/beats that referenced this pull request Nov 19, 2019
* Added support for intel.log zeek module

Enrich the elastic#14150 supporting intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Update fields.yml

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* intel.log

example intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* added default_field: false

added default_field: false

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Generate expected zeek/intel output event

* Add changelog entry

* Update field docs

* Misc improvements

Expand dots in zeek.intel.seen
Parse ts value without dropping millisecond value
Add event.ingested timestamp
Convert ingest node pipeline to YAML
Save JSON message in event.original

* Updates to zeek.intel.seen

Expand dots of all seen.* fields
Change name of zeek.intel.seen.fa_file to zeek.intel.seen.f as documented by Zeek.

* Update field docs

(cherry picked from commit 7ad14e6)
andrewkroh added a commit that referenced this pull request Nov 22, 2019
* Added support for intel.log zeek module

Enrich the #14150 supporting intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Update fields.yml

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* intel.log

example intel.log

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* added default_field: false

added default_field: false

Co-Authored-By: Arcuri Davide <dadokkio@gmail.com>

* Generate expected zeek/intel output event

* Add changelog entry

* Update field docs

* Misc improvements

Expand dots in zeek.intel.seen
Parse ts value without dropping millisecond value
Add event.ingested timestamp
Convert ingest node pipeline to YAML
Save JSON message in event.original

* Updates to zeek.intel.seen

Expand dots of all seen.* fields
Change name of zeek.intel.seen.fa_file to zeek.intel.seen.f as documented by Zeek.

* Update field docs

(cherry picked from commit 7ad14e6)
@@ -53,10 +53,29 @@ processors:
to: "destination.packets"

- from: "zeek.connection.conn_state"
to: "zeek.connection.state"

to: "zeek.connection.state.code"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is breaking change to the mapping. We need to call this out in the changelog and take action to not affect 7.x with this change. @alakahakai can you look into fixing the 7.x branch before the 7.6 branch gets created.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #15169

jorgemarey pushed a commit to jorgemarey/beats that referenced this pull request Jun 8, 2020
Add 30+ filesets to Filebeat Zeek module. Thanks to @Xander33 for the original PR elastic#13683.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants