Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add network traffic package #1176

Merged
merged 17 commits into from
Jul 12, 2021
Merged

Conversation

andrewstucki
Copy link

@andrewstucki andrewstucki commented Jun 22, 2021

What does this PR do?

This is a generic network traffic package that is a replacement for Packetbeat. It contains data streams and configuration for the following protocols:

  • amqp
  • cassandra
  • dhcpv4
  • dns
  • flow (not a protocol, just flow tracking stats)
  • http
  • icmp
  • memcached
  • mongodb
  • mysql
  • nfs
  • pgsql
  • redis
  • sip
  • thrift
  • tls

I still need to add test fixtures for it. As it uses the contains handlebars helper I extended in Kibana to support strings and conditional blocks, it requires Kibana 7.14.

Additionally, tests won't pass until the latest Kibana 7.14 snapshot comes out and until tomorrow's Packetbeat 7.14 snapshot comes out, as the tests and manifest rely on some bugfixes and the above functionality.

Testing is good to go.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Screenshots

Screen Shot 2021-06-22 at 11 24 33 AM

@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@elasticmachine
Copy link

elasticmachine commented Jun 22, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-07-06T22:28:33.068+0000

  • Duration: 61 min 19 sec

  • Commit: 1b1a3ce

Test stats 🧪

Test Results
Failed 0
Passed 103
Skipped 0
Total 103

Trends 🧪

Image of Build Times

Image of Tests

@andrewstucki andrewstucki changed the title Add network trafic package Add network traffic package Jun 22, 2021
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This is a good start. I know there are more configuration options that we will want to expose in the future like include_raw_certificates for TLS, etc.

vars:
- name: port
type: integer
# currently the Kibana UI doesn't support multi inputs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an issue for this that you can link to in the comment. Then it will be clear when we can start allowing numeric mutli-inputs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create one and link it

Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Do you think there is value in adding Tags as a variable?

type: text
title: Interface
required: false
show_user: false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be good to add a description that says what happens if you don't specify the interface (any interface or no interface?) Do we want to expose that you could give a filename here for pcap files?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be good to add a description that says what happens if you don't specify the interface (any interface or no interface?)

I'll try and come up with something for the description, felt the need to, but wasn't sure what to put since it's not super simple -- it actually follows the logic of Packetbeat's default template which is:

  • Mac OS X: en0
  • Linux: any
  • Windows: whatever the first interface is 😅

Do we want to expose that you could give a filename here for pcap files?

I think I'd probably want to avoid adding that, mainly because I don't see a good use-case for 99.99% of users to add a pcap file as the thing to read from.

@andrewstucki
Copy link
Author

I'm going to go ahead and just merge this so it doesn't sit out any longer, will add the descriptions and comments to an issue link in a follow-up PR. Still have additional dashboards to add and potentially some installation lifecycle management stuff to work out before this goes primetime anyway.

@andrewstucki andrewstucki merged commit a5d3c76 into elastic:master Jul 12, 2021
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
* Add network trafic package

* Add pcap tests

* run elastic-package format

* Fix up ports on http tests

* Fix up ports on mysql tests

* Fix field mappings for mongo, mysql, and nfs

* Fix up fields for pgsql, sip, thrift, and tls

* update README

* Update dns fields

* Fix up http fields

* Fix redis test configuration port

* Fix cassandra fields

* Fix missing fields

* Fix up test ids for dns

* Fix missing dns field

* Fix some more bad fields

* Revert keyword -> long changes and add testing config
@elasticmachine
Copy link

Package cloud_security_posture - 1.5.0 containing this change is available at https://epr.elastic.co/search?package=cloud_security_posture

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants