Skip to content

Add Zeek Broker reader plugin #1758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jul 8, 2021
Merged

Add Zeek Broker reader plugin #1758

merged 23 commits into from
Jul 8, 2021

Conversation

mavam
Copy link
Member

@mavam mavam commented Jul 5, 2021

📔 Description

This PR adds a new broker reader plugin, enabling VAST to be true native
Logger node. This approach takes a slightly different path than the usual way
of writing a Zeek plugin and implementing one's own WriterBackend. Here are
the pros and cons of this method.

Pros:

  • Easy rollout: works out-of-the-box with Zeek, just load a script and log
    remotely
  • Works seamless with the Cluster framework: VAST is a first-class logger node
  • Performance: no need to "log twice," once to the Logger node and then
    parsing/repacking the data again to ship it to VAST

Cons:

  • Does not use a standard API and reverse-engineered binary wire format
  • No support for Zeek logging filters, if used

Since filtering can also be implemented easily downstream in VAST, we find that
the pros dominate this method.

📝 Checklist

  • Replace poll with get(num, timeout)
  • All user-facing changes have changelog entries.
  • The changes are reflected on docs.tenzir.com/vast, if necessary.
  • The PR description contains instructions for the reviewer, if necessary.

🎯 Review Instructions

File-by-file.

@mavam mavam added the feature New functionality label Jul 5, 2021
@mavam mavam force-pushed the story/ch26067/broker-plugin branch from 2c4d96b to 2adcc47 Compare July 6, 2021 11:38
@mavam mavam force-pushed the story/ch26067/broker-plugin branch from 2adcc47 to 3493f19 Compare July 8, 2021 13:53
@mavam mavam requested a review from dominiklohmann July 8, 2021 13:55
@mavam mavam marked this pull request as ready for review July 8, 2021 13:56
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I just have some smaller things, plus this needs CI to run through. Let's get this in and then test this in our E2E setup outside of this repository.

@mavam mavam enabled auto-merge July 8, 2021 14:33
@mavam mavam merged commit ad144f1 into master Jul 8, 2021
@mavam mavam deleted the story/ch26067/broker-plugin branch July 8, 2021 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants