bring your own zeek

[alfred-landrum]

A user should be able to use their own Zeek executable & scripts to perform the pcap to zeek log conversion that occurs during pcap ingest.

This feature would enable:

• Zeek savvy users to use their own customized scripts.
• Zeek script authors to use Brim as a script development aid.
• Brim/zq developers to easily test modifications to the packaged Zeek bundle included in Brim.

To implement this, we should reorganize how Zeek is launched from Brim zqd. I suggest we use the idea of a "Zeek runner", a command or script that acts as the interface for Zeek to zqd. It's contract with zqd would be:

• zqd is passed a path to a runner script/executable.
• The runner is launched with no command arguments.
• The runner's working directory is set to the desired output location for the generated zeek logs.
• The runner is responsible for launching zeek with any needed command line arguments and environment variables.
• The runner is responsible for ensuring pcap data from stdin reaches zeek.

This is nearly what the 'zeek' script in brimsec/zeek does today, with the exception of the zeek cli arguments that are currently specified in the zqd source.

Brim would allow a user to specify a location to a runner, defaulting to the one we include in zdeps. We can also allow a user to override the runner with an environment variable.

[alfred-landrum]

I've done an initial round of work on the 'zeek runner'. Since it's across all 3 of our main dev repos, all of the tests are failing, but I've verified the PR's work with each other on mac & windows. I'll wait until #634 is in, then use that to make the Brim integration tests pass.

[philrz]

The attached video shows the complete feature in action as of Brim commit ff40f0e. I have a Zeek v3.1.3 compiled on my laptop in /usr/local/zeek-3.1.3. I copy the zeekrunner script from Brim to this base directory for my alternate Zeek, then import a pcap file in Brim. The ps output shows that the generated events in Brim are coming from my Zeek v3.1.3. Therefore I'm free to do whatever customizations/additions to my personal Zeek & see the results of that in the app.

Verify.mp4.zip