Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingest Zeek Logs into Brim #574

Closed
jameskerr opened this issue Apr 8, 2020 · 1 comment · Fixed by #594
Closed

Ingest Zeek Logs into Brim #574

jameskerr opened this issue Apr 8, 2020 · 1 comment · Fixed by #594
Assignees

Comments

@jameskerr
Copy link
Member

jameskerr commented Apr 8, 2020

When given a array of files and their associated types, we need to validate that the array contains:

  • One pcap
  • One zeek log
  • Multiple zeek logs

Any other combination will throw an error explaining what we expect.

If it passes validation, return the default dataDir and spaceName params.

If one file:
spaceName: same name as that file
dataDir: same directory as that file/spacename.brim

If more than one file:
spaceName:
if all logs in same dir, the name of that dir
else a randomly generated name
dataDir: $HOME/.brim/spaceName.brim

@jameskerr jameskerr changed the title Default DataDirs and Space Name Post to default dataDir Apr 8, 2020
@jameskerr jameskerr self-assigned this Apr 8, 2020
@jameskerr jameskerr changed the title Post to default dataDir Ingest Zeek Logs into Brim Apr 10, 2020
@henridf henridf mentioned this issue Apr 15, 2020
@philrz
Copy link
Contributor

philrz commented Apr 21, 2020

Verified in Brim commit 314f149 talking to zqd tagged v0.11.1. Per the attached video, now I can open a directory full of Zeek TSV logs in Brim, as opposed to having to start with just a pcap.

Verify.mp4.zip

This was done using the zq-sample-data. The count() shown in the video matches that seen with zq at the CLI.

~/work/zq-sample-data/zeek-default$ zq -f table "count()" *
COUNT
1462078

Some additional notes:

  1. I had to uncompress the log files before Brim would accept them. Whether Brim can accept GZIP'ed input files the way zq can is TBD.
  2. As described in the PR, the UX work to expose Zeek-centric icons & wording is still on the way.
  3. Also as described in the PR, ingest of typed Zeek NDJSON is not working yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants