Zeek Intel Framework #20
Comments
|
If you're talking about the various Zeek .log files themselves, currently just the delimited format is supported as that's how the Logstash filters parsing them are expecting them. Glad you figured it out. For now I'll leave this issue open and maybe see about making mapping zeek intel files into the containers more automatic as an enhancement. |
|
I did look in the reporter log and see it appears to have an issue with the formatting of the file. I’ll dig into that and update. This is what I get as an error in the reporter log: I did change the file from BroStuff.txt to master.... that is referenced above ^ |
|
@mmguero figured it out. It was just a docker-ism. Just needed an additional mapping for the intel file as a volume into the container. For reference I am using intel generated by intelstack's client that natively gets it into TSV format. Do you know off hand if Malcolm would support a tune to JSON or would that disturb other modifications? Thanks brother! |
|
I haven't really done anything with the intel framework in Malcolm yet, but I can give you a little bit of advice on how local.zeek gets loaded in Malcolm. The If you see a line in the Then you should be getting your local modifications to If you still aren't seeing things, check under Perhaps it is getting created, but I've got an issue parsing the file? |
|
So here is my local.zeek that is a volume on the host for the containers related file. I added the Intel::read_files as well as the 2 policies related to Intel as well below. #! Zeek local site policy. Customize as appropriate. redef Broker::default_listen_address = "127.0.0.1"; @load tuning/defaults --custom packages installed manually Here is the BroStuff.txt file generated from cifv5 that supposedly puts it into the correct bro tab separated format. #fields indicator indicator_type meta.desc meta.confidence meta.source meta.do_notice That IP (23.15.54.44) is in my instance from a malicious pcap I have. However as a test its not actually on that list as malicious in the wild. I checked in the logs and it isn't creating the intel log file. Either. I feel its a parsing issue as well but not sure yet. |
…lly under ./zeek/intel/ and it will be bind-mounted in without having to manually modify local.zeek) and fix some zeek log fields
…oose" intel files
|
with the v5.2.0 release Malcolm's made it easier to drop intel files in |
v5.2.0 release development
* New features
* Zeek Intelligence Framework (see idaholab#20)
* To quote Zeek's Intelligence Framework documentation, "The goals of Zeek’s Intelligence Framework are to consume intelligence data, make it available for matching, and provide infrastructure to improve performance and memory utilization. Data in the Intelligence Framework is an atomic piece of intelligence such as an IP address or an e-mail address. This atomic data will be packed with metadata such as a freeform source field, a freeform descriptive field, and a URL which might lead to more information about the specific item." Zeek intelligence indicator types include IP addresses, URLs, file names, hashes, email addresses, and more.
* Malcolm doesn't come bundled with intelligence files from any particular feed, but they can be easily included into your local instance. On startup, Malcolm's `malcolmnetsec/zeek` docker container enumerates the subdirectories under `./zeek/intel` (which is bind mounted into the container's runtime) and configures Zeek so that those intelligence files will be automatically included in its local policy. Subdirectories under `./zeek/intel` which contain their own `__load__.zeek` file will be `@load`-ed as-is, while subdirectories containing "loose" intelligence files will be loaded automatically with a `redef Intel::read_files` directive.
* New [**OPCUA Binary** protocol parser](https://github.com/cisagov/icsnpp-opcua-binary) for Zeek and corresponding dashboard.
* Improvements
* set `ecs.provider` to `arkime` for logs from Arkime's `capture` to make categorizing logs by source easier
* API
* allow bucketing multiple fields from `/agg/` API
* added `/fields/` API to list fields
added documentation
* ECS normalization to [`related.hosts`](https://www.elastic.co/guide/en/ecs/current/ecs-related.html#field-related-hosts) field for all applicable protocols
* updated documentation, screenshots and slides
* spreadsheet mapping STIX v1.2 fields to Zeek fields and Malcolm normalized fields
* updated MITRE ATT&CK mappings for Capa hits
* added a pseudo-read-only NGINX configuration
* Version bumps
* Arkime to [v3.3.0](https://github.com/arkime/arkime/blob/496ec1e5cd89d79e22ab1a0cddb9a7a2f301cd14/CHANGELOG#L25-L50)
* OpenSearch to [v1.2.4](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.2.4.md)
* Capa to [v3.1.0](https://github.com/mandiant/capa/releases/tag/v3.1.0)
* [cve-2021-44228 Log4Shell detector plugin](https://github.com/corelight/cve-2021-44228) for Zeek to v0.5.3 (see corelight/cve-2021-44228#46)
* Bug Fixes
* fix idaholab#71 (type mismatch for network.vlan.id between Malcolm and Arkime definitions) which prevented vlan traffic from indexing correctly from Arkime's `capture` with Malcolm's field template
* fix for ethernet/IP traffic which could lead to Zeek runaway memory allocation until crash: "Fixed bug with Request Paths containing Port Segments" (cisagov/icsnpp-enip@4696a43)
|
done in v5.2.1 |
From Malcolm created by cyamal1b4: cisagov#131
Greetings, so I've been up and down in the docs for Zeek and understand decently well how to create Intel in Zeek, however, that is largely aligned with a normal deployment of Zeek. I have loaded the required policies and pointed zeek (in local.zeek) to my local bro formatted file with an indicator. I tested everything in the TryZeek page with the same pcap, local.zeek changes, as well as the same bro intel file. However, when I replicate I cannot get Malcolm (under the Kibana-Zeek Intel or Notice dashboards) to pick up on my Intel hit. I have noticed some differences in the deployment with Malcolm so I figured it would be best to ask the developer directly. Thanks for your support and contribution!!
The text was updated successfully, but these errors were encountered: