Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update zeekdeploy.sh to add warning to not directly edit node.cfg #158

Merged
merged 1 commit into from
Feb 2, 2021

Conversation

ObsidianKnife
Copy link
Contributor

馃棧 Description

Adds a warning to not directly edit node.cfg.

馃挱 Motivation and context

My Zeek instance was being reset to sub optimal settings via edits to node.cfg of unknown origin. This warning would have saved me a bit of time tracking down the script responsible for the changes.

馃И Testing

ran modified script and node.cfg included warning without other content.

Adds a warning to not directly edit node.cfg.
@mmguero
Copy link
Collaborator

mmguero commented Feb 1, 2021

Thanks for the update. What settings in node.cfg do you feel would ideally be user-definable so that I could preserve them or read them from another location and integrate them into node.cfg when it's created?

@ObsidianKnife
Copy link
Contributor Author

I have a few ideas:

  1. You could check for a special string in node.cfg to know you just need to leave it alone such as a line matching the regex"^## NOCLOBBER ##$" this very easy to do but cuts the user off from some of the advantages of the automation but after installation this may not be a big deal especially considering how updates are currently performed (i.e. reinstall)
  2. You could implement more variables in /opt/sensor/sensor_ctl/control_vars.conf to correspond to the following:
    a. logger, manager, proxy and worker - pin_cpus
    b. worker lb_procs

The need for these is because I was experiencing a packet drop rate as high as 30% with 20 hardware cores working. I reduced this to 0.007% by dedicating 8 cores to worker processes and two dedicated to IRQ processing one of which is also doing logger and proxy processes while the other handles IRQ and and manager process. I told the kernel to not use these ten cores for anything so only zeek runs on them and the other ten cores are running the yara rules, clam_av etc. I would love to see this level of optimization in hedgehog but it seems like a heavier lift as the program would have to identify what cores are on the NUMA node that handles the PCI bus the capture card is on to reduce latency as that is what seems to be the limiting factor here. I have more details on this if you are interested, I am still tinkering with some of the finer points.

@mmguero mmguero merged commit ad3e65a into cisagov:master Feb 2, 2021
@mmguero
Copy link
Collaborator

mmguero commented Feb 2, 2021

Thanks for the suggestions. I'm going to open an issue on the subject for the implementation of more variables to expand from control_vars.conf into node.cfg.

mmguero added a commit that referenced this pull request Feb 5, 2021
Malcolm v2.6.1 contains the following changes:

v2.6.0...v2.6.1

* Added [TFTP](https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol) [Zeek parser](https://github.com/zeek/spicy-tftp) and corresponding Logstash parsing, Arkime WISE support and Kibana dashboards
* Provide browser-based access to zeek/extracted-files directory (idaholab#34)
* Fix LDAP analyzer not parsing all events (idaholab#35)
* Provide more fine-tuned controls for Zeek's node.cfg in Hedgehog sensor (idaholab#36, /pull/158)
* set zeek.uid to conn_uids for files.log entries (idaholab#33)
* Modify Zeek build chain to use default GCC compilers instead of LLVM/clang,which reduces build dependencies
* Use Firefox instead of Chromium for browser in ISO-installed versions of Malcolm and in Hedgehog Linux
* Updated copyright notices in text from "2020" to "2021" (which is the bulk of the changed files in this commit)
* Version bumps
  * Yara to 4.0.4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants