idaholab#475, it looks like we can pretty easily support multiple log…

…gers for filebeat on Malcolm

docs/live-analysis.md

@@ -86,6 +86,8 @@ The relevant environment variables related to tuning Zeek for live packet captur
 - `ZEEK_AF_PACKET_FANOUT_MODE` - AF_Packet [fanout mode](https://docs.zeek.org/en/master/scripts/base/bif/plugins/Zeek_AF_Packet.af_packet.bif.zeek.html#type-AF_Packet::FanoutMode) (default `FANOUT_HASH`)
 - `ZEEK_LB_PROCS_WORKER_DEFAULT` - ["Zeek is not multithreaded, so once the limitations of a single processor core are reached the only option currently is to spread the workload across many cores"](https://docs.zeek.org/en/master/cluster-setup.html#cluster-architecture). This value defines the number of processors to be assigned to each group of [workers](https://docs.zeek.org/en/master/frameworks/cluster.html#worker) created for each capture interface for [load balancing](https://docs.zeek.org/en/master/cluster-setup.html#load-balancing) (default `1`). A value of `0` means "autocalculate based on the number of CPUs present in the system."
 - `ZEEK_LB_PROCS_WORKER_n` - Explicitly defines the number of processor to be assigned to the group of workers for the *n*-th capture interface. If unspecified this defaults to the number of CPUs `ZEEK_PIN_CPUS_WORKER_n` if defined, or `ZEEK_LB_PROCS_WORKER_DEFAULT` otherwise.
+- `ZEEK_LB_PROCS_LOGGER` - Defines the number of processors to be assigned to the [loggers](https://docs.zeek.org/en/master/frameworks/cluster.html#logger) (default `1`)
+- `ZEEK_LB_PROCS_PROXY` - Defines the number of processors to be assigned to the [proxies](https://docs.zeek.org/en/master/frameworks/cluster.html#proxy) (default `1`)
 - `ZEEK_LB_PROCS_CPUS_RESERVED` - If `ZEEK_LB_PROCS_WORKER_DEFAULT` is `0` ("autocalculate"), exclude this number of CPUs from the autocalculation (defaults to `1` (kernel) + `1` (manager) + `ZEEK_LB_PROCS_LOGGER` + `ZEEK_LB_PROCS_PROXY`)
 - `ZEEK_PIN_CPUS_WORKER_AUTO` - Automatically [pin worker CPUs](https://en.wikipedia.org/wiki/Processor_affinity) (default `false`)
 - `ZEEK_PIN_CPUS_WORKER_n` - Explicitly defines the processor IDs to be to be assigned to the group of workers for the *n*-th capture interface (e.g., `0` means "the first CPU"; `12,13,14,15` means "the last four CPUs" on a 16-core system)
@@ -94,11 +96,6 @@ The relevant environment variables related to tuning Zeek for live packet captur
 - `ZEEK_PIN_CPUS_LOGGER` - list of CPUs to pin for the logger processes (default is unset; only used if `ZEEK_PIN_CPUS_OTHER_AUTO` is `false`)
 - `ZEEK_PIN_CPUS_PROXY` - list of CPUs to pin for the proxy processes (default is unset; only used if `ZEEK_PIN_CPUS_OTHER_AUTO` is `false`)
 
-These variables will aslo be honored, but it is not recommended to set them to any value greater than `1` as the Malcolm processes that monitor Zeek logs do not yet handle logs generated by multiple loggers to disparate locations:
-
-- `ZEEK_LB_PROCS_LOGGER` - Defines the number of processors to be assigned to the [loggers](https://docs.zeek.org/en/master/frameworks/cluster.html#logger) (default `1`)
-- `ZEEK_LB_PROCS_PROXY` - Defines the number of processors to be assigned to the [proxies](https://docs.zeek.org/en/master/frameworks/cluster.html#proxy) (default `1`)
-
 ### <a name="LiveAnalysisTuningArkime"></a>Arkime
 
 Arkime's `capture` process is controlled by [settings](https://arkime.com/settings) in its `config.ini` file. Arkime's documentation on [High Performance Settings](https://arkime.com/settings#high-performance-settings) outlines the settings that most influence performance and resource utilization.

filebeat/filebeat-logs.yml

@@ -29,7 +29,7 @@ filebeat.inputs:
 
 - type: log
   paths:
-    - ${FILEBEAT_ZEEK_LOG_LIVE_PATH:/zeek/live}/logs/current/*.log
+    - ${FILEBEAT_ZEEK_LOG_LIVE_PATH:/zeek/live}/spool/logger-*/*.log
   symlinks: true
   fields_under_root: true
   tags: ["_filebeat_zeek_malcolm_live"]