Skip to content

Releases: idaholab/Malcolm

Malcolm v26.08.0

Choose a tag to compare

@mmguero mmguero released this 25 Aug 22:26
f680217

Malcolm v26.08.0 adds a NetBox purdue_zone custom field that propagates ICS/OT network zone classifications to devices, prefixes, and virtual machines (and automatically to autopopulated devices from their containing prefix); Raspberry Pi 5 support for Hedgehog Linux; and, configurable Strelka scanner and disabled-Suricata-SID lists. This release also fixes five security vulnerabilities: an nginx RBAC bypass via percent-encoded, case-varied, or slash-doubled request paths; an archive-bomb bypass affecting raw-stream and lzip-compressed uploads; a case-variant path bypass of the nginx auth gate exposing the Arkime backend to forged identity headers; an Arkime authentication gap on sensor nodes that fell back to digest instead of enforcing s2s; and a CSRF vulnerability in the kiosk /script_call endpoint allowing unauthenticated data-destructive operations. Arkime, Zeek, NetBox, OpenSearch, OpenSearch Dashboards, Logstash, Filebeat, Keycloak, and other components have been updated as well. Several other bug fixes and general improvements are also included.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.07.1...v26.08.0

  • Features and enhancements

    • Add a purdue_zone NetBox custom field for devices, virtual machines, and IP prefixes, letting users assign a security zone (e.g., IT, OT, DMZ) to networked assets; the value is enriched into source/destination/related fields, surfaced in Arkime, and automatically propagated from a containing prefix to autopopulated devices and VMs (most specific prefix wins). This new field is used in a new Dashboard (named "Biomes") visualzing connections between zones. #1088
    • Add Raspberry Pi 5 support to the Hedgehog Linux build #376
    • Allow individual Strelka scanners to be selected in the configuration tool, setting the STRELKA_SCANNERS environment variable, which now accepts the keyword default (for Malcolm's built-in scanner list) alongside or combined with explicit scanner names (e.g., default,ScanStrings)
    • Add a to the configuration tool an option to populate Suricata's disable.conf with a comma-separated list of rule SIDs to disable, letting users turn off noisy or irrelevant detection rules without editing rule files directly
    • Add the ZEEK_ZAM environment variable to enable Zeek's ZAM script optimizer (-O ZAM) for both live capture and offline PCAP processing
    • Add experimental support for enabling OpenSearch Dashboards' multiple data sources feature via DASHBOARDS_DATA_SOURCES_ENABLED, letting one Dashboards instance query and visualize data from several separate Malcolm OpenSearch backends at once
    • Add NGINX_CSP_FORM_ACTION_EXTRA to allow additional cross-origin form submission targets in nginx's Content-Security-Policy, needed for authentication flows such as SAML HTTP-POST binding #1052
    • Restyle the artifact upload page to match Malcolm's landing-page visuals #1069, #530
  • 🛡️ Security Remediation & Hardening

    • Fix an RBAC bypass in Malcolm's nginx Lua authorization layer where percent-encoded, case-varied, or slash-doubled request paths could reach role-restricted locations (e.g., /htadmin, /auth, /admin_login, Arkime esadmin) that a user's role should not permit GHSA-jr6p-63pg-hr6g #1055
    • Fix an archive-bomb bypass in upload processing where raw-stream (gzip/bzip2/xz/lzma) and lzip-compressed files were decompressed without the entry-count, depth, and total-size limits already enforced for container archives, allowing disk-space exhaustion from a small, highly compressible upload GHSA-f2v6-8cj4-mhr6 #1054
    • Fix an unauthenticated bypass of the nginx auth gate where case-variant paths (e.g., /IDDASH2ARK/, /DASHBOARDS/APP/EXTRACTED-FILES/) entered case-insensitive location blocks but evaded the case-sensitive rewrite, falling through to an unguarded proxy_pass and allowing a caller to reach the Arkime backend unauthenticated while supplying a forged X-Forwarded-User identity header GHSA-7j32-cf27-cp6h #1081
    • Explicitly set Arkime's authMode rather than relying on undocumented autochoose behavior, and default sensor/live-capture nodes to server-to-server (s2s) authentication so port 8005 reachback is gated by the shared secret and timestamp it was designed around instead of silently falling back to digest GHSA-86h3-7rf8-8j34 #1086
    • Fix a CSRF vulnerability in the kiosk /script_call endpoint where wildcard CORS and missing origin validation allowed any page loaded in a browser tab on the sensor to trigger unauthenticated kiosk operations, including a full data wipe, by enforcing same-origin requests via the Sec-Fetch-Site header and replacing free-text command execution with a fixed operation allowlist GHSA-w8gq-4v5x-xrrm #1086
    • Harden nginx configuration template rendering by normalizing boolean environment variables, rejecting values containing unsafe configuration characters, limiting envsubst to an explicit variable allowlist, and rendering templates atomically
  • 🐛 Bug fixes

    • Fix NetBox API tokens (including SUPERUSER_API_TOKEN) breaking after netboxRestore until a full-stack restart, caused by a missing environment variable mapping, a superuser-recreation failure that was silently swallowed, and the NetBox service starting before superuser/token recreation completed #1064
    • Fix Zeek log-source detection for time-range rollover filenames (e.g., analyzer.03:00:00_03:49:33.log), including a follow-up fix for filenames carrying both a rotation timestamp and upload/source tags in either order #1065, #490
    • Fix filescan results not being tagged with the same uploaded-PCAP tags as their originating Zeek events, by preserving the tagged Zeek files.log symlink path through the filescan Filebeat pipeline #1063, #893
    • Fix the filescan_watcher process silently omitting a configured secondary Valkey watch for the lifetime of the process if that Valkey endpoint was not yet available at startup; the process now exits so it is restarted by supervisor and can recover once Valkey becomes available #1061, #1058
    • Fix the ISO installer being able to select its own boot USB media as an installation target when no other suitable disk was found; installation now fails safely instead #1062, #905
    • Fix OpenSearch Dashboards failing to deserialize Strelka documents containing Capa match-tree numeric values outside JavaScript's safe-integer range, by converting such values to strings before indexing #1066, #951
    • Fix several tree-layout and trends Vega dashboards (IP connections tree, RDP, SSH, File Tree, and the Modbus/BACnet/DNP3 trends visualizations) failing to load under OpenSearch 3.8.0's stricter Vega expression sandbox, which blocks computed-index bracket access into arrays and objects
    • Fix Arkime's GeoIP database update not honoring uppercase HTTP_PROXY, HTTPS_PROXY, and NO_PROXY container proxy variables; these are now mirrored to their lowercase equivalents when the lowercase forms are not already set, and download timeouts were added to prevent hangs behind an unreachable proxy #1068, #482
    • Fix ./scripts/control.py start failing with a SyntaxError on Python 3.11 and earlier due to a multiline expression inside a single-quoted f-string that only Python 3.12+'s updated f-string grammar could parse #1056
    • Fix the Malcolm ISO installer's rc.local-based initialization (capture-storage disk setup, first-boot AIDE database initialization, Ctrl+Alt+Del masking) being replaced with a proper systemd service (malcolm-system-init.service)...
Read more

Malcolm v26.07.1

Choose a tag to compare

@mmguero mmguero released this 22 Jul 14:35
3c0f5f2

Malcolm v26.07.1 adds a few minor changes on top of Malcolm v26.07.0, the most notable being a fix for a crash in the strelka-backend container on arm64 platforms. Malcolm v26.07.0 added IEC 60870-5-104 (IEC 104) protocol support using CERT.LV's Zeek plugin, including Logstash parsing, ECS normalization, Arkime fields, and a new OpenSearch Dashboards dashboard. This release also fixes three archive extraction and authentication security vulnerabilities; improves NetBox enrichment configuration; and addresses PostgreSQL major version upgrade, custom CA certificate for KeyCloak, container health check, privilege-drop signal chaining, and configuration script issues. Arkime, Zeek, Fluent Bit, Filebeat, Logstash, Supercronic, and Alpine-based images have been updated as well.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.06.1...v26.07.1

  • Features and enhancements

    • Add IEC 60870-5-104 (IEC 104) support using the CERT.LV spicy-iec104 Zeek plugin, including Zeek log ingestion, ECS field mapping, Arkime fields, and an IEC 104 dashboard #939
    • Make LOGSTASH_NETBOX_ENRICHMENT_DATASETS more flexible: it now accepts default, ics/ot, all, explicit provider.dataset values, and combinations such as default,ics #1037
    • Allow LOGSTASH_NETBOX_ENRICHMENT_DATASETS to be configured through checkboxes in the configuration TUI #1033
    • Improve ./scripts/start error messages by listing missing or invalid authentication-related files instead of reporting only a generic authentication setup failure #865
    • Have system-quickstart detect and prepopulate existing time synchronization settings when rerun #992
  • 🛡️ Security Remediation & Hardening

    • Fix an RBAC bypass caused by URI normalization differences between Nginx location matching and the Lua authorization layer CVE-2026-63177 #1042
    • Fix path traversal in archive extraction directory handling by validating resolved paths and using libarchive's secure extraction flags CVE-2026-63134 #1040
    • Limit archive entry count, nesting depth, and total expanded size to prevent inode- and resource-exhaustion denial of service during extraction CVE-2026-63133 #1041
    • Mark OpenID Connect session cookies as secure and improve handling of externally forwarded HTTPS schemes
  • 🐛 Bug fixes

    • Co-installation of opencv-python and opencv-contrib-python corrupts cv2.abi3.so, segfaulting strelka-backend at import on arm64 #1046 (fix)
    • Allow the configuration TUI to reset supported variables back to empty values after installation #1024, #1030
    • Fix the broken signal chain in docker-uid-gid-setup.sh so signals reach the final process after dropping privileges #1039 to ensure clean shutdown of containers
    • Fix PostgreSQL being reported unhealthy after a major-version upgrade, improve upgrade-state handling, and perform required post-upgrade extension and collation maintenance #1038
    • Fix the Nginx Lua/OpenID Connect helper not honoring user-provided CA certificates for KeyCloak when KEYCLOAK_SSL_VERIFY=true #1035
    • Restore curl to the the htadmin container for use by the health check script #1029
    • Reduce the size of the OpenSearch Dashboards image by copying only the permissions data needed from its upstream image layer #1031
    • Fix JSON handling of several Zeek fields whose names contain dots by normalizing them to underscore-separated field names
    • Fix additional Zeek and Suricata field normalization and ECS mapping inconsistencies found while updating dashboards and index templates
  • Component version updates

  • 🧹 Code and project maintenance

    • Broad spelling, grammar, naming consistency, and documentation cleanup across scripts, configuration, dashboards, and documentation #990
    • Expand and restructure documentation to provide better project context for developers and LLM-assisted code analysis #964
    • Improve installer validation, environment-variable mapping tests, and configuration item metadata
    • Refresh dashboards, index templates, field mappings, protocol documentation, and navigation links
    • Minor improvements to the Hedgehog Raspberry Pi image build process.
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

    • LOGSTASH_NETBOX_ENRICHMENT_DATASETS in logstash.env now defaults to default and may contain default, ics/ot, all, explicit provider.dataset values, or a comma-separated combination of these values
    • ZEEK_DISABLE_ICS_IEC104 in zeek.env controls whether the IEC 104 Zeek plugin is disabled
    • SAFE_EXTRACT_MAX_ENTRIES, SAFE_EXTRACT_MAX_DEPTH, and SAFE_EXTRACT_MAX_BYTES in upload-common.env set archive extraction resource limits for uploaded archive files (e.g., containing Zeek logs for processing); their defaults are 5,000 entries, 20 directory levels, and 4 GiB of expanded data

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in Gi...

Read more

Malcolm v26.06.1 (SEE NOTE BEFORE UPGRADING TO THIS RELEASE)

Choose a tag to compare

@mmguero mmguero released this 16 Jun 20:44
645e4c5

Malcolm v26.06.1 addresses a high severity RCE vulnerability (GHSA-8cvp-m7pg-qrp7) allowing unrestricted PHP file upload and ships security-fixing updates across Arkime, OpenResty, Valkey, and PostgreSQL. Many other component versions have also been bumped. Six Zeek log parsing bugs are fixed — all related to JSON mode (ZEEK_JSON=true) and affecting DHCP, Redis, ROC+, and WebSocket log types — along with a suricata disable.conf append-on-restart regression, updated MaxMind GeoLite MMDB download logic, and an OpenSearch indexing error from oversized file.strings values.

NOTE: An issue has been reported relating to postgreSQL databases (ie., NetBox, KeyCloak) not being compatible from previous releases of Malcolm to this one. While we investigate, we recommend that you don't update to this release, unless you backup your NetBox database prior to upgrade, then restore it AND you don't use keycloak as your auth method.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.06.0...v26.06.1

  • 🛡️ Security Remediation & Hardening
    • Fix RCE via unrestricted .php upload to the file-upload component (GHSA-8cvp-m7pg-qrp7, thanks Jan Kahmen, turingpoint (@kah-ja, jan@turingpoint.de)
    • Removed php-curl from htadmin container (as it's unused) to reduce attack surface
    • Updates to Arkime, OpenResty, Valkey, and PostgreSQL all contain significant security fixes (see Component version updates below)
  • 🐛 Bug fixes
    • ./scripts/start not returning to command line after containers have started #1025
    • Zeek DHCP logs are parsed incorrectly with ZEEK_JSON=true #1018
    • Zeek Redis logs' reply field not parsed correctly when ZEEK_JSON=true #1019
    • suricata disable.conf gets appended to every time container restarts #1022
    • Zeek roc_plus logs can be parsed incorrectly in Logstash when ZEEK_JSON=true #1021
    • Zeek websocket logs can be parsed incorrectly in Logstash when ZEEK_JSON=true #1020
    • fix downloading MaxMind MMDB files (URL and authentication method changed slightly); see also Secrets and variables in the documentation
    • cap file.strings length at 16k to avoid errors inserting into OpenSearch with very, very long values
  • Component version updates
  • 🧹 Code and project maintenance
    • A few documentation updates
    • added tag_on_exception to all ruby Logstash filters to make debugging easier
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
  • Errata
    • curl removed from htadmin container causes container health check to fail and report unhealthy (cisagov#1029)

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.06.0

Choose a tag to compare

@mmguero mmguero released this 01 Jun 23:04
e60beb4

Malcolm v26.06.0 is primarily a security hardening release, addressing fifteen vulnerabilities (2 high severity, 6 medium, and 7 low) identified in a security assessment. Bug fixes address an issue with the zeek container causing performance degredation over time and a fix for duplicate virtual machine entries in NetBox autopopulation. A few component versions have also been updated.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.05.2...v26.06.0

  • 🛡️ Security Remediation & Hardening (#996)
    • Unauthenticated reflected XSS / open redirect in /dashboards/app/refred; also added Content-Security-Policy framing headers (frame-ancestors, base-uri, form-action) and X-Frame-Options: SAMEORIGIN globally to mitigate clickjacking (#997)
    • Authenticated command injection in filebeat container via SFTP-uploaded filename (#998)
    • Password stored as MD5-crypt for SFTP (#1009)
    • Authenticated archive zip-slip file write in filebeat container (#999)
    • OpenSearch path injection via /mapi/fields?template (#1000)
    • submit.php Location: open redirect via Referer (#1007)
    • htadmin proxied with no nginx auth gate (#1003)
    • Keycloak OIDC ssl_verify always set to false (#1006)
    • NetBox SUPERUSER_PASSWORD=admin shipped default (#1011)
    • RBAC defaultdict(lambda: True) fail-open for unlisted handlers in Malcolm API (#1004)
    • Read-only Arkime deny-regex omits addtags/removetags (#1008)
    • Read-only deployment allows POST /mapi/event (#1002)
    • WISE auth path selectable by client User-Agent (#1001)
    • ARKIME_PASSWORD_SECRET=Malcolm shipped default (#1005)
    • requests CVE bump reverted in logstash image (#1010)
    • Fix API auth errors and hide NGINX version disclosure (#989)
  • 🐛 Bug fixes
    • auto-discovered Virtual Machines in NetBox seem to allow for duplicates (#978)
    • Ensure list of archive file types supported by Malcolm for uploading Zeek logs (application/gzip,application/vnd.rar,application/x-7z-compressed,application/x-bzip2,application/x-cpio,application/x-gzip,application/x-lzip,application/x-lzma,application/x-rar-compressed,application/x-tar,application/x-xz,application/zip) are consistently used across the platform.
    • zeek container continually grows /usr/local/zeek/crontab, causing Malcolm performance to gradually worsen (#1015)
  • ✅ Component version updates
  • 🧹 Code and project maintenance
    • Fixed some incorrect links in documentation (#988, thanks @jsoref)
    • Refactored NGINX error pages configuration into its own include file and added a 401.html page
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.05.2

Choose a tag to compare

@mmguero mmguero released this 20 May 21:55
e56fd62

Malcolm v26.05.2 focuses heavily on security updates, most notably upgrading OpenResty to address a critical NGINX remote code execution heap buffer overflow vulnerability. It also adds new Suricata OT detections for D-Link HNAP abuse, improves alerting webhook support, introduces the File Tree dashboard, and includes Suricata parsing/mapping fixes and documentation updates. Several other components received version bumps as well.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

Note: v26.05.1 was pulled shortly after release due to a regression in Malcolm's NetBox initialization process (#984). v26.05.2 is based on v26.05.1 with Arkime updated to v6.4.0 and fixes for #984 and #979.

v26.05.0...v26.05.2

  • ✨ Features and enhancements
  • ✅ Component version updates
  • 🐛 Bug fixes
    • Reference Counting (Use-After-Free) Bug for PyList_SetItem in filescan's python-statfs (#960 #962)
    • Added a few missing Suricata fields (suricata.tc_progress, suricata.ts_progress, suricata.tunnel.pcap_cnt, suricata.tunnel.pkt_src) to the index mapping template
    • When suricata.app_proto_ts and/or suricata.app_proto_tc reported that protocol parsing had failed (due to malformed input data), invalid data could be stored in HTTP, DNS, and/or TLS fields. This is now detected and those invalid values are dropped, and some combination of proto_parse_failed, client_stream_failed, or server_stream_failed are added to tags.
    • Suricata's HTTP version was not being normalized to network.protocol_version.
    • ./scripts/start on Hedgehog profile shouldn't try to generate OpenSearch keystore (#979)
    • Unaccounted-for changes in NetBox v4.5.x data model cause problems in permission creation (#984)
  • 🧹 Code and project maintenance

Malcolm v26.05.0

Choose a tag to compare

@mmguero mmguero released this 05 May 16:12
82e5fe1

Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.04.1...v26.05.0

  • ✨ Features and enhancements
    • #726 — use hierarchical structure for NetBox device roles
      • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks @Crubumble)
    • #867 — examine large chown'ed directories in container images and see if they can be reduced
    • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks @PrudhviChanda)
      • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
      • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
    • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
    • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
    • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
    • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
  • ✅ Component version updates
  • 🐛 Bug fixes
    • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks @scott-jeffery)
      • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
      • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
    • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
      • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
      • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
      • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
    • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
      • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
      • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
      • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
    • #957 — configuration script can disable ICS parsers unintentionally
    • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
    • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
  • 🧹 Code and project maintenance
    • Documentation improvements
    • #913 — replace ingress-nginx which is EOL
      • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
      • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
      • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
      • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
    • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
    • #917 — develop IronBank (US DoD) images for Malcolm
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
    • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
    • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
  • ❌ Errata
    • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.04.1

Choose a tag to compare

@mmguero mmguero released this 13 Apr 18:50
0a7cf96

Malcolm v26.04.1 contains improvements, bug fixes, security updates, and component bumps.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.02.0...v26.04.1

Note that v26.04.1 is the same as v26.04.0 released last week, apart from the fix for bug #943. If you're already running v26.04.0 and don't use the encrypted install option in the installer ISO, you probably don't need to worry about updating to v26.04.1. The full release notes from v26.04.0 are also included here.

  • ✨ Features and enhancements
    • implemented easier way to enable/disable Strelka scanners #935
    • Handle nested file scanning (e.g., from ZIP files) with Strelka #922
    • index selected Strelka result fields #919
  • ✅ Component version updates
  • 🐛 Bug fixes
    • Hedgehog Linux Breaking on Reboot after Encrypted Quick Install with Multiple Drives #943
    • Fix YAML syntax error in kubernetes/15-redis.yml due to missing end quote #926
    • Using remote elasticsearch data store uses deprecated ssl_certificate_verification setting [https://github.com/cisagov/issues/915]
    • fix Malcolm API loopback webhook to handle RBAC and non-JSON formatted events #916
    • fix issues in zeekdeploy.sh to handle long crypto handshakes and Zeek's state DB getting out of sync
  • 🧹 Code and project maintenance
    • swap redis out for valkey #882
    • pin all third-party GitHub CI actions at known good SHA sums to mitigate things like the Trivy supply chain attack #933
    • some minor tweaks to various Dockerfiles and ISO build scripts to address vulnerability scanner findings
    • some documentation updates
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
    • Added ARKIME_PCAP_LIBPCAP to arkime.env should uses wish to revert to older libpcap mode for PCAP file processing rather than faster scheme processing (default false)
    • FILEBEAT_SCANNER_FINGERPRINT_LENGTH's default in filescan.env has been changed from 1024 to 512
    • redis.env has been renamed to valkey.env and its variables also have been renamed accordingly
    • STRELKA_SCANNERS has been added to pipeline.env for #935
    • ZEEK_DISABLE_SPICY_ZIP has been added to zeek.env for #922 (default true)

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.04.0

Choose a tag to compare

@mmguero mmguero released this 07 Apr 07:18
6b0c7b7

Malcolm v26.04.0 contains improvements, bug fixes, security updates, and component bumps.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.02.0...v26.04.0

  • ✨ Features and enhancements
    • implemented easier way to enable/disable Strelka scanners #935
    • Handle nested file scanning (e.g., from ZIP files) with Strelka #922
    • index selected Strelka result fields #919
  • ✅ Component version updates
  • 🐛 Bug fixes
    • Fix YAML syntax error in kubernetes/15-redis.yml due to missing end quote #926
    • Using remote elasticsearch data store uses deprecated ssl_certificate_verification setting [https://github.com/cisagov/issues/915]
    • fix Malcolm API loopback webhook to handle RBAC and non-JSON formatted events #916
    • fix issues in zeekdeploy.sh to handle long crypto handshakes and Zeek's state DB getting out of sync
  • 🧹 Code and project maintenance
    • swap redis out for valkey #882
    • pin all third-party GitHub CI actions at known good SHA sums to mitigate things like the Trivy supply chain attack #933
    • some minor tweaks to various Dockerfiles and ISO build scripts to address vulnerability scanner findings
    • some documentation updates
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
    • Added ARKIME_PCAP_LIBPCAP to arkime.env should uses wish to revert to older libpcap mode for PCAP file processing rather than faster scheme processing (default false)
    • FILEBEAT_SCANNER_FINGERPRINT_LENGTH's default in filescan.env has been changed from 1024 to 512
    • redis.env has been renamed to valkey.env and its variables also have been renamed accordingly
    • STRELKA_SCANNERS has been added to pipeline.env for #935
    • ZEEK_DISABLE_SPICY_ZIP has been added to zeek.env for #922 (default true)

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.02.0

Choose a tag to compare

@mmguero mmguero released this 19 Feb 22:48
a5c6c91

Malcolm v26.02.0 fixes a few bugs, updates a few components, and provides some improvements to documentation.

v26.01.0...v26.02.0

  • ✨ Features and enhancements
    • add SURICATA_DISABLE_SIDS to disable noisy suricata rules (#896)
    • allow "offline" Zeek (processing uploaded PCAP) to be able to skip threat intel (#876)
    • add indices:admin/create to the capture_service role for OpenSearch (#885)
  • ✅ Component version updates
  • 🐛 Bug fixes
    • choosing "no authentication" Malcolm still won't start due to missing htpasswd (#869)
    • filescan container processes need to handle connection timing issues more resiliently (#888)
    • filescan logs not building zeek.files.extracted_uri correctly for files hosted on Hedgehog Linux (#877)
    • IP Connections Tree left panel ("Trees Mirror") is wrong visualization (#899)
    • Remove /var/lib/suricata/cache contents when building Suricata container image (to reduce size and prevent flagging by AV scanners)
    • Fixed filescan container returning unhealthy just because the extracted file download service isn't enabledn
  • 🧹 Code and project maintenance
    • document ports used in Malcolm <-> Hedgehog communicationenhancementNew feature or request (#887)
    • document "capture only without forwarding" mode for Hedgehog (#889)
    • other minor documentation improvements
    • store the originating host name in host.name in file scanning results rather than the host name of where the scan was performed (only really makes a difference for Kubernetes deployments)
    • cryptography (Python library) to v46.0.5 (addresses CVE-2026-26007)
    • Pillow (Python library) to v12.1.1 (addresses CVE-2021-25289)
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
    • Added REDIS_MAXMEMORY, REDIS_MAXMEMORY_POLICY, REDIS_AUTO_AOF_REWRITE_MIN_SIZE, REDIS_CACHE_MAXMEMORY, and REDIS_CACHE_MAXMEMORY_POLICY to redis.env for tuning the redis and redis-cache containers.
    • Added SURICATA_DISABLE_SIDS to suricata.env for #896
    • Added ZEEK_DISABLE_INTEL_LIVE to zeek-live.env and ZEEK_DISABLE_INTEL_OFFLINE to zeek-offline.env for [cisagov#876]

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

Malcolm v26.01.0

Choose a tag to compare

@mmguero mmguero released this 30 Jan 19:42
8e7d7a4

Malcolm v26.01.0 introduces a complete overhaul of its automatic file scanning capability, replacing its file scanning framework with Strelka, an open-source "real-time, container-based file scanning system used for threat hunting, threat detection, and incident response." This new framework offers more features and greater extensibility for developing future analytics. The release also includes several bug fixes and component version updates.

v25.12.1...v26.01.0

  • ✨ Features and enhancements

    • Malcolm's automatic file scanning capability now runs on Strelka, an open-source "real-time, container-based file scanning system used for threat hunting, threat detection, and incident response," providing more features and improved extensibility for future analytics. Some settings, especially which scanners are enabled or disabled, may require manual configuration. See the documentation for details. (#485)
    • Enabled fuzzy hashing (SSDeep, TLSH), in addition to SHA256, for all Zeek-scanned files (#859)
    • Added additional health check data (Redis, etc.) to the /mapi/ready API
    • Major performance improvements to the Logstash parse pipeline:
      • Replaced the cidr filter with a custom Ruby filter
      • Changed "broadcast-and-drop" communication between pipelines to more targeted messaging
      • Reworked the "compact event" filter to remove empty/null values before indexing
    • Enabled MAC addresses in Suricata events (#866)
    • Enabled parsing of JA4D log generated by Zeek alongside dhcp.log (#870)
    • Strip out test PCAP from Zeek image (#872) to reduce image size and prevent image scanners from triggering false positives for the PCAPs' contents
    • Improvements to the IP Connections Tree dashboard
  • ✅ Component version updates

  • 🐛 Bug fixes

    • Malcolm API Loopback Monitor was being created multiple times (#856)
    • Disabled hardware NIC timestamping for capture (#851)
    • Fixed error in stun_nat log parsing with multiple WAN addresses (#849)
    • Fixed pruning of old log files (#855)
    • zeek_intel_setup.sh could leave an orphaned lock file if the container is killed, blocking future intel pulls (#843)
    • Fixed various parsing/templating issues for uploaded Windows Event (.evtx) files
    • Added text/php to "interesting" MIME types, and added application/html, application/ocsp-response, "application/x-pem-file, application/xhtml+xml, application/xml-sitemap, text/css, text/html, text/ini to "common/plain text" MIME types for extraction/scanning decisions
    • pcap-monitor container not honoring maximum PCAP file size (#864)
  • ⚰️ Breaking changes and removed or deprecated functionality

    • The VirusTotal API key (VTOT_API2_KEY) environment variable for submitting extracted file SHA sums is not longer used. This functionality can now be achieved via the Google Threat Intelligence service (VirusTotal is now part of Google Threat Intelligence) or any supported intelligence feed.
    • Adding new a new log source parsing pipeline to Logstash no longer uses LOGSTASH_PARSE_PIPELINE_ADDRESSES. The new method uses a mapping file, described in the documentation.
    • EXTRACTED_FILE_ENABLE_CAPA, EXTRACTED_FILE_ENABLE_CLAMAV, and EXTRACTED_FILE_ENABLE_YARA are no longer used; scanners are now configured via the Strelka backend config file. See the documentation.
    • The definition of a file scanner "hit" is now more nuanced. quarantine/ and preserved/ subdirectories are no longer used in the Extracted Files web interface. Extracted files are more easily browsed and downloaded from the Files or File Scanning dashboards in OpenSearch Dashboards.
    • Malcolm's Zeek container image now places Zeek-related files under /usr/local/zeek instead of /opt/zeek. Update any custom volume mounts or references to /opt/zeek.
  • 🧹 Code and project maintenance

    • Moved most scripts from ./shared/bin to their respective container folders. ./shared/bin/ now only contains scripts shared across multiple containers.
    • Updated Dockerfile syntax (ENV/ARG) to recommended format
    • Updated copyright year (2025 -> 2026)
    • Based the Malcolm Zeek image on the official Zeek image instead of building a custom image
  • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

    • Updated default NetBox-enriched datasets (LOGSTASH_NETBOX_ENRICHMENT_DATASETS in logstash.env): filescan.strelka, suricata.alert, zeek.conn, zeek.dce_rpc, zeek.dhcp, zeek.dns, zeek.known_hosts, zeek.known_routers, zeek.known_services, zeek.login, zeek.ntlm, zeek.notice, zeek.rdp, zeek.rfb, zeek.signatures, zeek.smb_cmd, zeek.smb_files, zeek.smb_mapping, zeek.software, zeek.ssh, zeek.weird
    • Added FILEBEAT_SCANNER_FINGERPRINT_OFFSET and FILEBEAT_SCANNER_FINGERPRINT_LENGTH in filebeat.env to customize FileBeat filestream .file_identity and .prospector.scanner.fingerprint. See here, here, and here. FILEBEAT_WATCHER_POLLING now controls native- vs. fingerprint-based file identification.
    • Added filescan.env, filescan-secret.env, and pipeline.env; added variables to redis.env for Strelka-based file scanning. zeek-secret.env has been removed, with its values now in filescan-secret.env. Many values from zeek.env are now in filescan.env or [`pipe...
Read more