Skip to content

Releases: atomdrift-project/cleave

v2.4.0

Choose a tag to compare

@tstromberg tstromberg released this 25 Jul 18:56

Faster

  • Large-archive scans up to ~8× faster (a 63k-member archive: 2,865s → matches the 337s loose-file baseline).

Detection

  • ava exfil now needs explicit exfil vocabulary
  • IP filter recovers real hosts like 3.147.61.167 that were dropped as version-strings.
  • Better CRX signed-but-unverified detection.
  • New filetype: static libs (.a).

Correctness

  • Dependency findings dedup per-artifact, not per-report

Output

  • --format=interpret renders LLM-friendly ASCII windows with offset gutters (cheaper for local models); terminal view unchanged.

v2.3.0 (migrated from Codeberg)

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Features

  • length_min/length_max bound regex match length; replaces costly {4000,} repetitions.
  • KV length bounds now match strings by byte length (previously never matched).
  • New regex-memory validator flags patterns that exceed budget or silently never compile.
  • Benign fixtures can require a minimum score.
  • Compact JSON: parent id (pid), trait IDs in descriptions.
  • Archive entries report entropy, magic bytes, and extension/type mismatch.
  • PyInstaller entries tagged by kind.
  • Tiny render: benign files show their strongest traits; binary windows get hex-offset rows.
  • FreeBSD/arm64 builds again!
  • Automated trait publishing on a timer (unsigned for now).

Fixes

  • near_* windows could match on any: legs alone while an all: leg sat far away; windows now require every all: leg. Fixes false positives.
  • Java exfil triggers narrowed to explicit exfil terms; upload alone no longer fires.
  • Mach-O malformed-header check skips files under 4 KB.
  • XOR payload noise in text files suppressed unless it looks like runnable source.

Optimizations

  • Regex engine rebuilt on regex_automata with byte-budgeted caches; removes 1.5-4 GB RSS spikes on large binaries. Match semantics unchanged.
  • Parallel prefilter and path-trait evaluation for large archives.
  • Indexed unless: suppression; was a linear scan per finding.
  • YARA scanner cache 4 to 64; avoids ~31k scanner rebuilds on a 13k-member archive.

v2.2.1

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Performance improvements - lazy loading indexes & mappers

v2.2.0

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Features

  • New filetype recognition so rules can target WASM, Android DEX, CAB, and APK.
  • Local reference graphs linking files to their package/provenance origins (new reference_graph type).
  • Richer package registry lookups, with registry records grafted into analysis output.
  • New crit: exception composite rules for clearer intent signalling for trait authors.
  • Facts now carry metric spans (byte ranges) pinpointing where each measurement's evidence lives.
  • Tiny output gains component-level line detail and a top-5 files summary.

Fixes

  • Fixed YARA cold-bucket and init deadlocks; stopped clearing the YARA cache unnecessarily.
  • Fixed base64/XOR false positives in SVG files; added XOR source-detection test fixtures.
  • Loosened IP-address and metrics validation to reduce false positives.
  • Description-length requirements now also apply to composite rules.

Performance

  • Non-blocking cache cleanup.

v2.1.2

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16
  • matcher: fix subtle hex variable gap bug
  • matcher: add initial entropy window metrics
  • deps: pin to rusqlite 0.39.0 for rust 1.94 compat
  • validate: increase max suppress cap

v2.1.1

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16
  • Precompiled YARA files — load prebuilt YARA rules (new yara-precompile tool)
  • Dynamic YARA tier loading — tiers load on demand instead of all upfront
  • YARA tier preheating — warms the right tiers based on incoming file type for lower latency.
  • validate --soft flag — skips directory caps and other taxonomy-cleanliness rules
  • Loads of dependency updates

v2.1.0

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16
  • DMG & go.mod support
  • Experimental --fetch support
  • Moved more parsing onto filefacts.
  • Added iter-files --newer-than.
  • Added validate --soft.
  • Tightened trait validation checks.
  • Tightened testdata validation.
  • Improved archive decompression handling.
  • Fixed Alpine APK multi-member parsing.
  • Reduced compact JSON size.
  • Improved batch file symbol handling.
  • Improved Java class handling.
  • Improved source symbol matching.
  • Improved binary metadata anchoring.
  • Improved rizin/filefacts cache maintenance.
  • Updated manifest generator behavior.
  • Added pre-commit hook support.
  • Updated dependencies.

v2.0.0

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Since our last release (1.4.0):

  • engine swap — all parsing moved to the filefacts crate; breaking JSON schema v3 +
    CLI renames
  • Context-centric output — context as sole evidence surface, grep-style hex, Z85 compact
  • Package + platform coverage — npm/crate/NuGet/APK/ASAR/CRX archives;
    FortiOS/PanOS/JunOS/Ivanti, RouterOS, JCL; Program Identity
  • Signed trait-bundle updates — R2-backed .tar.zst with sha256, pin/check/update, daily check
  • Memory & perf — admission gate, bounded regex cache (fixed ~7.7 GB leak), SIMD matching
  • Correctness — Mach-O offsets, archive XOR/payload recursion, generic JSON, fewer false
    positives

Since rc5:

  • Context-centric output — new context module (+925) and major output rewrite: context is the
    sole evidence surface, grep-style hex windows with bold evidence, overlap-dedup by
    confidence×level
  • New appliance platforms — FortiOS, PanOS, JunOS, Netscaler, Ivanti detection; plus RouterOS
    (.rsc) and JCL (z/OS) rule support
  • Program Identity — publisher/package metadata surfaced; signing info promoted to notable
  • Z85 binary encoding for compact output; improved embedded-binary display, per-trait offsets
  • Binary fixes — corrected Mach-O FAT offsets; tighter tiny-binary context windows
  • Rule eval timeouts extended (90s) for heavily loaded systems
  • Stricter trait-reference validation, perf sweep, dependency updates

v2.0.0-rc.5

v2.0.0-rc.5 Pre-release
Pre-release

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Features

  • Added R2-backed trait bundle updates with sha256 verification, atomic install, pin/check/update support, and
    signed manifest publishing.
  • Updated to filefacts 0.9.5 and stng 1.6.0 for richer package, PE/CLR, Mach-O, PE import, and string-extraction
    signals.
  • Added archive routing for package ecosystems like gem, npm, crate, conda, egg, NuGet, IPA, VSIX, Android/
    Alpine APK, and Arch/FreeBSD pkg.
  • Archive/outer-scoped composites can now pool findings across archive members, useful for browser extensions
    and split payload packages.
  • JVM bytecode imports now include owner-qualified symbols like java/lang/Runtime.exec.
  • npm script capability checks moved toward YAML traits for more precise per-hook supply-chain features.

Performance

  • Trait update artifacts now use .tar.zst bundles instead of git clones/pulls.
  • Manifest generation now compat-tests release versions and HEAD, caches validation, and supports pre-publish
    integrity checks.
  • Archive members now share the same encoded-payload processing path as standalone files instead of duplicating
    logic.

Bugfixes

  • Fixed archive members missing encoded-payload findings and decoded payload recursion.
  • Fixed archive members skipping XOR extraction, which weakened detection inside npm/zips/jars.
  • Tiny structured manifests under 100 bytes now still evaluate KV/value traits.
  • UTF-16 test matching now normalizes text like production analysis.
  • Office external-template findings now distinguish remote injection targets from benign local template paths.
  • IP matching no longer lifts false IPv4 hits out of OIDs or long version strings.
  • Edge extension update URLs are treated as first-party, avoiding Chrome extension false positives.
  • Embedded-code evidence offsets now map back to parent-file line proximity correctly.

v2.0.0-rc.4

v2.0.0-rc.4 Pre-release
Pre-release

Choose a tag to compare

@tstromberg tstromberg released this 24 Jul 05:16

Update notifications (initial implementation, not yet live)

  • Once-a-day update check against a static TOML manifest (no telemetry); disable with --no-update-check or CLEAVE_NO_UPDATE_CHECK
  • New manifest tooling: manifest-gen (Go, soak-gated beta/stable channels) and update-manifest (bash, one-shot builds with cosign signing)

Memory & performance

  • Memory-aware admission gate: large archives serialize while small files fill the pool, using live available-memory readings so concurrent instances share the host
  • Archive members analyzed in bounded byte-windows instead of all-resident; per-member reports folded and dropped immediately
  • Trait regexes moved to a bounded LRU cache — the old design leaked every compiled engine (~7.7 GB RSS on member-heavy archives)
  • word:/substr: matching via SIMD literal searchers, no regex engine; ASCII raw patterns skip the unicode engine entirely
  • Evidence values capped at storage time; exact match counts computed only when a rule needs them
  • Accurate physical-core detection (SMT siblings excluded, Apple Silicon P-cores, illumos, FreeBSD/arm64 sysctl fix) drives pool sizing
  • Instantaneous RSS readings on FreeBSD/illumos so the memory gate can reopen; optional memprofile feature for phase-tagged heap accounting

Matching semantics & correctness

  • AST query cache made unbounded with serialized compiles — fixes nondeterministic finding loss during parallel archive analysis
  • UTF-16 scripts normalized to UTF-8 before text matching — a UTF-16 VBS now matches like its UTF-8 twin
  • not: exact: is now case-sensitive, so excluding PowerShell doesn't excuse pOwErShElL
  • AST walks bounded by CPU time, not wall clock — starved threads no longer silently drop detections
  • Markdown code blocks treated as documentation, not embedded scripts
  • New trait surface: type: comment, type: path (basename/dirname scopes), import alias: filters, multi-arg args: matching, is: valid_ip, bare section names (rsrc,
    __cstring) for PE/Mach-O
  • Cache-hit reports re-stamp file.basename/file.stem — no more phantom diffs between identical files

Analyzers

  • Chrome/WebExtension manifests now emit neutral per-permission and per-host capability traits instead of hardcoded risk scores
  • XML parsing rejects pathologically deep documents (stack-overflow DoS guard)
  • VBA identifier metrics (length, entropy, distinct triggers); compact JSON schema bumped to v6 with smaller evidence locations

Rule validation

  • New checks: scope: archive with non-archive for:, repeated inline unless: exclusions, duplicate detection scoped to matching sections
  • Explicit for: type limit relaxed to 8; per-condition-type caps removed; hardware/gpu taxonomy section added

API & maintenance

  • New scan_files() library entry point (explicit path list, shared resources, streaming callback)
  • Symbol source field removed from output; deadlock detection now an opt-in feature
  • Dependency updates