Releases: atomdrift-project/cleave
Releases · atomdrift-project/cleave
Release list
v2.4.0
Faster
- Large-archive scans up to ~8× faster (a 63k-member archive: 2,865s → matches the 337s loose-file baseline).
Detection
- ava exfil now needs explicit exfil vocabulary
- IP filter recovers real hosts like 3.147.61.167 that were dropped as version-strings.
- Better CRX signed-but-unverified detection.
- New filetype: static libs (.a).
Correctness
- Dependency findings dedup per-artifact, not per-report
Output
- --format=interpret renders LLM-friendly ASCII windows with offset gutters (cheaper for local models); terminal view unchanged.
v2.3.0 (migrated from Codeberg)
Features
- length_min/length_max bound regex match length; replaces costly {4000,} repetitions.
- KV length bounds now match strings by byte length (previously never matched).
- New regex-memory validator flags patterns that exceed budget or silently never compile.
- Benign fixtures can require a minimum score.
- Compact JSON: parent id (pid), trait IDs in descriptions.
- Archive entries report entropy, magic bytes, and extension/type mismatch.
- PyInstaller entries tagged by kind.
- Tiny render: benign files show their strongest traits; binary windows get hex-offset rows.
- FreeBSD/arm64 builds again!
- Automated trait publishing on a timer (unsigned for now).
Fixes
- near_* windows could match on any: legs alone while an all: leg sat far away; windows now require every all: leg. Fixes false positives.
- Java exfil triggers narrowed to explicit exfil terms; upload alone no longer fires.
- Mach-O malformed-header check skips files under 4 KB.
- XOR payload noise in text files suppressed unless it looks like runnable source.
Optimizations
- Regex engine rebuilt on regex_automata with byte-budgeted caches; removes 1.5-4 GB RSS spikes on large binaries. Match semantics unchanged.
- Parallel prefilter and path-trait evaluation for large archives.
- Indexed unless: suppression; was a linear scan per finding.
- YARA scanner cache 4 to 64; avoids ~31k scanner rebuilds on a 13k-member archive.
v2.2.1
Performance improvements - lazy loading indexes & mappers
v2.2.0
Features
- New filetype recognition so rules can target WASM, Android DEX, CAB, and APK.
- Local reference graphs linking files to their package/provenance origins (new reference_graph type).
- Richer package registry lookups, with registry records grafted into analysis output.
- New crit: exception composite rules for clearer intent signalling for trait authors.
- Facts now carry metric spans (byte ranges) pinpointing where each measurement's evidence lives.
- Tiny output gains component-level line detail and a top-5 files summary.
Fixes
- Fixed YARA cold-bucket and init deadlocks; stopped clearing the YARA cache unnecessarily.
- Fixed base64/XOR false positives in SVG files; added XOR source-detection test fixtures.
- Loosened IP-address and metrics validation to reduce false positives.
- Description-length requirements now also apply to composite rules.
Performance
- Non-blocking cache cleanup.
v2.1.2
- matcher: fix subtle hex variable gap bug
- matcher: add initial entropy window metrics
- deps: pin to rusqlite 0.39.0 for rust 1.94 compat
- validate: increase max suppress cap
v2.1.1
- Precompiled YARA files — load prebuilt YARA rules (new yara-precompile tool)
- Dynamic YARA tier loading — tiers load on demand instead of all upfront
- YARA tier preheating — warms the right tiers based on incoming file type for lower latency.
- validate --soft flag — skips directory caps and other taxonomy-cleanliness rules
- Loads of dependency updates
v2.1.0
- DMG & go.mod support
- Experimental --fetch support
- Moved more parsing onto filefacts.
- Added iter-files --newer-than.
- Added validate --soft.
- Tightened trait validation checks.
- Tightened testdata validation.
- Improved archive decompression handling.
- Fixed Alpine APK multi-member parsing.
- Reduced compact JSON size.
- Improved batch file symbol handling.
- Improved Java class handling.
- Improved source symbol matching.
- Improved binary metadata anchoring.
- Improved rizin/filefacts cache maintenance.
- Updated manifest generator behavior.
- Added pre-commit hook support.
- Updated dependencies.
v2.0.0
Since our last release (1.4.0):
- engine swap — all parsing moved to the filefacts crate; breaking JSON schema v3 +
CLI renames - Context-centric output — context as sole evidence surface, grep-style hex, Z85 compact
- Package + platform coverage — npm/crate/NuGet/APK/ASAR/CRX archives;
FortiOS/PanOS/JunOS/Ivanti, RouterOS, JCL; Program Identity - Signed trait-bundle updates — R2-backed .tar.zst with sha256, pin/check/update, daily check
- Memory & perf — admission gate, bounded regex cache (fixed ~7.7 GB leak), SIMD matching
- Correctness — Mach-O offsets, archive XOR/payload recursion, generic JSON, fewer false
positives
Since rc5:
- Context-centric output — new context module (+925) and major output rewrite: context is the
sole evidence surface, grep-style hex windows with bold evidence, overlap-dedup by
confidence×level - New appliance platforms — FortiOS, PanOS, JunOS, Netscaler, Ivanti detection; plus RouterOS
(.rsc) and JCL (z/OS) rule support - Program Identity — publisher/package metadata surfaced; signing info promoted to notable
- Z85 binary encoding for compact output; improved embedded-binary display, per-trait offsets
- Binary fixes — corrected Mach-O FAT offsets; tighter tiny-binary context windows
- Rule eval timeouts extended (90s) for heavily loaded systems
- Stricter trait-reference validation, perf sweep, dependency updates
v2.0.0-rc.5
Features
- Added R2-backed trait bundle updates with sha256 verification, atomic install, pin/check/update support, and
signed manifest publishing. - Updated to filefacts 0.9.5 and stng 1.6.0 for richer package, PE/CLR, Mach-O, PE import, and string-extraction
signals. - Added archive routing for package ecosystems like gem, npm, crate, conda, egg, NuGet, IPA, VSIX, Android/
Alpine APK, and Arch/FreeBSD pkg. - Archive/outer-scoped composites can now pool findings across archive members, useful for browser extensions
and split payload packages. - JVM bytecode imports now include owner-qualified symbols like java/lang/Runtime.exec.
- npm script capability checks moved toward YAML traits for more precise per-hook supply-chain features.
Performance
- Trait update artifacts now use .tar.zst bundles instead of git clones/pulls.
- Manifest generation now compat-tests release versions and HEAD, caches validation, and supports pre-publish
integrity checks. - Archive members now share the same encoded-payload processing path as standalone files instead of duplicating
logic.
Bugfixes
- Fixed archive members missing encoded-payload findings and decoded payload recursion.
- Fixed archive members skipping XOR extraction, which weakened detection inside npm/zips/jars.
- Tiny structured manifests under 100 bytes now still evaluate KV/value traits.
- UTF-16 test matching now normalizes text like production analysis.
- Office external-template findings now distinguish remote injection targets from benign local template paths.
- IP matching no longer lifts false IPv4 hits out of OIDs or long version strings.
- Edge extension update URLs are treated as first-party, avoiding Chrome extension false positives.
- Embedded-code evidence offsets now map back to parent-file line proximity correctly.
v2.0.0-rc.4
Update notifications (initial implementation, not yet live)
- Once-a-day update check against a static TOML manifest (no telemetry); disable with --no-update-check or CLEAVE_NO_UPDATE_CHECK
- New manifest tooling: manifest-gen (Go, soak-gated beta/stable channels) and update-manifest (bash, one-shot builds with cosign signing)
Memory & performance
- Memory-aware admission gate: large archives serialize while small files fill the pool, using live available-memory readings so concurrent instances share the host
- Archive members analyzed in bounded byte-windows instead of all-resident; per-member reports folded and dropped immediately
- Trait regexes moved to a bounded LRU cache — the old design leaked every compiled engine (~7.7 GB RSS on member-heavy archives)
- word:/substr: matching via SIMD literal searchers, no regex engine; ASCII raw patterns skip the unicode engine entirely
- Evidence values capped at storage time; exact match counts computed only when a rule needs them
- Accurate physical-core detection (SMT siblings excluded, Apple Silicon P-cores, illumos, FreeBSD/arm64 sysctl fix) drives pool sizing
- Instantaneous RSS readings on FreeBSD/illumos so the memory gate can reopen; optional memprofile feature for phase-tagged heap accounting
Matching semantics & correctness
- AST query cache made unbounded with serialized compiles — fixes nondeterministic finding loss during parallel archive analysis
- UTF-16 scripts normalized to UTF-8 before text matching — a UTF-16 VBS now matches like its UTF-8 twin
- not: exact: is now case-sensitive, so excluding PowerShell doesn't excuse pOwErShElL
- AST walks bounded by CPU time, not wall clock — starved threads no longer silently drop detections
- Markdown code blocks treated as documentation, not embedded scripts
- New trait surface: type: comment, type: path (basename/dirname scopes), import alias: filters, multi-arg args: matching, is: valid_ip, bare section names (rsrc,
__cstring) for PE/Mach-O - Cache-hit reports re-stamp file.basename/file.stem — no more phantom diffs between identical files
Analyzers
- Chrome/WebExtension manifests now emit neutral per-permission and per-host capability traits instead of hardcoded risk scores
- XML parsing rejects pathologically deep documents (stack-overflow DoS guard)
- VBA identifier metrics (length, entropy, distinct triggers); compact JSON schema bumped to v6 with smaller evidence locations
Rule validation
- New checks: scope: archive with non-archive for:, repeated inline unless: exclusions, duplicate detection scoped to matching sections
- Explicit for: type limit relaxed to 8; per-condition-type caps removed; hardware/gpu taxonomy section added
API & maintenance
- New scan_files() library entry point (explicit path list, shared resources, streaming callback)
- Symbol source field removed from output; deadlock detection now an opt-in feature
- Dependency updates