Skip to content

ThreatCull 2.4.0

Choose a tag to compare

@github-actions github-actions released this 03 Oct 11:29
· 119 commits to dev since this release
56272b2

Fetch and Compile are about twice as fast, runs show how long each stage took, and Settings shows
how much disk the database and Outputs use. Nothing to do when upgrading. After the upgrade, the
first Fetch of each Source parses its list in full once.

Changed

  • 🚀 Faster Fetch and Compile. Domain validation skips steps that can't change its result, the
    database looks each new entry up once instead of three times, and Outputs read from one sorted
    index instead of sorting once each. Measured with scripts/bench.py on 1 million lines: the first
    Fetch went from 45 s to 19 s and a Compile from 29 s to 16 s. What counts as a valid entry is
    unchanged.
  • ⏱️ Run timings per stage. The run history has a "Took" column with the time each stage took
    (download, parse, apply for a Fetch; prune, score, allowlist, stats, outputs, publish for a
    Compile), and /api/v1/runs returns duration_seconds and timings.
  • 💽 Storage shown. Settings and the dashboard footer show the size of the database and of the
    Outputs; /api/v1/status returns database_bytes and outputs_bytes.
  • 📏 Sizing guide for any host. The Sizing section of the README gives measured times, memory and
    database sizes for lists of 0.5 to 3 million lines and advice for small hosts.
    scripts/bench.py measures your own host.