Skip to content

ThreatCull 2.3.1

Choose a tag to compare

@github-actions github-actions released this 03 Oct 09:29
· 129 commits to dev since this release
1f8aa41

Fetches keep large lists on disk instead of in memory, so a big Source no longer needs hundreds of
MB of RAM. Nothing to do when upgrading. After the upgrade, the first Fetch of each Source parses
its list in full once.

Changed

  • 💾 Large downloads go to disk. A Fetch writes the download to a temporary file on the data
    volume (anything over 8 MB) and parses it line by line, instead of holding the whole list in
    memory twice (as bytes and as text). On a 3 million-line, 49 MB list the peak memory of a Fetch
    fell from 165 MB to 76 MB, and it no longer grows with the list's size. The image sets
    TMPDIR=/data so the temporary file stays on the volume.