Repository navigation
v0.2.3
Each archive holds mbox2nntp, uatexport, ncexport and iafetch for one system, with README and LICENSE; unpack and run, nothing to install. Check downloads against the _checksums.txt file (sha256sum -c).
.gz, .zip, mbox and directory inputs, ncexport and iafetch need no other program. .7z inputs need 7-Zip, UAT archives (.usenet) need zstd, and .usenet.xz needs xz or 7-Zip: see the README.
Changes since v0.2.2
Fixes from a code review. If you run mbox2nntp with the default tmpdir (.), update: v0.2.2 can delete a NAME.mbox that sits next to the NAME.mbox.gz you import.
mbox2nntp
- Archives are unpacked into a new directory of the run's own,
tmpdir/.mbox2nntp-*, and only that directory is removed. It is removed whatever the outcome, also on exit 1 and on SIGINT or SIGTERM. Before, an unpackedNAME.mboxreplaced and then deleted a file of that name next to the archive, and two runs sharing a tmpdir overwrote each other's copy. - A worker adds its counts to the totals before it reports done. Before, a run could report 0/0, or miss its last error and rename the input to
.done. - Header lines are dot-stuffed too, as RFC 3977 3.1.1 asks. Before, a header line starting with
.lost the dot at the server. - The Message-ID comes from the first Message-ID field only. Two such fields used to give
<a@b> <a@b>. Over the 5.28 million messages of the test corpus, 3 IDs change. - A dropped header field (Message-ID copy, Expires, a line without a colon) takes its continuation lines with it, so they no longer join the field before. A continuation line of only spaces no longer drops the article.
- A message with headers and no body no longer merges into the next one.
- An article is never sent cut short. A read error stops the file, and the input is not marked done.
- Size limits:
- An article over 1 MiB is not sent. It is counted as a 437 and as
too-big, and the input can still be marked done. - A line over 1 MiB stops the file, and the input is not marked done.
- In directory mode, files over 1 MiB are not read.
- In UAT archives, a message over 1 MiB is skipped and counted. Before, a 99 KB crafted
.usenetcould make mbox2nntp allocate 3 GiB.
- An article over 1 MiB is not sent. It is counted as a 437 and as
uatexport and ncexport
- Body lines starting
#! rnewsare quoted with>, likeFromlines. Before, the importer split such articles there. - uatexport uses the archive's group name for the default output file only if it is a safe file name. Before, a name like
../../xcould write anywhere. - ncexport checks the gzip trailer of a
.tar.gz, so a damaged tarball no longer passes unnoticed. - ncexport skips and counts
.artNfiles over 1 MiB and.dbfiles over 256 MiB. uatexport skips and counts messages over 1 MiB.
iafetch
- A download that receives no data for
-stall(default 2m) is given up and retried, resuming the.part. Before, a stalled server could hang the whole run. - No panic when two workers hit different kinds of database error.
- A file
Xis not downloaded throughX.partwhen the item also has a file namedX.part. Before, the two overwrote each other. - A
.partlonger than the file restarts with a fresh MD5 state. Before, it cost a second download.
Full Changelog: v0.2.2...v0.2.3