Unpack and par-check failures on Unraid/Docker

[woble]

It seems that nzbget recursively unpacks archives that fail because of CRC Failed error. Double checked using 7z to confirm the CRC error. I've been having this issue for quite a while now with some archives, prior and in v20. Because of that the pp-queue piles up, regardless of the configured strategy for post-processing.

To reproduce you would obviously need an archive that returns a CRC Failed error.

Expected behavior - download should be marked as bad.

Running on unraid as binhex-nzbget docker.

[hugbug]

If possible please send me the nzb to nzbget@gmail.com.

[woble]

I normally remove these downloads as soon as I notice to not confuse other apps with the clogged up queue. I will keep an eye on it and will send one when I experience it again, although I think it might also be related to how fresh the NZBs are and their propagation on various usenet server. Download the same NZB at a later point might result in a normal behavior.

[hugbug]

I've tried to reproduce the issue with a manually constructed corrupted download. The download completed correctly with failure status.

Maybe you have the log-file at least?

Do you remember if these were rar- or 7z-archives?

[woble]

I suspect it's somehow related to nzbToMedia. I've removed those scripts for now and will report back once I've tested it for a a day or so. As I said earlier, it seems to happen with fresh NZBs (usually in the morning in my case) that fail because they haven't fully propagated yet.

[hugbug]

that fail because they haven't fully propagated yet.

Downloads with missing articles usually are not unpacked at all if they have par2-files. CRC errors may happen in two cases:

• download don't have par2-file and nzbget therefore was not able to determine file correctness. In that case archives are unpacked (tried);
• archive files were damaged before creating par2-files. This is although possible but should be a rare case as such posts will fail for everyone and poster should know of that pretty soon.

---

If the issue happens again please save the log-file of the download before deleting download.

[woble]

Happened again on a fairly fresh nzb. Emailed both nzb and log. Tried downloading again, it failed correctly. Very odd.

Edit: downloaded again with success.

[hugbug]

The log doesn't show a recursive unpack (if with recursive you mean the unpack going on and on).

First it tried to directly unpack during downloading. That failed. Then it tried to unpack after downloading and this is where the log ends. Interesting fact: in both unpack attempts there is no output from unrar process as if the process could not start. This is suspicious.

Can you please send me the global nzbget log? It contains more info which may be useful.

[woble]

(if with recursive you mean the unpack going on and on).

Yeah, when I observe the file being unpacked inside the respective folder, I see its file size increase to what looks like its full size, presumably it fails silently and starts from 0 bytes again. Hence why it seems recursive.

[woble]

Keep experiencing the issue even with archives that seem to be OK, but not all the time. There's nothing in the log other than it's unpacking and where it's unpacking. When I look inside _unpack folder I see the file growing in size, then starting from 0 again. Extracting it manually using 7z CLI works just fine.

Can't confirm 100% yet, but restarting the docker seems to solve the problem temporarily until the next time it for some reason gets stuck with some NZB.

Any chance it could somehow be related to DirectUnpack? Or perhaps unrar process?

[hugbug]

In the log you have sent me it makes exactly two unpack attempts: one attempt during direct unpack. Unrar reports failure, direct unpack cancels. Later, when everything is downloaded another unpack attempt is made during postprocessing. That attempt also fails with unrar reporting an error. No further attempts are made. That's not a recursive unpack or unpack in the loop but rather a normal behaviour.

Do you have downloads for which more unpack attempts are made? If so can you please send me their logs?

[hugbug]

The logs exhibit the very similar behaviour as reported by other user on forum in topic I'm getting numerous PAR: FAILURE errors:

• no errors are reported during downloading;
• unrar fails to start properly (it hangs) or unrar reports checksum error;
• first par-check reports files are OK and not repair is necessary; that first par-check is made in quick mode (that's OK) and it relies on file checksums calculated during downloading; the succeeded par-check confirms the files were downloaded correctly;
• another unpack is made (after par-check) which fails again;
• second par-check is performed; this time in full mode where all downloaded files are read from disk and their checksums are calculated again. This par-check reports errors in multiple files. That means the downloaded files were not properly written into disk; otherwise their checksums would fail with first quick par-check;
• par-repair is performed: at the end of repair the repaired files are checked and they don't have correct checksums; the par-repair fails with message "repair completed but the data files still appear to be damaged". Once again that means the repaired files computed during repair were not written properly into disk as subsequent reading of files from disk and recompilation of checksums fails.

What @woble and ChapeL (forum user) have in common? It's Unraid with docker.

• What media is used for InterDir: is this a software RAID? Can you put InterDir into a simple non-RAID disk (formatted as EXT4 which doesn't use any fancy Unraid drivers or software)?
• Since when the issue started to happen? Were there updates of Unraid software recently before that?
• Are there any way to check disks for errors?

[woble]

What media is used for InterDir: is this a software RAID? Can you put InterDir into a simple non-RAID disk (formatted as EXT4 which doesn't use any fancy Unraid drivers or software)?

6 equally sized and branded SSDs in btrfs RAID10

Since when the issue started to happen? Were there updates of Unraid software recently before that?

Difficult to tell, but could be around unRAID 6.5.x or maybe even 6.4.x

Are there any way to check disks for errors?

btrfs scrub can be done, but it's unlikely to be related to that. Will run a scrub to see if there are any corruptions. Edit: ran scrub, it did not report any errors.

I had the issue this morning again (seems to be every morning or early in the day; at least as that's when nzbget is doing most of the work). With a queue being stuck with one NZB not unpacking correctly I restarted nzbget and lo and behold everything unpacked as expected. So could it be that the unrar process somehow hangs or gets stuck in a loop and doesn't communicate properly back to nzbget about it?