Windows: illegal char in resulting filename: SAB bails out, but manual unrar OK with nice renaming

[sanderjo]

Triggered by https://www.reddit.com/r/SABnzbd/comments/i7nz5j/weird_ascii_characters_cause_download_to/

On Windows, SAB bails out with this crafted NZB:
https://raw.githubusercontent.com/sanderjo/NZBs/master/reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd.nzb with this error:

ERROR: write error ( \\?\C:\Users\blabla\AppData\Local\sabnzbd\Downloads\complete\_UNPACK_reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd.2\bbbb�zzzz.bin)

Quite understandably: there is an illegal hex 1A in the filename.

But ... if SAB just Downloads the post (no unpacking), a manual unrar goes OK, resulting in a renamed filename. See below. So ... SAB can do better by just letting unrar do the renaming?

C:\Users\blabbab\AppData\Local\sabnzbd\Downloads\complete\reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd>"\Program Files\SABnzbd3\win\unrar\x64\UnRAR.exe" x *1.rar

UNRAR 5.90 x64 freeware      Copyright (c) 1993-2020 Alexander Roshal


Extracting from illegal_char.part1.rar

Cannot create bbbb→zzzz.bin
De syntaxis van de bestandsnaam, mapnaam of volumenaam is onjuist.
WARNING: Attempting to correct the invalid file name
Renaming bbbb→zzzz.bin to bbbb_zzzz.bin
Extracting  bbbb_zzzz.bin                                             49%

Extracting from illegal_char.part2.rar

...         bbbb→zzzz.bin                                             99%

Extracting from illegal_char.part3.rar

...         bbbb→zzzz.bin                                             OK
Total errors: 1

It says errors: 1, but the file is there, renamed without the illegal char:

14-08-2020 08:38 10.485.760 bbbb_zzzz.bin

The magic happens here:

Cannot create bbbb→zzzz.bin
De syntaxis van de bestandsnaam, mapnaam of volumenaam is onjuist.
WARNING: Attempting to correct the invalid file name
Renaming bbbb→zzzz.bin to bbbb_zzzz.bin

It is a warning, but all goes well.

BTW: I don't understand why the SAB-call of unrar does not trigger this behaviour in unrar (but just an OS error). Maybe because of all the CLI options SAB gives to unrar?

2020-08-14 09:22:59,374::DEBUG::[newsunpack:748] Analyzing rar file ... RAR5 found
2020-08-14 09:22:59,374::DEBUG::[newsunpack:749] Running unrar "C:\Program Files\SABnzbd3\win\unrar\x64\UnRAR.exe" "x" "-idp" "-o-" "-or" "-ai" "-p-" "C:\Usersblablaa\AppData\Local\sabnzbd\Downloads\incomplete\refte.1\illegal_char.part1.rar" "\\?\C:\Usersblablaa\AppData\Local\sabnzbd\Downloads\complete\_UNPACK_reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd.2\"
2020-08-14 09:22:59,482::ERROR::[newsunpack:845] ERROR: write error ( \\?\C:\Usersblablaa\AppData\Local\sabnzbd\Downloads\complete\_UNPACK_reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd.2\bbbb�zzzz.bin)
2020-08-14 09:22:59,484::DEBUG::[newsunpack:901] UNRAR output 
UNRAR 5.90 x64 freeware      Copyright (c) 1993-2020 Alexander Roshal


Extracting from C:\Usersblablaa\AppData\Local\sabnzbd\Downloads\incomplete\refte.1\illegal_char.part1.rar

Cannot create \\?\C:\Usersblablaa\AppData\Local\sabnzbd\Downloads\complete\_UNPACK_reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd.2\bbbb�zzzz.bin

EDIT: no, the CLI options are not the cause; with CLI options, a manual unrar goes OK too:

C:\Users\dddd\AppData\Local\sabnzbd\Downloads\complete\reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd>"\Program Files\SABnzbd3\win\unrar\x64\UnRAR.exe" x -idp -o- -or -ai -p- *1.rar

UNRAR 5.90 x64 freeware      Copyright (c) 1993-2020 Alexander Roshal


Extracting from illegal_char.part1.rar

Cannot create bbbb→zzzz.bin
De syntaxis van de bestandsnaam, mapnaam of volumenaam is onjuist.
WARNING: Attempting to correct the invalid file name
Renaming bbbb→zzzz.bin to bbbb_zzzz.bin
Extracting  bbbb_zzzz.bin

Extracting from illegal_char.part2.rar

...         bbbb→zzzz.bin

Extracting from illegal_char.part3.rar

...         bbbb→zzzz.bin                                             OK
Total errors: 1

C:\Users\dddd\AppData\Local\sabnzbd\Downloads\complete\reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd>

[Safihre]

After we read the error-line, we stop parsing the output, so maybe unrar actually fixed it but we just didn't log it.
This is a duplicate of #1304 that I closed because I couldn't reproduce it.. Maybe with this NZB I can.

[Safihre]

Indeed we p.kill() the process right after reading the error. Manually debugging I am able to read more lines, but it still fails:

proc.readline()
b'WARNING: Attempting to correct the invalid file name\r\n'
proc.readline()
b'Cannot create \\\\_\\C_\\Users\\saf\\Videos\\Complete\\_UNPACK_reftestnzb_illegal_char_HEX_1A_in_filename_0cf049082ecd\\bbbb_zzzz.bin\r\n'

Reason being is that we use long-path notation, so that we can still unpack jobs that exceed Windows's 270-char limit on paths. However, it seems that unrar's internal correction scheme can't handle that.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Safihre]

Closing as stale.