New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpack fails if filename is too long [Windows 10] #127

Closed
RollingStar opened this Issue Dec 15, 2015 · 14 comments

Comments

Projects
None yet
6 participants
@RollingStar

RollingStar commented Dec 15, 2015

This is on NZBGet 16.3. I tried to download a file with a really long name - then I think it was supposed to be decompressed into a file with an equally-long name. Together it's probably over 256 characters, which is a problem on Windows 10 x64. I'm guessing that's why the unpack failed.

Now the .out.tmp files are in the DestDir. "Post-process again" does nothing.

Could not move file d:\use\int[145 character string].#97\2071.out.tmp to d:\use\int[145 character string].#97[151 character string].nfo: No such file or directory
Could not move file d:\use\int[145 character string].#97\2072.out.tmp to d:\use\int[145 character string].#97[151 character string].nzb: No such file or directory
Could not move file d:\use\int[145 character string].#97\2073.out.tmp to d:\use\int[145 character string].#97[151 character string].par2: No such file or directory
Could not move file d:\use\int[145 character string].#97\2074.out.tmp to d:\use\int[145 character string].#97[151 character string].part001.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2075.out.tmp to d:\use\int[145 character string].#97[151 character string].part002.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2076.out.tmp to d:\use\int[145 character string].#97[151 character string].part003.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2077.out.tmp to d:\use\int[145 character string].#97[151 character string].part004.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2078.out.tmp to d:\use\int[145 character string].#97[151 character string].part005.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2079.out.tmp to d:\use\int[145 character string].#97[151 character string].part006.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2080.out.tmp to d:\use\int[145 character string].#97[151 character string].part007.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2081.out.tmp to d:\use\int[145 character string].#97[151 character string].part008.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2082.out.tmp to d:\use\int[145 character string].#97[151 character string].part009.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2083.out.tmp to d:\use\int[145 character string].#97[151 character string].part010.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2084.out.tmp to d:\use\int[145 character string].#97[151 character string].part011.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2085.out.tmp to d:\use\int[145 character string].#97[151 character string].part012.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2086.out.tmp to d:\use\int[145 character string].#97[151 character string].part013.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2087.out.tmp to d:\use\int[145 character string].#97[151 character string].part014.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2088.out.tmp to d:\use\int[145 character string].#97[151 character string].part015.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2089.out.tmp to d:\use\int[145 character string].#97[151 character string].part016.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2090.out.tmp to d:\use\int[145 character string].#97[151 character string].part017.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2091.out.tmp to d:\use\int[145 character string].#97[151 character string].part018.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2092.out.tmp to d:\use\int[145 character string].#97[151 character string].part019.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2093.out.tmp to d:\use\int[145 character string].#97[151 character string].part020.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2094.out.tmp to d:\use\int[145 character string].#97[151 character string].part021.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2095.out.tmp to d:\use\int[145 character string].#97[151 character string].part022.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2096.out.tmp to d:\use\int[145 character string].#97[151 character string].part023.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2097.out.tmp to d:\use\int[145 character string].#97[151 character string].part024.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2098.out.tmp to d:\use\int[145 character string].#97[151 character string].part025.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2099.out.tmp to d:\use\int[145 character string].#97[151 character string].part026.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2100.out.tmp to d:\use\int[145 character string].#97[151 character string].part027.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2101.out.tmp to d:\use\int[145 character string].#97[151 character string].part028.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2102.out.tmp to d:\use\int[145 character string].#97[151 character string].part029.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2103.out.tmp to d:\use\int[145 character string].#97[151 character string].part030.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2104.out.tmp to d:\use\int[145 character string].#97[151 character string].part031.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2105.out.tmp to d:\use\int[145 character string].#97[151 character string].part032.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2106.out.tmp to d:\use\int[145 character string].#97[151 character string].part033.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2107.out.tmp to d:\use\int[145 character string].#97[151 character string].part034.rar: No such file or directory
Could not move file d:\use\int[145 character string].#97\2108.out.tmp to d:\use\int[145 character string].#97[151 character string].part035.rar: No such file or directory

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 15, 2015

Member

As you know it's a Windows limitation. What can NZBGet do about this?

Member

hugbug commented Dec 15, 2015

As you know it's a Windows limitation. What can NZBGet do about this?

@arogl

This comment has been minimized.

Show comment
Hide comment
@arogl

arogl Dec 16, 2015

Internally you could represent the drive via the ?\d: representation as this negates the 255 char limit.

arogl commented Dec 16, 2015

Internally you could represent the drive via the ?\d: representation as this negates the 255 char limit.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 16, 2015

Member

First that would require a total rework of disk access in nzbget, since that trick works only for unicode versions of disk functions.

Second, what's the point if Explorer and other programs will not be able to access the files anyway?

Third, does unrar support that extended syntax? Because if doesn't we'll not got much further.

The only plausible solution I see is to limit the length of created folders (instead of blindly using nzb-names as folder names).

Member

hugbug commented Dec 16, 2015

First that would require a total rework of disk access in nzbget, since that trick works only for unicode versions of disk functions.

Second, what's the point if Explorer and other programs will not be able to access the files anyway?

Third, does unrar support that extended syntax? Because if doesn't we'll not got much further.

The only plausible solution I see is to limit the length of created folders (instead of blindly using nzb-names as folder names).

@arogl

This comment has been minimized.

Show comment
Hide comment
@arogl

arogl Dec 16, 2015

@hugbug commented on 16 Dec 2015, 10:32 AM AEST:

First that would require a total rework of disk access in nzbget, since that trick works only for unicode versions of disk functions.

The representation works for all types of disk functions. Does it matter if it is not unicode?

https://msdn.microsoft.com/en-au/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath

Second, what's the point if Explorer and other programs will not be able to access the files anyway?

Explorer can access the files it is unable to move the directory structure nor can it delete the file. However a delete in the command prompt using \?\ does work.

Third, does unrar support that extended syntax? Because if doesn't we'll not got much further.

Initial testing shows it handles the ?\d: if the file does not exist it gives an error that the network path does not exist.

The only plausible solution I see is to limit the length of created folders (instead of blindly using nzb-names as folder names).

arogl commented Dec 16, 2015

@hugbug commented on 16 Dec 2015, 10:32 AM AEST:

First that would require a total rework of disk access in nzbget, since that trick works only for unicode versions of disk functions.

The representation works for all types of disk functions. Does it matter if it is not unicode?

https://msdn.microsoft.com/en-au/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath

Second, what's the point if Explorer and other programs will not be able to access the files anyway?

Explorer can access the files it is unable to move the directory structure nor can it delete the file. However a delete in the command prompt using \?\ does work.

Third, does unrar support that extended syntax? Because if doesn't we'll not got much further.

Initial testing shows it handles the ?\d: if the file does not exist it gives an error that the network path does not exist.

The only plausible solution I see is to limit the length of created folders (instead of blindly using nzb-names as folder names).

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 16, 2015

Member

The representation works for all types of disk functions. Does it matter if it is not unicode?

Yes it does, only unicode functions work with paths longer than 260 chars, see your link.

Explorer can access the files it is unable to move the directory structure nor can it delete the file.

Not good. Creating files that are not deletable or movable is even worse than failing.

When testing (unrar, etc.) please test with long paths, not just with prefix "?". Can unrar work with paths longer than 260 characters? And what about media players? Can they play such files?

Member

hugbug commented Dec 16, 2015

The representation works for all types of disk functions. Does it matter if it is not unicode?

Yes it does, only unicode functions work with paths longer than 260 chars, see your link.

Explorer can access the files it is unable to move the directory structure nor can it delete the file.

Not good. Creating files that are not deletable or movable is even worse than failing.

When testing (unrar, etc.) please test with long paths, not just with prefix "?". Can unrar work with paths longer than 260 characters? And what about media players? Can they play such files?

@RollingStar

This comment has been minimized.

Show comment
Hide comment
@RollingStar

RollingStar Dec 16, 2015

How about this:

if (internal archive name + folder name > 255 characters)
    x = length of (255 - archive name)
    rename folder to first x characters of old folder name

I don't think it accounts for every situation, and obviously it's pseudocode, but it's the kind of solution that might work on Windows without getting into edge cases that Windows & Windows programs may not support.

And at the very least, the error could be more informative when NZBget tries to make a file path too long. "No such file or directory" doesn't accurately report to the user what went wrong.

RollingStar commented Dec 16, 2015

How about this:

if (internal archive name + folder name > 255 characters)
    x = length of (255 - archive name)
    rename folder to first x characters of old folder name

I don't think it accounts for every situation, and obviously it's pseudocode, but it's the kind of solution that might work on Windows without getting into edge cases that Windows & Windows programs may not support.

And at the very least, the error could be more informative when NZBget tries to make a file path too long. "No such file or directory" doesn't accurately report to the user what went wrong.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 17, 2015

Member

I'll try the \?\ approach. Need to refactor all disk access functions for this. This will take some time as I'm working on other parts right now.

In the meantime please send me a couple of test nzbs (nzbget@gmail.com).

"No such file or directory" doesn't accurately report to the user what went wrong.

The error text comes from Windows. I agree it could be better.

Member

hugbug commented Dec 17, 2015

I'll try the \?\ approach. Need to refactor all disk access functions for this. This will take some time as I'm working on other parts right now.

In the meantime please send me a couple of test nzbs (nzbget@gmail.com).

"No such file or directory" doesn't accurately report to the user what went wrong.

The error text comes from Windows. I agree it could be better.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 27, 2015

Member

I'm working on it right now.
Guys, please send me test nzbs to nzbget@gmail.com.

Member

hugbug commented Dec 27, 2015

I'm working on it right now.
Guys, please send me test nzbs to nzbget@gmail.com.

@hugbug hugbug referenced this issue Dec 28, 2015

Closed

Unicode and extra long file paths on Windows #136

7 of 7 tasks complete
@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 29, 2015

Member

Fixed in #136 (for v17.0).

If someone wants give it a try (before v17.0-testing is available) please send me a note to nzbget@gmail.com and I'll provide with the test version.

Member

hugbug commented Dec 29, 2015

Fixed in #136 (for v17.0).

If someone wants give it a try (before v17.0-testing is available) please send me a note to nzbget@gmail.com and I'll provide with the test version.

@hugbug hugbug closed this Dec 29, 2015

@raymono

This comment has been minimized.

Show comment
Hide comment
@raymono

raymono Feb 16, 2016

I was also facing same problem so tried long path tool ,its really helpful & easy to use.

raymono commented Feb 16, 2016

I was also facing same problem so tried long path tool ,its really helpful & easy to use.

@justn6

This comment has been minimized.

Show comment
Hide comment
@justn6

justn6 Feb 17, 2016

Same issue here, definitely contacting you for the pre-release version.

justn6 commented Feb 17, 2016

Same issue here, definitely contacting you for the pre-release version.

@Bec-de-Xorbin

This comment has been minimized.

Show comment
Hide comment
@Bec-de-Xorbin

Bec-de-Xorbin Mar 20, 2016

Download goes ok but unpack fails with error code 10. Shall I send you a test nzb?

Unrar: Cannot create C:\xxxxx\xxxxxx\xxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\_unpack\\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xx

Unrar: The filename, directory name, or volume label syntax is incorrect.

Version is of course 17.0-testing-r1660

Bec-de-Xorbin commented Mar 20, 2016

Download goes ok but unpack fails with error code 10. Shall I send you a test nzb?

Unrar: Cannot create C:\xxxxx\xxxxxx\xxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\_unpack\\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xx

Unrar: The filename, directory name, or volume label syntax is incorrect.

Version is of course 17.0-testing-r1660

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Mar 20, 2016

Member

Shall I send you a test nzb?

Yes, please.

Member

hugbug commented Mar 20, 2016

Shall I send you a test nzb?

Yes, please.

@hugbug hugbug reopened this Mar 20, 2016

hugbug added a commit that referenced this issue Mar 22, 2016

#136, #127: forcing extended paths with unrar (Windows)
Instead of using extended (extra long) path notation only if the path
is really long, we now always use extended notation since we don’t know
how long the paths inside archive are.

@hugbug hugbug closed this Mar 22, 2016

hugbug added a commit that referenced this issue Oct 9, 2017

#136, #127: forcing extended paths with unrar (Windows)
Instead of using extended (extra long) path notation only if the path
is really long, we now always use extended notation since we don’t know
how long the paths inside archive are.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment