Backup stops on too long filename not found under Windows #236
Comments
|
@pblahman it's a new problem, IMHO unrelated to the initial one, hence it has its place in a new issue. The filename is indeed next to or bigger than 260 characters, which is the default limit for path sizes on Windows. Is this an issue you did NOT have with rdiff-backup 1.2.8 or have you just started to use rdiff-backup this specific way? |
|
Eric,
Thank you for the response.
What follows this paragraph is what I first thought. Then I
followed/read your link to path sizes for Windows and discovered Enable
Long Paths in Windows 10, Version 1607, and Later
Since I am running Windows 10 Version 1909 on all my Windows systems,
I will pursue enabling Long Paths before I submit the request I considered,
below.
The file name length issue/problem is new to me. This is my first
"real attempt" to establish a Windows backup with rdiff-backup.
Since the backup process stops on this error...
My present opinion. I should/will enter a new issue and plead for an
option that allows the backup to continue instead of terminating the backup
process. I searched the "man page" and did not find an option to ignore
this type of error. If I missed it, please let me know.
Why? The length of the name (directories and file) has been created
(somehow) and now it will not backup. This problem exists in one "Users"
directory. The directory that contains the problem is not the last one
that needs backup.
…On Sat, Jan 11, 2020 at 9:57 AM Eric L. ***@***.***> wrote:
@pblahman <https://github.com/pblahman> it's a new problem, IMHO
unrelated to the initial one, hence it has its place in a new issue.
The filename is indeed next to or bigger than 260 characters, which is the
default limit for path sizes on Windows
<https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file>.
Is this an issue you did NOT have with rdiff-backup 1.2.8 or have you just
started to use rdiff-backup this specific way?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#236?email_source=notifications&email_token=AE2PV6DIQDHXQNFALL7DJUDQ5HT6NA5CNFSM4KFSRVGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIWE4CA#issuecomment-573328904>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE2PV6C4JFSCZFMXMVEO6CDQ5HT6NANCNFSM4KFSRVGA>
.
--
*Regards: Phillip Lahman*
|
|
Fine with me, just two comments:
|
|
The long path problem in Windows and Python can be easily solved:
|
|
"Easily solved" is relative, as the code is currently using bytes throughout the code and most of the time slashes instead of backslashes, so it's rather a major re-write IMHO. |
|
This ticket does not make sense for me. Rdiff-backup already have a way to
handle long file name.
What is the issue?
…On Mon., Feb. 24, 2020, 4:20 a.m. Eric L., ***@***.***> wrote:
"Easily solved" is relative, as the code is currently using bytes
throughout the code *and* most of the time slashes instead of
backslashes, so it's rather a major re-write IMHO.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#236?email_source=notifications&email_token=AAHA5IZZR3BMPF3DB27BJM3REOGN3A5CNFSM4KFSRVGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMXDBRI#issuecomment-590229701>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHA5I6SJ4O5ZZOEA4LY3IDREOGN3ANCNFSM4KFSRVGA>
.
|
When the accessed path and filename total length exceeds 260 characters, get "[WinError 206] The filename or extension is too long" exception, and the backup process stops. This causes incomplete backup repository and requires special attention when the backup runs unattended. As a result, it is currently not possible to backup long paths directly when the filename and path total length exceeds 260 characters with rdiff-backup. This is basically Win32 API limitation: https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation Unfortunately, enabling "Enable NTFS long paths" feature in Windows 10 does not resolve this issue in Python. The only workaround currently is to use Unicode and the \\?\ prefix in paths. In this case, the internal Windows automatic path canonization does not work, so only backslash can be used in a paths. This, according to ericzolf, requires a major rewrite. |
|
Could you provide the specific command line used to run the backup.
And I would appreciate if you could provide an error log with it with a
stack trace. Preferably running rdiff-backup with `-v 9` would be great.
…On Tue, Feb 25, 2020 at 9:14 AM elektrocad ***@***.***> wrote:
When the accessed path and filename total length exceeds 260 characters,
get "[WinError 206] The filename or extension is too long" exception, and
the backup process stops. This causes incomplete backup repository and
requires special attention when the backup runs unattended.
As a result, it is currently not possible to backup long paths directly
when the filename and path total length exceeds 260 characters with
rdiff-backup.
This is basically Win32 API limitation:
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation
Unfortunately, enabling "Enable NTFS long paths" feature in Windows 10
does not resolve this issue in Python. The only workaround currently is to
use Unicode and the \\?\ prefix in paths. In this case, the internal
Windows automatic path canonization does not work, so only backslash can be
used in a paths. This, according to ericzolf, requires a major rewrite.
This ticket does not make sense for me. Rdiff-backup already have a way to
handle long file name. What is the issue?
… <#m_5383364948161071841_>
On Mon., Feb. 24, 2020, 4:20 a.m. Eric L., *@*.***> wrote: "Easily
solved" is relative, as the code is currently using bytes throughout the
code *and* most of the time slashes instead of backslashes, so it's
rather a major re-write IMHO. — You are receiving this because you are
subscribed to this thread. Reply to this email directly, view it on GitHub <
#236 <#236>?email_source=notifications&email_token=AAHA5IZZR3BMPF3DB27BJM3REOGN3A5CNFSM4KFSRVGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMXDBRI#issuecomment-590229701>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHA5I6SJ4O5ZZOEA4LY3IDREOGN3ANCNFSM4KFSRVGA
.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#236?email_source=notifications&email_token=AAHA5IYICTK6ZNBEPY73WCDREURSZA5CNFSM4KFSRVGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM4DMFA#issuecomment-590886420>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHA5I3KWG6EN3DTTCGJQODREURSZANCNFSM4KFSRVGA>
.
--
Patrik Dufresne <ikus060@gmail.com>
|
I created some sample directory structures that produces an error. Please unpack the attachments one by one to root of drive, and run: long_path_no_error_but_incomplete_backup.zip |
|
I have exactly the same problem. The operating system is Windows 10 Enterprise 2019 LTSC x64. Unfortunately, this makes rdiff-backup unusable here, because not only does it not copy the file in question, but it also completely stops the whole backup process on the error. |
|
Same issue for me :-( |
|
Short notes while I work on this topic: Reboot then try again: I remembered too late that the registry key extends the path length but not the name length, which is still limited to 255 characters (I validated this by setting again the key to 0, rebooting and trying again). Next step, try this stuff with Python... |
|
The Python stuff: >>> path = "\\temp\\" + ("0123456789"*10+'\\')*2+"0123456789"*10
>>> with open(path) as fd: print(fd.read())
...
test
>>> path = b"\\temp\\" + (b"0123456789"*10+b'\\')*2+b"0123456789"*10
>>> path
b'\\temp\\0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789\\012345
6789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789\\0123456789012345678901
234567890123456789012345678901234567890123456789012345678901234567890123456789'
>>> with open(path) as fd: print(fd.read())
...
test
>>> len(path)
308
>>> path = b"\\temp\\" + (b"0123456789"*10+b'\\')*2+b"0123456789"*20
>>> with open(path,'w') as fd: print(fd.write('anothertest'))
...
11
>>> with open(path) as fd: print(fd.read())
...
anothertest
>>> path = b"..\\..\\temp\\" + (b"0123456789"*10+b'\\')*2+b"0123456789"*20
>>> with open(path) as fd: print(fd.read())
...
anothertest
>>> path = b"../../temp/" + (b"0123456789"*10+b'/')*2+b"0123456789"*20
>>> with open(path) as fd: print(fd.read())
...
anothertest
>>> path = b"/temp/" + (b"0123456789"*10+b'/')*2+b"0123456789"*20
>>> with open(path) as fd: print(fd.read())
...
anothertest OK, python doesn't seem to be the issue (even with slash, and relative paths, and with bytes)... Next step, rdiff-backup itself. |
|
|
|
I've simplified the testing a little bit: the file |
|
OK, I have a good and a bad news: the code of rdiff-backup doesn't seem to have an issue with long names, but it seems that the result compiled with PyInstaller has an issue with those: |
|
Issue should be solved after a rebuild with PyInstaller 4.2, assuming pyinstaller/pyinstaller#5424 is merged successfully. |
|
If possible, I would clearly state somewhere that the long paths are supported only in Windows 10 v1607, and also that registry or GPO modifications are required for this to work (see Enable Long Paths in Windows 10, Version 1607, and Later). |
|
This is wonderful news! Is there a way for me (a normal end-user) to get a hold of a test version somehow? |
|
Not immediately, our pipeline is currently blocked (clarifying with Travis) and PyInstaller 4.2 isn't yet out. It should be possible to create your own binary using the Vagrant setup under |
|
Just FYI, https://pyinstaller.readthedocs.io/en/stable/CHANGES.html 4.2 is out and has the requested fix to support long filenames. Follow the discussion on the mailing list. |
PyInstaller doesn't support pkgutil.iter_modules out of the box but there is a workaround: pyinstaller/pyinstaller#1905 FIX: support long paths under Windows 10 v1607 or later, once enabled in registry/GPO (see Windows README for details), closes #236
PyInstaller doesn't support pkgutil.iter_modules out of the box but there is a workaround: pyinstaller/pyinstaller#1905 FIX: support long paths under Windows 10 v1607 or later, once enabled in registry/GPO (see Windows README for details), closes #236
|
2022, checking in to say that a standard:
lol (psst, the issue is that it failed. no I don't have a stack trace to provide any insight. maybe next time it fails I will) |
|
Making fun of others doesn't help, especially as the question was valid, and constructively answered. It also seems that you didn't understand that the issue has been fixed but not yet released. |
|
I think that is a fair criticism. I knee-jerked 3 hours ago, unfairly I will follow the discussion in the mailing list |
The backup stopped prematurely. Below is the "end" of what was on
the screen. It appears that backup of "C:/Users/????/AppData" folder is
a problem.
The error about "filename or extension too long" appears to have
terminated the backup. 16 gigabytes of "stuff" is in the backup folder.
I will try a backup that excludes the "AppData" folders.
The text was updated successfully, but these errors were encountered: