long file name support broken in windows

[mhammond]

BPO	542314
Nosy	@mhammond, @brettcannon

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2007-09-15.02:00:29.551>
created_at = <Date 2002-04-11.04:23:54.000>
labels = ['OS-windows']
title = 'long file name support broken in windows'
updated_at = <Date 2007-09-15.02:00:29.463>
user = 'https://github.com/mhammond'

bugs.python.org fields:

activity = <Date 2007-09-15.02:00:29.463>
actor = 'mhammond'
assignee = 'none'
closed = True
closed_date = <Date 2007-09-15.02:00:29.551>
closer = 'mhammond'
components = ['Windows']
creation = <Date 2002-04-11.04:23:54.000>
creator = 'mhammond'
dependencies = []
files = []
hgrepos = []
issue_num = 542314
keywords = []
message_count = 6.0
messages = ['10253', '10254', '10255', '10256', '55902', '55923']
nosy_count = 4.0
nosy_names = ['mhammond', 'brett.cannon', 'stunorton', 'titty']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue542314'
versions = []

[mhammond]

From c.l.py, thread ""

Peter D:
I'm using windows and trying to call os.path.getmtime
() after using os.path.walk... However, I'm choking
with "[Errno 38] Filename too long" on paths with len

~260

Adding Martin's reply in a new comment (so it is not
at the top of each and every mail I get on this bug :)

[mhammond]

Logged In: YES
user_id=14198

Martin v. Loewis's reply on c.l.py:

Since you are asking for a work-around: cd into one of the
more nested directories when the path gets longer
(os.chdir), and use relative paths from then on.

If you want to study how to solve the problem: the relevant
code snippet is in Modules/posixmodule.c

	/* the library call can blow up if the file name is 
too long! */
	if (pathlen > MAX_PATH) {
		PyMem_Free(pathfree);
		errno = ENAMETOOLONG;
		return posix_error();
	}

There might be different ways to approach this:

• challenge the correctness of the comment:
  • try to establish why the author of that code
    thought that the C library could blow up
  • analyse whether these arguments are still true
    with the current compiler version
    -or-
  • prove the argument wrong by analysing the source
    code of the C library
• then remove the constraint
  -or-
• use different API to achieve the same effect without
  suffering from the constraint.

I'm serious about these suggestions: users would appreciate
if this gets fixed somehow - apparently, the system allows
longer file names, and apparently, the system itself can
deal with that quite well. This
can be only true if the system either doesn't use its C
library, or if the C library does not have this restriction.

[stunorton]

Logged In: YES
user_id=1152606

I came across this suggestion while googling... and I would
have expected it to work, but this code:

import os

os.chdir("\\\\ussvs-file02
\\radpubs\\wip\\zStaging\\translation\\D10
\\previous_test\\cumulative\\Common\\Reference\\API\\Borland
.Eco.Persistence.Configuration\\classes\\PersistenceMapper
DefinitionCollection\\Methods")
for filename in os.listdir("."):
    print filename
    infile = file(filename)

gives me

C:\Documents and 
Settings\snorton\Desktop\h2build\buildtools>test.py
PersistenceMapperDefinitionCollection.AddRange.xml
PersistenceMapperDefinitionCollection.Assign.xml
PersistenceMapperDefinitionCollection.FindByName.xml
PersistenceMapperDefinitionCollection.NameExists.xml
PersistenceMapperDefinitionCollection.PersistenceMapperDefi
nitionCollection.xml
Traceback (most recent call last):
  File "C:\Documents and 
Settings\snorton\Desktop\h2build\buildtools\test.py", line 6, 
in ?
    infile = file(filename)
IOError: [Errno 2] No such file or 
directory: 'PersistenceMapperDefinitionCollection.Persistence
MapperDefinitionCollection.xml'

... funny that the file with the long path comes up in listdir()
but isn't found with file()...

[titty]

or if the c library is just insane:
from http://msdn2.microsoft.com/en-us/library/aa365247.aspx:

Maximum Path Length

In the Windows API, the maximum length for a path is MAX_PATH, which is defined as 260 characters. A path is structured in the following order: drive letter, colon, backslash, components separated by backslashes, and a null-terminating character, for example, the maximum path on the D drive is D:\<256 chars>NUL.

The Unicode versions of several functions permit a maximum path length of approximately 32,000 characters composed of components up to 255 characters in length. To specify that kind of path, use the "\\?\" prefix.

Note  The maximum path of 32,000 characters is approximate, because the "\\?\" prefix can be expanded to a longer string, and the expansion applies to the total length.

For example, "\\?\D:\<path>". To specify such a UNC path, use the "\\?\UNC\" prefix. For example, "\\?\UNC\<server>\<share>". These prefixes are not used as part of the path itself. They indicate that the path should be passed to the system with minimal modification, which means that you cannot use forward slashes to represent path separators, or a period to represent the current directory. Also, you cannot use the "\\?\" prefix with a relative path. Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so long that you cannot not append an 8.3 file name.

The shell and the file system may have different requirements. It is possible to create a path with the API that the shell UI cannot handle.

-----

the explicit checks for pathnames longer than 260 characters seem to be removed in python 2.5
and one can use the "magic prefix" to stat/create files with pathnames longer than 260 characters.

[brettcannon]

Can someone verify that the length issue has been fixed since Ptyhon 2.5?

[mhammond]

I can confirm the code in question was removed and that long filenames
are possible in 2.5. Eg:

import os
p = "\\\\?\\" + os.getcwdu()
for i in range(10):
    p = os.path.join(p, 'x' * 100)
    os.mkdir(p)
    os.stat(p)
print len(p)

I don't think Python should try and hide this madness, so I'm marking it
as closed.