test_ensure_mkdir_p: OSError: [Errno 36] File name too long

[jayvdb]

https://travis-ci.org/jayvdb/vistir/jobs/504095856 Python 3.6

_____________________________ test_ensure_mkdir_p ______________________________
    @given(legal_path_chars(), legal_path_chars())
>   @example(base_dir=u'0', subdir=u'\x80')
    @settings(suppress_health_check=(HealthCheck.filter_too_much,))
    def test_ensure_mkdir_p(base_dir, subdir):
tests/test_path.py:49: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_path.py:62: in test_ensure_mkdir_p
    target = join_with_dir(base_dir, subdir)
src/vistir/path.py:255: in decorated
    mkdir_p(path, mode=mode)
src/vistir/path.py:244: in mkdir_p
    os.makedirs(os.path.join(head, tail), mode)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
name = b'/tmp/tmp_nab54vs/0/\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\...80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80'
mode = 511, exist_ok = False
    def makedirs(name, mode=0o777, exist_ok=False):
        """makedirs(name [, mode=0o777][, exist_ok=False])
    
        Super-mkdir; create a leaf directory and all intermediate ones.  Works like
        mkdir, except that any intermediate path segment (not just the rightmost)
        will be created if it does not exist. If the target directory already
        exists, raise an OSError if exist_ok is False. Otherwise no exception is
        raised.  This is recursive.
    
        """
        head, tail = path.split(name)
        if not tail:
            head, tail = path.split(head)
        if head and tail and not path.exists(head):
            try:
                makedirs(head, mode, exist_ok)
            except FileExistsError:
                # Defeats race condition when another thread created the path
                pass
            cdir = curdir
            if isinstance(tail, bytes):
                cdir = bytes(curdir, 'ASCII')
            if tail == cdir:           # xxx/newdir/. exists if xxx/newdir exists
                return
        try:
>           mkdir(name, mode)
E           OSError: [Errno 36] File name too long: b'/tmp/tmp_nab54vs/0/\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80\xf0\x90\x80\x80'
../../../virtualenv/python3.6.3/lib/python3.6/os.py:220: OSError

[jayvdb]

Possibly a bug due to legal_path_chars() + /tmp/ + TemporaryNamedDirectory length being longer than the legal length?

[techalchemy]

Lol. Yeah it’s like 255 total characters on windows. Fixable for sure

[techalchemy]

Oh on Linux it’s 4096 though

[StefanBruens]

The assumption is insufficient, as the check limits the number of codepoints, but for most Linux filesystems, NAME_MAX (i.e. each path component) is 255 bytes.

e.g. len(u"Ä") == 1, but len(u"ä".encode('utf-8')) == 2

As can be seen above, legal_path_chars may generate the utf-8 codepoint \xf0\x90\x80\x80:
len(b'\xf0\x90\x80\x80'.decode('utf-8'))) == 1.

The max_size is 64, thus the returned path may decode to 4 * 64 bytes:
https://github.com/sarugaku/requirementslib/blob/master/tests/unit/strategies.py#L140