os.makedirs('dir1/dir2', 0) always fails

[serhiy-storchaka]

BPO	19930
Nosy	@loewis, @gpshead, @pitrou, @vadmium, @serhiy-storchaka, @vajrasky
PRs	bpo-19930: The mode argument of os.makedirs() no longer affects the file #799
Files	os_makedirs_mode.patch os_makedirs_mode_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2017-03-24.11:28:49.908>
created_at = <Date 2013-12-08.11:09:05.817>
labels = ['3.7', 'type-bug', 'library']
title = "os.makedirs('dir1/dir2', 0) always fails"
updated_at = <Date 2020-11-16.09:55:44.064>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2020-11-16.09:55:44.064>
actor = 'gregory.p.smith'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2017-03-24.11:28:49.908>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2013-12-08.11:09:05.817>
creator = 'serhiy.storchaka'
dependencies = []
files = ['33043', '33047']
hgrepos = []
issue_num = 19930
keywords = ['patch']
message_count = 7.0
messages = ['205543', '205570', '205581', '268093', '268101', '290082', '381083']
nosy_count = 6.0
nosy_names = ['loewis', 'gregory.p.smith', 'pitrou', 'martin.panter', 'serhiy.storchaka', 'vajrasky']
pr_nums = ['799']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19930'
versions = ['Python 3.7']

[serhiy-storchaka]

os.makedirs() can't create a directory with cleared write or list permission bits for owner when parent directories aren't created. This is because for parent directories same mode is used as for final directory.

Note that the mkdir utility creates parent directories with default mode (0o777 & ~umask).

$ mkdir -p -m 0 t1/t2/t3
$ ls -l -d t1 t1/t2 t1/t2/t3
drwxrwxr-x 3 serhiy serhiy 4096 Dec  7 22:30 t1/
drwxrwxr-x 3 serhiy serhiy 4096 Dec  7 22:30 t1/t2/
d--------- 2 serhiy serhiy 4096 Dec  7 22:30 t1/t2/t3/

The proposed patch emulates the mkdir utility.

See also bpo-19921.

[vajrasky]

Fails on Windows Vista.

======================================================================
FAIL: test_mode (main.MakedirTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "Lib\test\test_os.py", line 907, in test_mode
    self.assertEqual(stat.S_IMODE(os.stat(parent).st_mode), 0o775)
AssertionError: 511 != 509

---

Ran 157 tests in 1.865s

FAILED (failures=1, skipped=61)
Traceback (most recent call last):
  File "Lib\test\test_os.py", line 2511, in <module>
    test_main()
  File "C:\Users\vajrasky\Code\cpython\lib\test\support\__init__.py", line 1831,
 in decorator
    return func(*args)
  File "Lib\test\test_os.py", line 2507, in test_main
    FDInheritanceTests,
  File "C:\Users\vajrasky\Code\cpython\lib\test\support\__init__.py", line 1719,
 in run_unittest
    _run_suite(suite)
  File "C:\Users\vajrasky\Code\cpython\lib\test\support\__init__.py", line 1694,
 in _run_suite
    raise TestFailed(err)
test.support.TestFailed: Traceback (most recent call last):
  File "Lib\test\test_os.py", line 907, in test_mode
    self.assertEqual(stat.S_IMODE(os.stat(parent).st_mode), 0o775)
AssertionError: 511 != 509

The permission of directory on Windows no matter what mode you give or umask you give to support.temp_umask, is always 0o777 (or 511). I think this test does not make sense in Windows.

>>> os.mkdir('cutecat', 0o555)
>>> os.mkdir('cutecat2', 0o777)
>>> os.stat('cutecat')
os.stat_result(st_mode=16895, st_ino=3940649674207852, st_dev=3960548439, st_nli
nk=1, st_uid=0, st_gid=0, st_size=0, st_atime=1386517061, st_mtime=1386517061, s
t_ctime=1386517061)
>>> os.stat('cutecat2')
os.stat_result(st_mode=16895, st_ino=5066549581050708, st_dev=3960548439, st_nli
nk=1, st_uid=0, st_gid=0, st_size=0, st_atime=1386517067, st_mtime=1386517067, s
t_ctime=1386517067)

Either that, or I am missing something.

[serhiy-storchaka]

Thank you Vajrasky. Now this check is skipped on Windows.

[vadmium]

I’ve never considered this sort of scenario properly, so I don’t know if leaving the default mode for the parent directories is the best way or not. The obvious but more complicated alternative would be to call chmod() on all the new directories in a second step.

Anyway, if we are going to make any change, it should be documented. And I would say the current patch changes behaviour, so is probably not applicable as a bug fix.

[serhiy-storchaka]

The obvious but more complicated alternative would be to call chmod() on all the new directories in a second step.

This is dangerous, because if you create read-only or unlistable directory, you couldn't remove it without changing the permission of parent directory. shutil.rmtree() would fail.

[serhiy-storchaka]

New changeset e304e33 by Serhiy Storchaka in branch 'master':
bpo-19930: The mode argument of os.makedirs() no longer affects the file (#799)
e304e33

[gpshead]

This API change was not strictly a bugfix, it removed a feature existing code was relying on.

https://bugs.python.org/issue42367 opened to reconcile the two.