directory permission error with make install in 3.0

[legerf]

BPO	4601
Nosy	@amauryfa, @vstinner, @tarekziade
Files	altinstall.log setup_chmod.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-07-02.23:57:32.161>
created_at = <Date 2008-12-08.21:27:52.807>
labels = ['library', 'performance']
title = 'directory permission error with make install in 3.0'
updated_at = <Date 2009-07-02.23:57:32.159>
user = 'https://bugs.python.org/legerf'

bugs.python.org fields:

activity = <Date 2009-07-02.23:57:32.159>
actor = 'amaury.forgeotdarc'
assignee = 'none'
closed = True
closed_date = <Date 2009-07-02.23:57:32.161>
closer = 'amaury.forgeotdarc'
components = ['Distutils']
creation = <Date 2008-12-08.21:27:52.807>
creator = 'legerf'
dependencies = []
files = ['12288', '12292']
hgrepos = []
issue_num = 4601
keywords = ['patch']
message_count = 11.0
messages = ['77344', '77351', '77352', '77354', '77356', '77364', '77404', '77406', '84129', '90025', '90037']
nosy_count = 4.0
nosy_names = ['amaury.forgeotdarc', 'vstinner', 'tarek', 'legerf']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue4601'
versions = ['Python 3.0', 'Python 3.1']

[legerf]

Under Debian/Lenny, with Python3.0.final install from the tarball, any
user can't import lib-dynload, launch IDLE ... but local root can do them.

When I login in root and run "chmod -R o+rx /usr/lib/python3.0/*", users
can use normaly python3.0.

Install detail : "configure --with-pydebug --with-doc-strings
--enable-shared --enable-profiling --enable-ipv6 --with-threads
--with-tsc --prefix=/usr ; make ; make test" and after "make altinstall

altinstall.log 2>&1". I join altinstall.log.

[legerf]

My root umask = 0027 (hardened system), so the altinstall/install script
don't manage umask parameter.

[vstinner]

I'm able to reproduce the bug with umask set to 0027:
drwxr-x--- 2 root root 3072 déc 8 22:59
(...)/lib/python3.1/lib-dynload

With umask 0077, it's:
drwx------ 2 root root 3072 déc 8 22:59
(...)/lib/python3.1/lib-dynload

The problem is specific to this directory.

[vstinner]

install() method of the install_lib command
(Lib/distutils/command/install_lib.py) calls self.copy_tree() which
calls copy_tree() from Lib/distutils/dir_util.py. By default, this
function preserve the modes (perserve_mode=1 by default). Or the build
created a directory with permissions "drwx------".

I wrote that only lib-dynload directory has a problem. But other files
are install with permissions -rw-------:

• .../lib/python?.?/*.{pyc,pyo}
• .../lib/python?.?/lib2to3/PatternGrammar*.pickle
• .../lib/python?.?/lib2to3/Grammar*.pickle
• .../lib/python?.?/lib-dynload/Python*.egg-info

[vstinner]

With Python trunk, the directory has the right permission (drwxr-xr-x)
whereas build/lib.linux(...) has permission
drwx------.

But the problem is still open for listed files
(.pyc, .pyo, .picle, .egg-info).

[vstinner]

Gotcha! os.path.walk() was replaced by os.walk() but walk() argument is
no more a callback because walk() is a generator! Here is a fix.

[amauryfa]

The patch is fine.

If it were me, I'd change os.walk to accept keyword-only arguments:
def walk(top, *, topdown=True, onerror=None, followlinks=False):
...

[vstinner]

Hum, there is not fixer for 2to3. But I might be hard to write such
fixer because walk() generates (dirpath, dirnames, filenames) instead
of (dirpath, filenames).

Python2 prototype:
os.path.walk(path, visit, arg) -> None
with visit: callback(arg, dirpath, filenames)

Python3 prototype:
os.walk(top[, topdown=True[, onerror=None[, followlinks=False]]]) ->
generator
with onerror: callback()
the generator produces (dirpath, dirnames, filenames)

Example:
os.path.walk('Lib/xml/', callback, 42)
can be replaced by
for dirpath, dirnames, filenames in os.walk('Lib/xml/'):
callback(42, dirpath, dirnames + filenames)

About the keyword only: +1 (or +2 :-)).

Index: Lib/os.py
===================================================================
--- Lib/os.py (révision 67652)
+++ Lib/os.py (copie de travail)
@@ -192,7 +192,7 @@

 __all__.extend(["makedirs", "removedirs", "renames"])

-def walk(top, topdown=True, onerror=None, followlinks=False):
+def walk(top, *, topdown=True, onerror=None, followlinks=False):
"""Directory tree generator.

 For each directory in the directory tree rooted at top (including 

top

[vstinner]

amaury> The patch is fine.

Cool :-) Anyone to commit the fix? Maybe, tarek?

amaury> If it were me, I'd change os.walk to accept
amaury> keyword-only arguments:
amaury> def walk(top, *, topdown=True, onerror=None,
amaury> followlinks=False):
amaury> ...

I like the idea but it should be proposed in a new issue :-) I
proposed an API change for open() (issue bpo-4121) but it was rejected by
Guido:

"(...) Beyond 3.0, I'm still rather reluctant -- I expect most users
will be wise and use keyword args anyway; I'm not sure what we buy by
forcing this. (...)"

But walk() is a different case than open().

[vstinner]

ping

[amauryfa]

Ok, fixed in r73788 and r73789.