test_compileall fails when run by unprivileged user on installed Python

[zware]

BPO	25099
Nosy	@brettcannon, @birkenfeld, @pfmoore, @ncoghlan, @tjguk, @benjaminp, @ericsnowcurrently, @zware, @serhiy-storchaka, @zooba
Files	issue25099.diff issue25099.diff issue25099.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/brettcannon'
closed_at = <Date 2015-10-14.15:57:50.893>
created_at = <Date 2015-09-14.04:32:53.028>
labels = ['type-bug', 'tests', 'OS-windows']
title = 'test_compileall fails when run by unprivileged user on installed Python'
updated_at = <Date 2015-10-14.15:57:50.892>
user = 'https://github.com/zware'

bugs.python.org fields:

activity = <Date 2015-10-14.15:57:50.892>
actor = 'brett.cannon'
assignee = 'brett.cannon'
closed = True
closed_date = <Date 2015-10-14.15:57:50.893>
closer = 'brett.cannon'
components = ['Tests', 'Windows']
creation = <Date 2015-09-14.04:32:53.028>
creator = 'zach.ware'
dependencies = []
files = ['40580', '40734', '40736']
hgrepos = []
issue_num = 25099
keywords = ['patch']
message_count = 15.0
messages = ['250614', '250633', '250690', '250694', '250697', '251609', '251716', '252640', '252658', '252663', '252668', '252669', '252695', '252723', '252732']
nosy_count = 11.0
nosy_names = ['brett.cannon', 'georg.brandl', 'paul.moore', 'ncoghlan', 'tim.golden', 'benjamin.peterson', 'python-dev', 'eric.snow', 'zach.ware', 'serhiy.storchaka', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25099'
versions = ['Python 3.5', 'Python 3.6']

[zware]

======================================================================
FAIL: test_no_args_respects_force_flag (test.test_compileall.CommandLineTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "C:\Program Files\Python 3.5\lib\test\test_compileall.py", line 229, in test_no_args_respects_force_flag
    self.assertRunOK('-f', PYTHONPATH=self.directory)
  File "C:\Program Files\Python 3.5\lib\test\test_compileall.py", line 179, in assertRunOK
    *self._get_run_args(args), **env_vars)
  File "C:\Program Files\Python 3.5\lib\test\support\script_helper.py", line 135, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "C:\Program Files\Python 3.5\lib\test\support\script_helper.py", line 121, in _assert_python
    err))
AssertionError: Process return code is 1
command line: ['C:\\Program Files\\Python 3.5\\python.exe', '-X', 'faulthandler', '-S', '-m', 'compileall', '-f']

stdout:
---
(... truncated stdout ...)rror: [Errno 13] Permission denied: 'C:\\Program Files\\Python 3.5\\lib\\pycache\\sre_compile.cpython-35.pyc.978608563184'
Compiling 'C:\\Program Files\\Python 3.5\\lib\\sre_constants.py'...

<output continues with many more lines like the above two>

No such problem on installed 3.4.3.

[serhiy-storchaka]

The failure can be reproduced on Linux if make the Lib/__pycache__/ directory non-writable.

[zooba]

IIRC there's an existing issue for this. (Or it may have just been mentioned in a "things that don't work when you're not root" list on an issue.)

[serhiy-storchaka]

Existing bpo-21264 reports about failure of different test. It may be the same issue or different but related issues.

[brettcannon]

It sounds like some tests just need to have a decorator that blocks execution if the relevant __pycache__ isn't writable.

[brettcannon]

Here is a patch that should work. Can someone who has had the problem test it? It works in my limited testing but I want to make sure this solves it before I commit it.

[serhiy-storchaka]

Added comments on Rietveld.

[brettcannon]

Here is a new patch that addresses Serhiy's comments.

[serhiy-storchaka]

Added new comments.

[brettcannon]

I have addressed Serhiy's comments again.

[python-dev]

New changeset 34bbd537b3e6 by Brett Cannon in branch '3.5':
Issue bpo-25099: Skip relevant tests in test_compileall when an entry on
https://hg.python.org/cpython/rev/34bbd537b3e6

New changeset 21f3a92e0c6d by Brett Cannon in branch 'default':
Merge for issue bpo-25099
https://hg.python.org/cpython/rev/21f3a92e0c6d

[brettcannon]

Thanks to Serhiy for all of the reviews!

[serhiy-storchaka]

Do you want to backport the fix to 3.4 (and 2.7?)? bpo-21264 was reported for 3.4.

[brettcannon]

I feel I have already spent way too much time on this issue, but if you want to backport even further then feel free to.

[python-dev]

New changeset fc0a7aa7ae61 by Brett Cannon in branch '3.4':
Issue bpo-25099: Skip relevant tests in test_compileall when an entry on
https://hg.python.org/cpython/rev/fc0a7aa7ae61