"python -S" should be robust against e.g. "from site import addsitedir"

[carljm]

BPO	11591
Nosy	@brettcannon, @carljm, @merwok
Files	87df1d37c88e.diff ebe5760afa08.diff test_site-11591.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2011-03-23.04:09:31.883>
created_at = <Date 2011-03-17.19:47:00.007>
labels = ['type-bug', 'library']
title = '"python -S" should be robust against e.g. "from site import addsitedir"'
updated_at = <Date 2011-06-09.15:26:13.570>
user = 'https://github.com/carljm'

bugs.python.org fields:

activity = <Date 2011-06-09.15:26:13.570>
actor = 'eric.araujo'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2011-03-23.04:09:31.883>
closer = 'eric.araujo'
components = ['Library (Lib)']
creation = <Date 2011-03-17.19:47:00.007>
creator = 'carljm'
dependencies = []
files = ['21274', '21327', '22293']
hgrepos = ['4', '5']
issue_num = 11591
keywords = ['patch']
message_count = 16.0
messages = ['131281', '131286', '131294', '131322', '131536', '131580', '131681', '131706', '131709', '131748', '131761', '131848', '131850', '134568', '136968', '137988']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'carljm', 'eric.araujo', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11591'
versions = ['Python 3.3']

[carljm]

If python is run with the -S flag, that declares the intent of the user to not have site-specific additions to sys.path.

However, some code in that process may have a legitimate need for a function defined in site.py - for instance, addsitedir. But the act of importing site.py, as a side effect, adds the standard site-specific directories to sys.path.

python -S would be more useful and reliable if it prevented importing site from automatically making the sys.path additions. There is no loss of flexibility here, as user code could still explicitly call site.main() to achieve all of the current side-effects of "import site".

The fix is a one-liner, and is in the linked hg repository.

[merwok]

Thanks. Would you mind adding tests in test_site?

[carljm]

Adding a test is easier said than done. The behavior change here depends on python being run with -S. Currently test_site skips itself if the test suite is run with -S, and if I remove that skip it crashes under -S.

Options as I see it:

1. Declare this one-liner correct by inspection. It doesn't break any existing tests.

2. Add a new test file (test_no_site.py?) that only runs with -S and tests that importing something from site doesn't trigger sys.path additions. This seems like the most reasonable test, but I'm not sure how useful it is, since I doubt most people ever try running the test suite with -S.

3. Make the fix more complicated such that it uses an intermediary variable which can be mocked (unlike sys.flags.no_site, which is read-only), and then add a test which mocks this variable, temporarily removes "site" from sys.modules, tries importing it again, and checks whether main() is called. This creates a complex test which is highly coupled to the implementation in site.py, but would be run under normal conditions (without -S).

Which option do you prefer?

[merwok]

Fair argument. Brett is the author of recent changes in site, let him decide.

Brett: Would you agree to 1)?

[brettcannon]

This is what I get for trying to clean up site.py years ago. =)

I'm fine with the change as long as there is a very clear Misc/NEWS message that the semantics on import have changed (and obviously this is not backported).

[merwok]

I think this requires a note in NEWS but also in Doc/library/site.rst. Carl, would you like to make a new changeset with that edition in your repo? (It will also provide a test for the Create Patch button after.)

[carljm]

Added documentation to Doc/library/site.rst and Misc/NEWS.

[merwok]

Looks great, thank you. I think I’ll also add a docstring to main before committing, now that the function is publicly documented.

Did you have to manually click “Create Patch” to make roundup generate it? Did you try first to click on the button of the existing repo before adding a new repo entry? (Still learning how to use it, thanks for experimenting along :)

Brett: Thanks for the review. If you don’t comment negatively on the doc change, I will commit this.

(Side concern: the module does not define __all__, even though only 4 functions and 4 constants are officially documented. I’d like to define __all__, but the recentish huge thread on public/private APIs scared me.)

[carljm]

Did you have to manually click “Create Patch” to make roundup generate it?

Yes - the first time too.

Did you try first to click on the button of the existing repo before adding a new repo entry?

That would probably have worked fine. The "Remote hg repo" field was just empty when I made my latest comment, so I filled it in again. Wasn't sure if it would duplicate, or be smart enough to tell they were the same repo, or what. I guess it duplicated :/

[merwok]

The "Remote hg repo" field was just empty when I made my latest comment

Looks like this field is always empty: its goal is to add a repo, just like the File field is always empty unless you add a file. The existing files and repositories are however listed underneath the form, so your screen just probably did not go low enough for you to see that the repo was still listed.

[brettcannon]

Doc changes seem fine to me.

[python-dev]

New changeset a364719e400a by Éric Araujo in branch 'default':
Do not touch sys.path when site is imported and python was started with -S.
http://hg.python.org/cpython/rev/a364719e400a

[merwok]

Committed with small wording changes and more docs. Thank you, and good luck for cpythonv!

[python-dev]

New changeset 9a1ca0062950 by Éric Araujo in branch 'default':
Add versionchanged for a364719e400a (bpo-11591)
http://hg.python.org/cpython/rev/9a1ca0062950

[python-dev]

New changeset 6ee5443773cb by Éric Araujo in branch 'default':
Also add versionchanged directive to the function doc (bpo-11591)
http://hg.python.org/cpython/rev/6ee5443773cb

[merwok]

Now that site can be imported without side effects under -S, I think the tests could be updated: they don’t have to be all skipped under -S. See attached patch.