Improve -O docs

[fijall]

BPO	17232
Nosy	@brettcannon, @terryjreedy, @ncoghlan, @pitrou, @vstinner, @ezio-melotti, @berkerpeksag, @vadmium, @csabella, @miss-islington
PRs	bpo-17232: Clarify docs for -O and -OO command line options #5839 [3.7] bpo-17232: Clarify docs for -O and -OO command line options (GH-5839) #5867 [3.6] bpo-17232: Clarify docs for -O and -OO command line options (GH-5839) #5868
Files	issue17232.diff 17232-O.diff: Add :const:, tweek -OO

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2018-02-25.04:08:45.856>
created_at = <Date 2013-02-19.06:25:01.174>
labels = ['3.8', 'type-bug', '3.7', 'docs']
title = 'Improve -O docs'
updated_at = <Date 2018-02-25.04:08:45.854>
user = 'https://bugs.python.org/fijall'

bugs.python.org fields:

activity = <Date 2018-02-25.04:08:45.854>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2018-02-25.04:08:45.856>
closer = 'terry.reedy'
components = ['Documentation']
creation = <Date 2013-02-19.06:25:01.174>
creator = 'fijall'
dependencies = []
files = ['29165', '29392']
hgrepos = []
issue_num = 17232
keywords = ['patch']
message_count = 24.0
messages = ['182365', '182373', '182377', '182380', '182680', '182681', '182682', '182729', '182738', '182742', '184047', '184048', '184049', '184051', '184070', '184141', '195475', '312597', '312600', '312611', '312658', '312771', '312772', '312773']
nosy_count = 15.0
nosy_names = ['brett.cannon', 'terry.reedy', 'ncoghlan', 'pitrou', 'vstinner', 'ezio.melotti', 'Arfrever', 'eli.bendersky', 'docs@python', 'fijall', 'Ramchandra Apte', 'berker.peksag', 'martin.panter', 'cheryl.sabella', 'miss-islington']
pr_nums = ['5839', '5867', '5868']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17232'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[fijall]

This is what the current documentation says:

-O
Turn on basic optimizations. This changes the filename extension for compiled (bytecode) files from .pyc to .pyo. See also PYTHONOPTIMIZE.

-OO
Discard docstrings in addition to the -O optimizations.

As far as I know, the only "optimization" that's done is removal of __debug__ sections and assert statements and has been like this for years. Maybe it should say so "-O does not do any optimizations, only removes assert statement" or so.

[RamchandraApte]

It should also add that in the future, more optimizations may be added i.e. a peephole optimizer, etc.

[fijall]

There were not for at least 10 years. I would also be the first one to strongly object adding optimizations only under -O, because that already changes semantics.

[ncoghlan]

Ramchandra, as it turns out, if we deem an optimization semantically safe, we do it without -O, it we deem it unsafe, we don't do it at all.

Thus, the real effect is to remove assert statements and optimise code as if "__debug__" was replaced by a literal zero (effectively).

So a more meaningful description would be:

-O
Removes assert statements and any code conditional on the value of __debug__. This changes the filename extension for compiled (bytecode) files from .pyc to .pyo. See also PYTHONOPTIMIZE.

[terryjreedy]

I agree that we should document exactly what is now. Patch replaces first sentence with Nick's. It is against 3.4, but should be identical for all versions. Maciej, thanks for reminding us to finally fix this.

[pitrou]

Should probably be "Remove", not "Removes" (we use infinitives to describe the actions undertaken by a command / option / method ...).

[terryjreedy]

Perhaps '__debug__' need markup, but if so, I don't know how.
And I agree with Antoine.

[ncoghlan]

+1 for Remove instead of Removes

For the online docs, :const:`__debug__` should work (resolving to http://docs.python.org/3/library/constants.html#__debug__, which is currently described using some slightly brain-bending phrasing)

We should also tweak the output of "python -h" (the help string is in ./Modules/main.c)

[elibendersky]

+1, I've been bothered by this description of "optimization" for a long time.

Terry's patch LGTM

[fijall]

Also IMO -OO should stop talking about optimizations. Maybe "Do what -O does and discard docstrings"?

[terryjreedy]

I added :const: and tweaked -OO entry and -h startup display.
Tested new html and python_d -h. Any other comments before I apply?

[pitrou]

There's a typo in your patch:

+-O : remove assert and __debug__-dependent statements; change .py\n\
+ to .pyo; also PYTHONOPTIMIZE=x\n\

should say ".pyc", not ".py".

[pitrou]

Also, in 3.2 and higher I'm not sure there's a point in mentioning pyc/pyo files; they're all shelved in __pycache__ now.

[terryjreedy]

I corrected my copy of the .diff.

Since this issue is so far focused on removing the false optimize claim, hiding .pyx info is a new sub-issue. I will follow whatever the consensus is, but since this is a cpython-specific doc and help, I would prefer to give complete info. In fact, I would like to add 'stored in __pycache__' or even 'hidden away in __pycache__', the latter to suggest that most people should generally forget about them. On Windows, _xxx files like __pycache__ appear in both Command Prompt dir and Explorer file listings, so beginners need to known that __cache__ is both normal and ignorable.

[ncoghlan]

On Tue, Mar 12, 2013 at 1:18 PM, Antoine Pitrou <report@bugs.python.org> wrote:

Antoine Pitrou added the comment:

Also, in 3.2 and higher I'm not sure there's a point in mentioning pyc/pyo files; they're all shelved in __pycache__ now.

It still makes a difference for bytecode-only files, which aren't in __pycache__

Cheers,
Nick.

[ezio-melotti]

I left a review on rietveld.

[ezio-melotti]

Terry, do you want to update your patch?

[csabella]

Should I make a pull request for Terry's last patch?

[terryjreedy]

Cheryl: yes, with changes in responses to Ezio's review.

Nick or Antoine: has there been any change to meaning of -O or -OO that I am not remembering?

[ncoghlan]

We don't change the extension on optimised pyc files any more, we add an optimisation marker to the name without changing the file extension: https://www.python.org/dev/peps/pep-0488/

(This means -O and -OO don't tread on each other any more)

[csabella]

I've made a PR and tried to integrate the changes from PEP-488.

[terryjreedy]

New changeset 186b606 by Terry Jan Reedy (Cheryl Sabella) in branch 'master':
bpo-17232: Clarify docs for -O and -OO command line options (bpo-5839)
186b606

[miss-islington]

New changeset b5655f3 by Miss Islington (bot) in branch '3.7':
bpo-17232: Clarify docs for -O and -OO command line options (GH-5839)
b5655f3

[miss-islington]

New changeset fc9471a by Miss Islington (bot) in branch '3.6':
bpo-17232: Clarify docs for -O and -OO command line options (GH-5839)
fc9471a