Silent mode for compileall

[takluyver]

BPO	21338
Nosy	@ezio-melotti, @merwok, @stevendaprano, @bitdancer, @takluyver, @berkerpeksag
Files	compileall_silent.patch issue21338.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/berkerpeksag'
closed_at = <Date 2014-10-15.08:11:56.244>
created_at = <Date 2014-04-23.23:29:32.230>
labels = ['easy', 'type-feature', 'library']
title = 'Silent mode for compileall'
updated_at = <Date 2014-10-15.08:11:56.243>
user = 'https://github.com/takluyver'

bugs.python.org fields:

activity = <Date 2014-10-15.08:11:56.243>
actor = 'berker.peksag'
assignee = 'berker.peksag'
closed = True
closed_date = <Date 2014-10-15.08:11:56.244>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2014-04-23.23:29:32.230>
creator = 'takluyver'
dependencies = []
files = ['35014', '36808']
hgrepos = []
issue_num = 21338
keywords = ['patch', 'easy']
message_count = 12.0
messages = ['217100', '217102', '217106', '217107', '217108', '217109', '217111', '217167', '227800', '228534', '229406', '229407']
nosy_count = 7.0
nosy_names = ['ezio.melotti', 'eric.araujo', 'steven.daprano', 'r.david.murray', 'python-dev', 'takluyver', 'berker.peksag']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21338'
versions = ['Python 3.5']

[takluyver]

The compileall module's command line interface has a -q (quiet) flag which suppresses most of the output, but it still prints error messages. I'd like an entirely silent mode with no output.

My use case is byte-compiling Python files as part of a graphical installer. I do this by running:

py -${PY_QUALIFIER} -m compileall -q "$INSTDIR\pkgs"

I'd like to be able to use pyw so a terminal doesn't pop up, but if I do that, it exits early. I think this is due to the issue with stdout/stderr buffers filling up on pythonw.

[ezio-melotti]

This seems a reasonable request. Do you want to propose a patch?

[takluyver]

Patch attached.

This works by making the -q flag countable, so you pass -qq to suppress all output. In the Python API, the quiet parameter has become an integer, so passing 2 is equivalent to -qq. This should be fully backwards compatible with passing True and False, which are equivalent to 1 and 0.

[stevendaprano]

Can't you just re-direct stdout or stderr? I'm sure that works even on Windows. E.g.

py -${PY_QUALIFIER} -m compileall -q "$INSTDIR\pkgs" 2> null

Is there a reason you cannot do this? I think that adding functionality to compileall to duplicate what your OS already provides should be a last resort.

I also don't understand what you mean by "it exits early". What exits early? Surely python doesn't exit until compileall finishes running. If it does, that's a bug.

[takluyver]

Sorry, I somehow attached an old version of the patch. This one should be correct.

Steven: Redirection relies on a shell - the native 'run a process' interface that the installer uses can't do that. There are ways to work around this, but I think this approach makes sense.

It exits early because stdout and stderr don't go anywhere, so once either of them has filled up a fairly small buffer, attempts to write to it throw an error. I'd love to see that fixed, but it's been known about for years (see bpo-706263), and it seems unlikely to change any time soon.

[takluyver]

Gah, still wrong. Trying again.

[takluyver]

In fact, I will probably end up working around this anyway, because I'll have to support versions of Python without this fix for some time. So I don't feel strongly that it needs to go in, but I will do any revisions or changes requested if people think it would be useful.

[merwok]

Patch looks to me comprehensive and backward-compatible. Thanks Thomas!

[bitdancer]

Berker: I had some review comments pending on the docs, but reitveld isn't letting me publish them for some reason (it says the patch set doesn't exist).

The comments are: the method docs should probably say "False or 0 (the default)" (and all the numbers should be marked up as code), and the versionchanged I think should say that the option was changed to a multilevel value.

Otherwise the patch looks good to me, too.

[berkerpeksag]

Here's an updated patch. Thanks David.

[python-dev]

New changeset 34436ae65523 by Berker Peksag in branch 'default':
Issue bpo-21338: Add silent mode for compileall.
https://hg.python.org/cpython/rev/34436ae65523

[berkerpeksag]

Thanks for the patch, Thomas.