compiler package: "global a; a=5"

[arigo]

BPO	1251748
Nosy	@arigo, @nascheme, @pitrou
Files	comp-scope.diff: patch Support "bpo-" in Misc/NEWS #1 (mwh) compiler2.diff: 2.4/Lib/compiler package diff from PyPy (Rename README to README.rst and enhance formatting #2)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-03-24.10:22:38.449>
created_at = <Date 2005-08-04.10:28:57.000>
labels = ['interpreter-core']
title = 'compiler package: "global a; a=5"'
updated_at = <Date 2008-03-25.19:27:27.986>
user = 'https://github.com/arigo'

bugs.python.org fields:

activity = <Date 2008-03-25.19:27:27.986>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2008-03-24.10:22:38.449>
closer = 'arigo'
components = ['Interpreter Core']
creation = <Date 2005-08-04.10:28:57.000>
creator = 'arigo'
dependencies = []
files = ['6747', '6748']
hgrepos = []
issue_num = 1251748
keywords = ['patch']
message_count = 12.0
messages = ['48636', '48637', '48638', '48639', '64368', '64409', '64411', '64452', '64470', '64474', '64496', '64507']
nosy_count = 3.0
nosy_names = ['arigo', 'nascheme', 'pitrou']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue1251748'
versions = ['Python 2.6', 'Python 2.5']

[arigo]

The stdlib compiler package produces wrong code for
"global a; a=5". It produces a STORE_NAME instead of a
STORE_GLOBAL.

Quick patch from Michael attached.

[arigo]

Logged In: YES
user_id=4771

The compiler package contains several bugs along the lines
of the one reported here. It doesn't seem to be that much
used; with PyPy it is probably the first time that this
package is tested in-depth, as we are trying to pass
CPython's own test suite by using only this compiler
package. We had to make a few involved fixes.

Attached is a diff against the 2.4 maintenance head of
CPython (we based our changes on 2.4.1). If there is
interest, I can extract individual parts of the big diff
and/or port them to 2.5 CVS HEAD. For now I will only give
a quick overview of the diff.

• the more involved problem was related to scoping of
  variables. This turned out to be more difficult to fix than
  mwh's patch (included in this tracker). (Disclaimer:
  Michael never pretended his patch was more than a quick
  hack.) For reference, see the following code snippets:
  http://codespeak.net/svn/pypy/dist/pypy/interpreter/test/test_syntax.py

• we try to always raise SyntaxErrors with line number
  information. (Note sure if this has been done completely yet.)

• pyassem.py has some heavy changes that I cannot comment on
  (don't know about them). Other changes there are related to
  scoping flags.

• a number of situations should raise a SyntaxError, but
  were silently accepted (and generally produce weird bytecodes).

• fixed docstring handling. Previously, in interactive
  mode, entering "hello" produced no result at all -- because
  the string was interpreted as a docstring! Also, a genuine
  docstring on the same line as a statement -- "docstring";
  x=5 -- would result in the statement being ignored.

• we can pass future flags from outside now.

• lambda functions didn't get the official name <lambda>.
  (Yeah, I know it's a detail, but a CPython test fails
  because of that.)

• added raising of SyntaxWarnings.

• the st->ast transformer 'single' mode now returns a Module
  AST node instead of a Stmt. This matches more closely what
  the rest of the package expects.

• some minor changes in transformer.py are probably not
  interesting to CPython (e.g. changing some () into []).
  Their goal was to make transformer.py statically typeable
  for PyPy.

[nascheme]

Logged In: YES
user_id=35752

These fixes should definitely go into 2.5, IMHO. If you
need help porting the patch, let me know.

[arigo]

Logged In: YES
user_id=4771

The patch appears to apply as it is on the CVS HEAD, with at most offsets of 10 lines, and test_compiler.py passes afterwards. I suppose that someone (else than me) needs to give the patch a quick review anyway, so feel free to consider it as a 2.4 + 2.5 patch.

[pitrou]

Armin, if you still care about the compiler package, could you (or some
other pypy coder) take a look at bpo-2459? As part of the patch, it
rewrites the flow graph block ordering algorithm in a cleaner way (IMHO).

[arigo]

Sorry, I don't think many PyPy coders care about microoptimizations in
the bytecode. To be honest we've stopped worrying about the stdlib
compiler package because no one seemed to care. I'm closing the present
patch as incomplete and outdated; if someone is interested in reviving
the stdlib compiler package, my offer to extract from PyPy an up-to-date
diff for the compiler package still stands.

[pitrou]

Armin, this is not about micro-optimizations per se, but about fixing
some parts of the compiler package. To make it easier, I've created a
separate entry for the fixing part in bpo-2472 :)

This was the first time I was using the compiler package and I think it
can be useful in order to experiment with bytecode ideas (of course you
probably prefer to do that with PyPy). That's why I tried to fix what
was failing for me.

[nascheme]

On Sun, Mar 23, 2008 at 04:17:12PM +0000, Antoine Pitrou wrote:

Armin, if you still care about the compiler package, could you (or some
other pypy coder) take a look at bpo-2459? As part of the patch, it
rewrites the flow graph block ordering algorithm in a cleaner way (IMHO).

I'm interested to update the compiler package but I'm short of time
at the moment. If you want to assign bugs and patches to me, I will
try to get to them.

Neil

[arigo]

Thanks Antoine for the clarification.

The situation is that by now PyPy has found many many more bugs in
trying to use the compiler package to run the whole stdlib and
real-world applications. What I can propose is to extract what we have
got and put it back into CPython's stdlib, keeping the current
documented API. This will require a little bit of work (e.g. first
finishing to add all new 2.5 and 2.6 features into PyPy's compiler)
but IMHO it's more worthwhile than going through the process of
rediscovering and fixing all the current bugs one by one.

[pitrou]

Armin, it would be nice to merge back your work indeed. If you submit a
patch I can take a look at it.

[nascheme]

On Tue, Mar 25, 2008 at 09:18:38AM +0000, Armin Rigo wrote:

The situation is that by now PyPy has found many many more bugs in
trying to use the compiler package to run the whole stdlib and
real-world applications. What I can propose is to extract what we have
got and put it back into CPython's stdlib, keeping the current
documented API.

I'm not sure it can go back into the stdlib since there doesn't seem
to be enough energy available to keep it up-to-date with the Python
release schedule. I would like to make it available as an add-on
package.

This will require a little bit of work (e.g. first
finishing to add all new 2.5 and 2.6 features into PyPy's compiler)
but IMHO it's more worthwhile than going through the process of
rediscovering and fixing all the current bugs one by one.

Indeed it would make no sense to redo that work. Can the version in
the PyPy tree be used as a direct replacement for the stdlib version
or does it need some changes? You had mentioned being able to
produce a patch against the stdlib version. Is that easy or would
it be better just to take the PyPy version and package it up.

Neil

[pitrou]

It seems to me that releasing it as a third-party package would be a
little contradictory, since the compiler package is supposed to follow
the CPython core (compile.c etc.) evolutions.

In any case, whatever the decision (stdlib package or separate
releases), I think it would be nice if there was a slightly more
comprehensive test suite. :)

Something like running some selected non-controversial parts of the test
suite compiled with the compiler package (like PyPy does apparently)
could be a good start.