Reorganize OPy/byterun testdata.

New gold/ directory of Python scripts, which can be compared like this:

    $ ./test.sh gold

Also, opy/doc/opcodes.md now has a survey of different bytecodes.

opy/README.md

@@ -62,14 +62,14 @@ Run Oil unit tests, compiled with OPy, under byterun (OPyPy):

    $ ./test.sh oil-unit-byterun   # Run Oil unit tests, compiled with OPy, under CPython

Gold tests in gold/:

Another way I test it like this:
    $ ./test.sh gold

    $ testdata/regex_compile.py  # run with CPython
    $ ../bin/opyc run testdata/regex_compile.py

(TODO: these should be gold tests)
They can be run manually like this:

    $ gold/regex_compile.py  # run with CPython
    $ ../bin/opyc run gold/regex_compile.py

Demo: speed difference between OSH under CPython and OSH under byterun:

opy/opcodes.md → opy/doc/opcodes.md

@@ -1,13 +1,112 @@
Notes on VM Opcodes
===================

2018 Bytecode assessement
-------------------------

### Same

Stack Management Bytecodes that are the same:

- `POP_TOP`
- `DUP_TOP_TWO`
- `ROT_{TWO,THREE,FOUR}`

(We could switch to a register VM, but that would be an completely orthogonal
change.)

Control Flow Bytecodes that are the same:

- `SETUP_LOOP`
- `SETUP_WITH`
  - `WITH_CLEANUP`
- `SETUP_{EXCEPT,FINALLY}`
  - `END_FINALLY`
- `POP_BLOCK`
- `JUMP_{FORWARD,ABSOLUTE,...}`
- `POP_JUMP_*`
- `{BREAK,CONTINUE}_LOOP`
- `RETURN_VALUE`
- `RAISE_VARARGS` -- although it's not as general
- `GET_ITER`

Data structure bytecodes that are likely the same:

- `{LIST,SET,MAP}_ADD`
- `STORE_MAP`
- `BUILD_{TUPLE,LIST,SET,MAP}`
- `SLICE_*`

At least in the beginning they are the same.  Later we might have specialized
data structures, e.g. `Array<Str>`, which is extremely common in shell.

### Changed

Load / Store bytescodes that will take indices instead of names:

- `{LOAD,STORE}_NAME`
  - fast variants go away: `{LOAD,STORE}_FAST`
- `{LOAD,STORE}_GLOBAL`
- `{LOAD,STORE}_ATTR` - for object members


Highly Changed based on language semantics

- `CALL_FUNCTION_*` -- Instead of four variants, we may just have one more
  static kind.
  - It will support `f(msg, *args)` and `f(*args, **kwargs)`, but maybe not
    much else?

Bytecodes that can be type-specialized:

- `BINARY_*`
- `UNARY_*`
- `COMPARE_OP` -- or maybe just don't allow nonsensical comparisons

Maybe type-specialized:

- `FOR_ITER` -- iterating items in a list, iterating characters in a string
  could be compiled statically.  In other words, the iterator protocol isn't
  quite necessary.

### Removed

Dynamic bytecodes that will go away, because names are statically resolved:

- `BUILD_CLASS`
- `MAKE_FUNCTION`
- `IMPORT_{NAME,STAR}`
- maybe: `MAKE_CLOSURE`: this should be done statically?  Closures and classes
  should be the same?  It's like calling a constructor.

Other Removed:

- `DELETE_NAME`: Namesapces are static
- Might be unnecessary for our purposes: `YIELD_FROM`
- `EXEC_STMT`: I want a different interface to the compiler, for
  metaprogramming purposes.

Deprecated:

- `PRINT_*` -- this should just be a normal function call

### Additions

- Bytecodes for ASDL structures?
- Bytecodes for shell?
- For parsing VM?

2017
----

This is an elaboration on:

https://docs.python.org/2/library/dis.html

I copy the descriptions and add my notes, based on what I'm working on.



`SETUP_LOOP(delta)`

Pushes a block for a loop onto the block stack. The block spans from the

opy/tests/genexpr.py → opy/gold/genexpr.py

@@ -1,4 +1,5 @@
#!/usr/bin/env python
from __future__ import print_function
"""
Test for generator expressions.
"""

opy/tests/genexpr_simple.py → opy/gold/genexpr_simple.py

@@ -1,4 +1,5 @@
#!/usr/bin/env python
from __future__ import print_function
"""
Simpler test for generator expressions.
"""

opy/testdata/hello_py2.py → opy/gold/hello_py2.py

@@ -1,3 +1,4 @@
#!/usr/bin/env python
from __future__ import print_function
import sys
print("hi from Python 2", file=sys.stderr)

opy/testdata/hello_py3.py → opy/gold/hello_py3.py

@@ -1 +1,2 @@
#!/usr/bin/env python
print("hi from Python 3")

opy/test.sh

@@ -212,4 +212,19 @@ opy-speed-test() {
  time opyc-run _tmp/speed_main.pyc $n
}

gold() {
  for script in gold/*.py; do

    $script > _tmp/gold-cpython.txt 2>&1
    ../bin/opyc run $script > _tmp/gold-opy-byterun.txt 2>&1

    if diff -u _tmp/gold-{cpython,opy-byterun}.txt; then
      echo "OK $script"
    else
      echo "FAIL $script"
      return 1
    fi
  done
}

"$@"

opy/testdata/hello_py2_print.py

@@ -0,0 +1,4 @@
#!/usr/bin/python

import sys
print >>sys.stderr, 'hi'

opy/testdata/tuple_args.py

@@ -0,0 +1,13 @@
#!/usr/bin/python
from __future__ import print_function
"""
tuple_args.py: Testing out this little-known feature of Python, which compiler2
suppports.
"""

# Python 3 no longer supports this!
def f(a, (b, c), (d, e, f)):
  print(a, b, c, d, e, f)

if __name__ == '__main__':
  f(1, (2, 3), (4, 5, 6))