Closure Compiler with the Python demo?

[timdawborn]

I've compiled the Python 2.7.1 source to a .ll file successfully and passed it through emscripten successfully to get out a JavaScript file. The problem I now have is that the Closure Compiler dies when I try to run it over the emscripten-produced JavaScript file. In your Python demo, the JS file looks like it has been Closure Compiled. I was wondering if there was anything you need to do in order to get this to work?

$ ./emscripten.py ~/downloads/Python-2.7.1/bin/pylibs/python.ll > input.js
$ java -Xmx2g -jar ../compiler/build/compiler.jar --js_output_file output.js --js input.js
java.lang.RuntimeException: java.lang.RuntimeException: INTERNAL COMPILER ERROR.
Please report this problem.
Analysis did not terminate after 200000 iterations
...
$ 

[kripken]

Are you using the latest closure compiler from svn, compiled yourself? It has recently gotten several fixes that are important for large files (one in particular filed by us).

[timdawborn]

I tried both the latest download and trunk (r1193). They both produced this same error. Do you have the issue number handy which you filed?

[timdawborn]

FYI: my ticket http://code.google.com/p/closure-compiler/issues/detail?id=492

[kripken]

Issue 435, http://code.google.com/p/closure-compiler/issues/detail?id=435

If it doesn't work on their svn, then the best thing is probably to open an issue with them. If you do so, please mention the issue number here.

Perhaps the difference results we are seeing is due to different compilation options. I did llvm_gcc_0_1 in the test runner (no llvm opts, yes to typed arrays, reloop and optimize).

edit: thanks for posting the issue #!

[kripken]

I also run closure compiler differently than you,

java -jar CLOSURE_COMPILER --compilation_level ADVANCED_OPTIMIZATIONS --variable_map_output_file src.cpp.o.js.vars --js src.cpp.o.js --js_output_file src.cpp.o.cc.js

[timdawborn]

Yup, I tried it with those settings from tests/runner.py too, but it didn't make any difference.

I still haven't got this working without crashing :( What setting maps to "no llvm opts"? Typed arrays, reloop and optimise are all attributes of src/settings.py, but I couldn't find anything obvious about llvm opts.

[kripken]

LLVM_OPTS is used in the test runner (tests/runner.py). If set to 1, llvm-opt is run.

Running tests with

python tests/runner.py clang_x_y.test_XXX

will set LLVM_OPT to the value of x (and y is optimizations/typed arrays). I compiled the python demo with llvm_gcc_0_1, so no llvm opts.

What command did you run to build the python demo?

[timdawborn]

Right. I was following the readme in tests/python/readme.txt on how to build from scatch. My shell script is as follows:

#!/bin/bash
# build as per ${EMSCRIPTEN}/tests/python/readme.txt

PYTHON=${HOME}/downloads/Python-2.7.1.tar.bz2
LLVM=${HOME}/downloads/llvm-2.9/cbuild/Release/bin
LLVM_GCC=${HOME}/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin
EMSCRIPTEN=${HOME}/reposs/emscripten

# extract Python
rm -rf Python-2.7.1
tar jxf ${PYTHON}
cd Python-2.7.1
mkdir -p bin/pylibs
cd bin

# setup ccproxy
cp ${EMSCRIPTEN}/tests/python/ccproxy.py .
chmod +x ccproxy.py
sed -i "s@\.\.LLVM_GCC_DIR\.\.@${LLVM_GCC}@; s@\.\.LLVM_DIR\.\.@${LLVM}@g" ccproxy.py

# configure
CC=./ccproxy.py ../configure --without-threads --without-pymalloc
sed -i 's@HAVE_GCC_ASM_FOR_X87@PY_NO_SHORT_FLOAT_REPR@' pyconfig.h
sed -i -r 's@(^#define\s+HAVE_SIG.*)@// \1@; s@^// (#define\s+HAVE_SIGNAL_H\s)@\1@' pyconfig.h

# make until failure
make

# manually link and disassemble
cd pylibs
ar x ../libpython2.7.a
cp ../Modules/python.o .
${LLVM}/llvm-link -o=python.bc *.o
${LLVM}/llvm-dis -show-annotations python.bc

# return
cd ../../..

# run byte code through emscripten
${EMSCRIPTEN}/emscripten.py Python-2.7.1/bin/pylibs/python.ll > python2.7.1.js

The resultant python2.7.1.js is then passed to the closure compiler as per my initial post.

[kripken]

If you haven't edited settings.js, then that will build with assertions and without OPTIMIZE and RELOOP. That generates larger (and slower) code. So that might be the difference that causes closure to get more confused.

Seems that they know what the issue is over there though, hopefully they can fix it.

Btw, you can compile the python demo using the one-line command I mentioned (through the test runner). That should work for now, unless you need to build manually for some reason.

[timdawborn]

Yeah, I worked that one out last night in fact. Was just about to update this issue :) Thanks!

We need some manual steps in the pipeline as we're building a slightly-nonstandard build of Python.

[kripken]

Note that you can change the stuff in settings.js by passing stuff to emscripten.py. The test runner does it that way. You basically pass a JSON object in a string, with RELOOP: 1 etc.