-
Notifications
You must be signed in to change notification settings - Fork 0
terrence2/millipede
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Millipede - Give your snake legs! ====== Millipede is a Python application compiler. Goal ---- Python is a blazingly fast language, but only if the programming task is abstract enough to make use of the full suite of Python's features -- e.g. a program written in C at a similar level of abstraction would have similar speed. Python appears slow compared to many other languages in benchmarks largely because many/most programming tasks (particularly in benchmarks) do not take advantage of the majority of Python's features. The goal of Millipede is to have our cake and eat it too -- to provide the full power of Python when it's called for and to automatically remove unused abstractions for maximal performance everywhere else. How we do this: --------------- 1) Static analysis -- We use offline analysis of a project's source to pull out semantic information about how it uses Python. Unfortunately, in order to get any level of useful knowledge out of a project's source we need to do whole-program analysis; furthermore, doing this sort of static analysis is slow. The upshot is that this works best if we compile a full project at once, rather than doing the analysis one-file-at-a-time. This is the one major incompatibility between Millipede and Python. However, it's not a total lost cause, since importing files we haven't analyzed simply emits a PyImport_ImportModule(...) and runs them in the normal Python interpreter loop. Therefore, we can still be useful for programs that have a central core of functionality and add (at runtime) extensions or plugins to that base. 2) Feedback from the Analysis -- If compilation is a completely opaque process, it is hard for a programmer to understand why a certain bit of code is compiled the way it is. Millipede will integrate a GUI that shows the results of static analysis and tag that analysis with the information a programmer needs to optimize their code. For example, if a module optionally aliases a builtin in a global variable, then using that builtin in a tight inner loop will be slow. This is something that is easy for a programmer to fix (e.g. by assigning the builtin to a local), but only if they notice the problem in the first place. It should be both fast and easy to scan a piece of code for potential trouble spots. We see this becoming something like predator vision modes for code: the highlight color of a name will be based on the specificity of their derived type, the amount of indirection required for accesses, etc. 3) Feed-forward to the Compiler -- Some semantic aspects of a program are deviously hard to work out (in a reasonable amount of time) in a static analyzer. For these cases (particularly in performance critical code) it may be helpful for a programmer to give hints to the compiler, based on what they see in the GUI. If these appear as documentation (e.g. type annotations) in a form that is also useful to the human reader, then the cost to build an maintain them will hopefully be small enough to be worth the performance win. Note well: Turning Python into C++ is not the goal! We want our full, type-agnostic dynamism, but we need our performance too. In cases where we can get that performance automatically, wonderful. That said, pragmatism beats purity and if we can get a solid speed improvement by adding the occasional type annotation, so be it. What about JITing? ------------------ A tracing JIT is simply a code execution cache. This is a planned feature for Millipede. Current Status -------------- Millipede is fully self hosting, and it's performance in most cases is roughly equal in speed to CPython. Halfway There: The GUI elements exist (built on PyQt4) although the highlighting engine is still mostly pointing at a prior experiment and are, thus, mainly broken at the moment. Halfway There: Many optimizations based on control flow and type analysis. Playing with Millipede ------------------- For the truely brave/foolhardy: > python3 run.py -P 3.1 -O asp path/to/translate.py ### This will spit out some stuff and maybe create a test.c, if you are lucky. ### Good candidates of things that are likely to work are in test/<foo>/<bar>.py > make ### This makefile will try to link against python3.3 (e.g. cpython tip). ### It will also work with 3.1 and 3.2, so just twiddle the Makefile if needed. > ./test-prog ### And check the output against the code, if you value correct answers. Or run all tests with: > py.test -k <interpreter>:<version>:test/ -v test/ ### <Interpreter> is one of "python", "millipede", or "hosted" and <version> is one of 3.1 or 3.3.
About
A Python compiler and visualization tool.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published