FAQ

Sun edited this page Jun 23, 2016 · 14 revisions
  1. Why target Python 2 and not Python 3?

    Because our employer is on Python 2 and we only have time right now to support one.

  2. Why not contribute to PyPy instead?

    We have number of goals that we think are not possible to meet on the PyPy platform: performance consistency and scalability, and first-class extension module support. Extension support essentially forces the starting point to be the CPython codebase, and our ideas for performance don't seem compatible with their approach.

  3. How is Pyston different from Unladen Swallow?

    Pyston is similar to Unladen Swallow in some ways, in that it is a performance-oriented Python implementation that is closely based on CPython, and uses LLVM as part of its execution. Beyond that, we use a number of different techniques such as our mini-tracing JIT.

  4. Pyston was originally advertised as being a "method JIT", but now talks about using tracing. What gives?

    Pyston uses concepts from both types of JITs, using both method-at-a-time compilation and tracing (which we sometimes call "rewriting") to achieve good performance. They are somewhat orthogonal: both types of JITting can be used independently from each other, but we get the best performance when we use them together.

    We've found that the method-JIT portion (the LLVM JIT tier) has good "broad" knowledge and can relate the operations of different bytecodes to each other. The tracing JIT is how we get good "deep" knowledge on how to most efficiently execute a single bytecode. We hope to integrate these together and have the tracing JIT pass off its per-bytecode knowledge to the method-JIT which will apply whole-function optimizations.

  5. Why only Ubuntu?

    Because that's what we're familiar with and what Dropbox runs. Our experience is that it's usually pretty easy to get Pyston up and running on other *nix-y platforms, but we're not usually familiar with the package managers and what not.

    Mac OSX support has been attempted a couple times and proved hard; we'd still be very interested in it, but it would take a dedicated contributor a while to work through the issues, and it's not a priority for the project at the moment.

  6. What's the threading situation?

    We use a GIL

  7. Do you support NumPy?

    We run their entire test suite(NumPy 1.11) with one test failure, for which they’ve accepted our patch. For their latest trunk, we have three test failures. We do need to use a modified version of part of their build chain (Cython), and we are currently slower on the test suite than CPython. SciPy support is under way.

  8. How compatible is Pyston with existing Python code?

    We have a couple major compatibility breaks with standard CPython semantics:

    • Our extension module support is pretty good, but 1) requires a recompile of the extension source (which is also true for different versions of CPython), and 2) is not quite complete; we're missing a few of the more esoteric features. We also have a couple extra restrictions due to our garbage collection scheme, usually requiring a couple extra annotations in your extension code. Our plan is to be able to infer those automatically, but for now they are quite easy to add manually.
    • Pyston does not use CPython's bytecode format at all. This means that existing pyc files will not be usable, and code that directly inspects or manipulates bytecodes (such as the dis module) will not work.
    • Pyston is still a work in progress so there are still libraries and features that are missing even if there is no fundamental reason why we can't have them. We just don't have them yet.
    • We don't use Python dicts for attribute storage. If you access the __dict__ property of an object, you will get back a special attrwrapper dict-like object which behaves like a dict but with a couple differences. Notably, you can't use non-strings as keys. We plan on removing this restriction at some point.