The Cython language makes writing C extensions for the Python language
as easy as writing in Python itself, but it currently has some
I propose to improve Cython's compiler design by developing and
* a new Parser,
* an Abstract Syntax Tree ,and
* a Flow Graph
These enhancements will allow:
* full parsing of all valid Python 2.5 code
* easier transition to Python 3000
* simpler support of new features
* better reasoning about the code for optimizations
This is not a from-scratch implementation. I worked on similar
projects before and many of the proposed ideas have been already
tested in the CPython compiler and in pypy, another open source python
compiler. (I participated in a pypy sprint in 2006 and worked on an
AST peephole optimizer)
By modifying and adapting existing open source code and leveraging
from my experience, I shall accomplish this project in the available
The ability to maintain Cython in the long term will be significantly
enhanced by these modifications.
For the suggested enhancements see:
For the complete application text see:
This document describes the complete proposal for this project.
What is the Objective ?
* Improve Cython's compiler design, enabling the generation of
code for Python 2.5 syntax.
* Pave the way for future improvements and optimizations.
What is Cython ? How it is related to Python?
The Cython language makes writing C extensions for the Python
language as easy as writing in Python.
The Cython Compiler generates C code that makes full use of the
CPython API, making it easy to mix compiled and non-compiled
code. It does not convert Python code to a pure C program.
What about the proposal ?
The current Cython implementation has some shortcomings.
After studying Cython's design, I propose to improve Cython's
compiler design by developing and integrating:
* a new Parser
* an Abstract Syntax Tree
* a Flow Graph
These enhancements will allow full parsing of all valid Python
The central idea in my design is to build an Abstract
Syntax Tree from the parsed input source.
The concrete benefit of this approach lead the team of Python
developers to implement an AST for the Python compiler during the 2.4 -> 2.5 transition
(More technical details in "How it will be done?" section on
Why the need of this project ? Why should be the project approved?
* Cython's long term maintainability will be significantly
enhanced by the project
* The project would benefit the python, Cython and Open Source
community in several ways
What will the accomplished project deliver?
What is in it for the Cython project ?
The proposed enhancements will allow:
1. Simpler support of new features
2. Full parsing of Python 2.5 syntax
3. Better reasoning about the code for optimizations
4. Pave the way to support Python 3000 features
5. Improve performance: many well know optimizing techniques could
be implemented, boosting extension speed execution
What is in it for the Python community ?
1. Better usability, Faster prototyping, development and code reuse
2. Better module support for the different Python API versions
3. Easier porting for existing C code, giving the python community
more extensions to use and leverage
4. Improve performance: existing python modules could be compiled to C
What is in it for the Open Source community ?
1. Easier way of integrating python with existing C/C++ applications
2. C Programs could leverage from the power of Python in a faster
What is in it for me?
1. This work will be the base of my master's thesis on
"automatic code generation from dynamics language for high
2. Gain a better understanding of Cython and Python internals
3. Improve my skill in writing optimized Python applications
4. Learn more on Python and C / C++ integration
5. Have fun with the community of Cython
6. Be sponsored by my favorite company (Google)
How do you know that your project is feasible?
During this week I spent time on the IRC Cython channel discussing
with the Cython community the pros, cons, caveats and pitfalls of
With the objective to understand my idea, to explore the entity of
the project, and obtain an inside understanding of the written
code, I started modifying the existing code, and wrote some first
At the moment I have a working test in my local repository for a
small subset of the above ideas. (see:
Cython has a robust test suite (~200 tests, ~4176 line of code).
I shall heavily use it during development.
To have a concrete idea of the amount of work to be done I wrote
some documents describing the expected behavior.
What is the Project Estimate ?
The schedule constraint is 12 Full Weeks.
Parser: 1 to 3 weeks
Ast: 2 to 5 weeks
Flow Graph: 4 to 6+ weeks
( This will be refined on the online version )
Why Me ?
1. I have done this before
2. I am an experienced Python and C/C++ programmer
3. I am competent about project's topic
4. I am passionate
(These claims are further developed in
By supporting this project, you can save a lot of time and neurons
for many developers. This means that more developers will use Cython
to develop their libraries/extensions, leading to more libraries
for Python, that will increase Python's adoption by companies...
And what is one of the key measurement of a language's success?
The number of companies using the language.
Help me improve further Python and Cython's success.
Thank you for your interest.
Note: for further detailed explanation read the detailed Description:
PyPy see : http://codespeak.net/pypy/dist/pypy/doc/home.html