This is a fork of CPython with pluggable memory allocation routines, made to demo the allocators built by students in MIT's 6.172 (Performance Engineering of Software Systems).
It turns out that even the allocate-only reference implementation provided here has a 60% faster startup as compared to base CPython, and in some test cases has marginally better runtime performance as well.
With a fully fleshed out allocator I've seen the same 60% startup time increase, along with up to 10% faster runtimes across the board, again as compared to base CPython.
Note: this implementation emulates sbrk in the same way as mdriver, in that it allocates a bunch of memory at start, and hands that out to malloc as needed. This is a possible reason for the performance increase, and it may be considered to be hacky, but if thats all it takes to get the kind of speedups we're seeing, I'm all for it.
To use your own memory allocator with this Python, implement the my_malloc, my_realloc, my_free, and my_init functions in Objects/mymalloc.h.
Be sure to obey the real semantics of realloc, which state that passing NULL as the previous pointer causes realloc to become malloc. The original staff reference does not do this, and as such we will not require submissions to do this, but it is needed for your implementation to work in CPython.
Testing your implementation --------------------------To test just startup:: $ time ./python.exe StartupTime.py
Starting CPython requires something like 2000 malloc's, so small increases in performance here can have a large impact on end user experience. The original CPython starts up in 50ms on my computer, but after subbed to use even this most simple implementation of malloc, the resultant Python requires only 20ms for startup.
- To test with some sample Python routines::
$ ./python.exe Demo.py
I've seen around 10% speedup on these routines, but theres a decent amount of variance.
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation. All rights reserved.
See the end of this file for further copyright and license information.
- Website: https://www.python.org
- Source code: https://github.com/python/cpython
- Issue tracker: https://bugs.python.org
- Documentation: https://docs.python.org
- Developer's Guide: https://devguide.python.org/
For more complete instructions on contributing to CPython development, see the Developer Guide.
Installable Python kits, and information about using Python, are available at python.org.
On Unix, Linux, BSD, macOS, and Cygwin:
./configure
make
make test
sudo make installThis will install Python as python3.
You can pass many options to the configure script; run ./configure --help to find out more. On macOS and Cygwin, the executable is called python.exe; elsewhere it's just python.
If you are running on macOS with the latest updates installed, make sure to install openSSL or some other SSL software along with Homebrew or another package manager. If issues persist, see https://devguide.python.org/setup/#macos-and-os-x for more information.
On macOS, if you have configured Python with --enable-framework, you should use make frameworkinstall to do the installation. Note that this installs the Python executable in a place that is not normally on your PATH, you may want to set up a symlink in /usr/local/bin.
On Windows, see PCbuild/readme.txt.
If you wish, you can create a subdirectory and invoke configure from there. For example:
mkdir debug
cd debug
../configure --with-pydebug
make
make test(This will fail if you also built at the top-level directory. You should do a make clean at the toplevel first.)
To get an optimized build of Python, configure --enable-optimizations before you run make. This sets the default make targets up to enable Profile Guided Optimization (PGO) and may be used to auto-enable Link Time Optimization (LTO) on some platforms. For more details, see the sections below.
PGO takes advantage of recent versions of the GCC or Clang compilers. If used, either via configure --enable-optimizations or by manually running make profile-opt regardless of configure flags, the optimized build process will perform the following steps:
The entire Python directory is cleaned of temporary files that may have resulted from a previous compilation.
An instrumented version of the interpreter is built, using suitable compiler flags for each flavour. Note that this is just an intermediary step. The binary resulting from this step is not good for real life workloads as it has profiling instructions embedded inside.
After the instrumented interpreter is built, the Makefile will run a training workload. This is necessary in order to profile the interpreter execution. Note also that any output, both stdout and stderr, that may appear at this step is suppressed.
The final step is to build the actual interpreter, using the information collected from the instrumented one. The end result will be a Python binary that is optimized; suitable for distribution or production installation.
Enabled via configure's --with-lto flag. LTO takes advantage of the ability of recent compiler toolchains to optimize across the otherwise arbitrary .o file boundary when building final executables or shared libraries for additional performance gains.
We have a comprehensive overview of the changes in the What's New in Python 3.8 document. For a more detailed change log, read Misc/NEWS, but a full accounting of changes can only be gleaned from the commit history.
If you want to install multiple versions of Python see the section below entitled "Installing multiple versions".
Documentation for Python 3.8 is online, updated daily.
It can also be downloaded in many formats for faster access. The documentation is downloadable in HTML, PDF, and reStructuredText formats; the latter version is primarily for documentation authors, translators, and people with special formatting requirements.
For information about building Python's documentation, refer to Doc/README.rst.
Significant backward incompatible changes were made for the release of Python 3.0, which may cause programs written for Python 2 to fail when run with Python 3. For more information about porting your code from Python 2 to Python 3, see the Porting HOWTO.
To test the interpreter, type make test in the top-level directory. The test set produces some output. You can generally ignore the messages about skipped tests due to optional features which can't be imported. If a message is printed about a failed test or a traceback or core dump is produced, something is wrong.
By default, tests are prevented from overusing resources like disk space and memory. To enable these tests, run make testall.
If any tests fail, you can re-run the failing test(s) in verbose mode. For example, if test_os and test_gdb failed, you can run:
make test TESTOPTS="-v test_os test_gdb"If the failure persists and appears to be a problem with Python rather than your environment, you can file a bug report and include relevant output from that command to show the issue.
See Running & Writing Tests for more on running tests.
On Unix and Mac systems if you intend to install multiple versions of Python using the same installation prefix (--prefix argument to the configure script) you must take care that your primary python executable is not overwritten by the installation of a different version. All files and directories installed using make altinstall contain the major and minor version and can thus live side-by-side. make install also creates ${prefix}/bin/python3 which refers to ${prefix}/bin/pythonX.Y. If you intend to install multiple versions using the same prefix you must decide which version (if any) is your "primary" version. Install that version using make install. Install all other versions using make altinstall.
For example, if you want to install Python 2.7, 3.6, and 3.8 with 3.8 being the primary version, you would execute make install in your 3.8 build directory and make altinstall in the others.
Bug reports are welcome! You can use the issue tracker to report bugs, and/or submit pull requests on GitHub.
You can also follow development discussion on the python-dev mailing list.
If you have a proposal to change Python, you may want to send an email to the comp.lang.python or python-ideas mailing lists for initial feedback. A Python Enhancement Proposal (PEP) may be submitted if your idea gains ground. All current PEPs, as well as guidelines for submitting a new PEP, are listed at python.org/dev/peps/.
See 569 for Python 3.8 release details.
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018 Python Software Foundation. All rights reserved.
Copyright (c) 2000 BeOpen.com. All rights reserved.
Copyright (c) 1995-2001 Corporation for National Research Initiatives. All rights reserved.
Copyright (c) 1991-1995 Stichting Mathematisch Centrum. All rights reserved.
See the file "LICENSE" for information on the history of this software, terms & conditions for usage, and a DISCLAIMER OF ALL WARRANTIES.
This Python distribution contains no GNU General Public License (GPL) code, so it may be used in proprietary projects. There are interfaces to some GNU code but these are entirely optional.
All trademarks referenced herein are property of their respective holders.