Projector helps you set up isolated environments for developing Python projects, and pack these environment into isolated packages that you can later install on production.
At Infinidat, we build software for system administrators, that expect a complete, packaged product, and not just a bunch of Python scripts. Since Python (at least until Python 3.2) does not provide a decent mechanism for building Python applications (with a bundled interpreter), we started building our own solution, and this is what we came up with.
By using some skeleton,
buildout recipes and some wrapping code, we provide:
- Isolated python builds for your applications
- Simple, easy-to-use version release mechanism
- Packaging Python applications into MSI, RPM, DEB
Projector is available on
infi.projector. To install just run:
pip install -U infi.projector
We will explain the bits and bolts about using project by a straight-forward walkthrough.
Creating a new project
We will start by creating a new project:
projector repository init --mkdir infi.example email@example.com:Infinidat/example.git "example on infi.projector" "An example by walkthrough"
This creates a new skeleton for the project, under a new directory, called
infi.example. Inside, you'll find the following files:
buildout.cfg. This project's configuration file. Already includes the project name and description strings.
setup.in. A template file for setup.py. Modify when necessary.
Also, you'll notice that the checked out branch is
develop. That is because we use
gitflow's branching model. That means that
master holds merges for final releases only, and that the development is on
develop or separate feature branches.
Building the development environment
If you just create a new project, or clone an existing one, you won't have a proper setup.py, or any console script. In order to generate those, run:
projector devenv build
This script will create the buildout environment, generate the version files from Git, and create the executable scripts under the
If you want to use our isolated python builds in our development environment, and not the global Python, run instead:
projector devenv build --use-isolated-python
This will download the platform-specific build from our servers, and use that.
There are other flags for this command, you can read about them by passing
Projects handled by
projector have two types of dependencies:
productiondependencies. written to setup.py, honored by
developmentdependencies. written to buildout.cfg, affects only the development environment.
Handling both types of dependencies/requirements is easy:
projector requirements list [--development] projector requirements add <requirement> [--development] [--commit-changes] projector requirements remove <requirement> [--development] [--commit-changes]
freeze the development environment by using the following commands:
projector requirements freeze [--with-install-requires] [--commit-changes] [--newest] projector requirements unfreeze [--with-install-requires] [--commit-changes]
What this does is adds the versions of the downloaded dependencies to
buildout.cfg, so that you'll always get the same set of dependencies when you run
If you pass the
--with-install-requires flag, this will also update the dependencies in
install_requires with a
>= requirement to the locally installed version.
Make sure to use the single quotes around version specification
'underscore<=2.0.0 and to use the full semantic version (e.g. 3.3.1 and not 3.3 or 3).
The dependencies will be downloaded and extracted from the npm registry.
projector js-requirements list projector js-requirements add <requirement> [--commit-changes] projector js-requirements remove <requirement> [--commit-changes] projector js-requirements freeze [--commit-changes] [--push-changes] projector js-requirements unfreeze [--commit-changes] [--push-changes]
projector js-requirements add 'jquery>=3.0.0' projector js-requirements add underscore projector js-requirements remove 'jquery>=3.0.0' porjector js-requirements list >>> ['underscore']
js-requirements plugin also supports freeze and unfreeze functionality, but first you must run
projector devenv build after adding dependencies so a
.package-lock.json file will be created.
projector devenv build projector js-requirements freeze
projector provide a simple command-line interface to manage
console_scripts entry points in setup.py:
projector console-scripts list projector console-scripts add <script-name> <entry-point> [--commit-changes] projector console-scripts remove <script-name> [--commit-changes]
Adding package data
If you wish to include additional (non-Python-source) files in your python package distribution,
projector can help you set it up:
projector package-data list projector package-data add <filename> [--commit-changes] projector package-data remove <filename> [--commit-changes]
As we mentioned earlier, we use gitflow's branching model and versioning scheme. However, there's a little more to do when releasing versions than just merging git branches:
- You need to find out the latest version number, and advance on top of it
- You need to bump the version-related files in the source
- You need to upload the version to PyPI
- You need to push the commit and tags to origin
projector solves all of these problems with the following command-line options:
projector version release <version> [--no-fetch] (--no-upload | [--distributions=DISTRIBUTIONS] [--pypi-servers=PYPI_SERVERS]) [--push-changes] projector version upload <version> [--distributions=DISTRIBUTIONS] [--pypi-servers=PYPI_SERVERS]
Where the options are:
release. Release a new version, including registering and uploading to pypi
upload. Upload an existing version to pypi
--distributions=DISTRIBUTIONS. Distributions to build [default: sdist,bdist_egg]
--pypi-servers=PYPI. PyPI server for publishing [default: pypi,]
<version>. x.y.z, or the keywords: major, minor, trivial (for 'release' only) or current, latest (for 'upload' command only)
--no-upload. Do not upload the package as part of the release process
--no-fetch. Do not fetch origin before releasing
--push-changes. We often forget to push the commits and tags to origin. This option does it for you
Packaging applications (not just modules)
If you're developing a complete application, and not just a python package, you probably want to generate a sysadmin-friendly package of your app, bundled with its own iterpreter.
In order to do this, you will have to add the following dependency to your project by running the command:
projector requirements add infi.recipe.console_scripts
Then, you'll need to build the development environment with the isolated python included:
projector devenv build --use-isolated-python
After the environment is ready, you can use
projector to build stand-alone, isolated, packages. just run:
projector devenv pack
Based on the platform you're running on, this command will generate a package under the
- MSI on Windows platform
- DEB on ubuntu
- RPM on redhat/centos
The target arhicture of the package will also match the platform you are using (x86 for 32bit, x64 for 64bit)
In some cases, you'd want to always use a set of command-line arguments for a specific project, or maybe some user will want to use the same arguments for all their projects.
You can do so by writing the following configuration file to
./.projector (per-project) or
[commandline-arguments] --use-isolated-python = True --pypi-servers=pypi,local
This is just an example of course.
Checking out the code
The easiest way to checkout the code is by using projector itself:
easy_install infi.projector projector clone git://github.com/Infindiat/infi.projector.git cd infi.projector projector devenv build
There are two alternatives.