Bazel PyPi Package
A Bazel macro for building Python packages and interacting with PyPi. The goal is to have the package configuration one would have in a
setup.py file inside a Bazel BUILD file as a
pypi_package rule. Interacting with PyPi is then done via Bazel commands, rather than the regualr commands, which are outside the build system.
Bazel offers the
py_binary rules. These are great for working inside a single application codebase. However, many Python projects are libraries, and are exported as packages in the Python Package Index - PyPi. Bazel does not offer anything for integrating with it, at the moment. As such, all the interaction is done through the regular mechanisms described in Packaging and Distributing Projects. This implies the existance of the
.pypirc files, as well as the existance of
*.egg-info and other top-level directories which contain Python build artifacts. This state of affairs is quite messy. There are two separate build systems present, each with their configuration and each producing different forms of output clutter. Ideally we'd want only one - Bazel.
The goal of this project is to correct this state of affairs, by providing a small set of tools which encapsulate all the configuration and steps necessary for managing the interaction with the Python package index. At some point in the future, it should be integrated with regular Bazel.
Installation and Usage
Bazel doesn't yet allow importing macro libraries in the workspace. Therefore, one has to copy
pypi_package.bzl in a location from which other BUILD files can load it.
The following usage example is borrowed from the tabletest package, which is a small utility library for writing neater unit-tests.
In a BUILD file, one has to first import the macro, like so:
We define a regular Python library, which will be included in the package. In general, we can have more than one such library.
py_library( name = "tabletest", srcs = ["tabletest/__init__.py"], visibility = ["//visibility:public"], srcs_version = "PY2" )
In a manner similar to the configuration for the
setup function, we then write the
pypi_package rule as:
pypi_package( name = "tabletest_pkg", version = "1.0.2", description = "Unit testing module for table-like test, for Python 2.", long_description = "README.md", classifiers = [ "Development Status :: 4 - Beta", "Environment :: Console", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Operating System :: POSIX", "Programming Language :: Python :: 2", "Programming Language :: Python :: 2.6", "Programming Language :: Python :: 2.7", "Topic :: Software Development :: Testing", "Topic :: Software Development :: Libraries :: Python Modules" ], keywords = "unittest test table testing", url = "http://github.com/horia141/tabletest", author = "Horia Coman", author_email = "firstname.lastname@example.org", license = "MIT", packages = [":tabletest"], test_suite = "nose.collector", tests_require = ["nose"], )
We must first register the package with PyPi. This is achieved by running the following binary:
bazel run //:tabletest_register -- --pypi_user=[your username] --pypi_pass=[your password]
After registering (which should be done only once, but is otherwise idempotent), we can upload the current version of the code (the result of building the
py_library rule from above) to the package index via:
bazel run //:tabletest_upload -- --pypi_user=[your username] --pypi_pass=[your password]
The name of the
pypi_package rule needs to end in
_pkg and the prefix for it will be used to generate the
Each time a new version needs to be updated, the
version field must be updated.
Interaction With Other Packages
tests_require fields, one can specify the required dependencies for installing the package and for testing it. These operate at the PyPi level however, so they are specified as regular strings. However, if one depends on a repository for a package which also uses Bazel and this macro, then one can depend on that rule, rather than just specifying it as a string.
For example, the
WORKSPACE file for the SDHash project, looks like this:
git_repository( name = "tabletest", remote = "https://github.com/horia141/tabletest.git", tag = "v1.0.3", )
pypi_package rule references the tabletest package like this:
pypi_package( name = "sdhash_pkg", version = "0.0.3", description = "Library for image hashing and deduplication.", long_description = "README.md", classifiers = [...], keywords = "photo image gif hash perceptual dedup deduplication fft", url = "http://github.com/horia141/sdhash", author = "Horia Coman", author_email = "email@example.com", license = "MIT", packages = [":sdhash"], install_requires = ["pillow", "numpy", "scipy"], test_suite = "nose.collector", tests_require = ["nose", "@tabletest//:tabletest_pkg"], )
tests_require line could also have looked like this:
... tests_require = ["nose", "tabletest"], ...
In both cases, the package must exist in PyPi.