PySpark bindings for Mozilla Telemetry
Online documentation is available at https://mozilla.github.io/python_moztelemetry/
Deploying a code change
After having your PR reviewed and merged create a new release on github. A new pypi release will be automatically triggered by Travis.
Installing from pypi
- To install this package from pypi run:
pip install python_moztelemetry
moztelemetry/parse_histograms.py is a mirror of its counterpart from mozilla-central. To update it to the latest version you can run
Note that this file was formerly called histogram_tools.py and was renamed in Bug 1419761.
moztelemetry/heka/message_pb2.py is generated from
To regenerate it, you'll need to install a
protobuf package for your system.
To avoid installing go-specific extensions, remove the
gogoproto options and then run
git clone https://github.com/mozilla-services/heka mkdir pythonfiles/ protoc -I heka/message --python_out pythonfiles/ heka/message/message.proto
To test/debug this package locally, you can run exactly the job that CircleCI runs for continuous integration by installing the CircleCI local CLI and invoking:
circleci build --job py36
See [.circleci/config.yml] for the other configured job names (for running tests on different python versions).
The above process takes a few minutes to run every time, so there
is also a
bin/test script that builds a docker image and
python environment (both of which are cached locally) and allows
you to run a subset of tests. Here are some sample invocations:
./bin/test tests/ -k test_unpack # runs only tests with key "test_unpack" ./bin/test tests/heka/ # runs only tests in tests/heka PYTHON_VERSION=2.7 ./bin/test # specify a python version
It's also possible to run the tests locally outside of docker
tox directly, but the details of doing so depend
on your local development environment and are outside the scope
of these docs. Be aware that you will need to have a working
installation of Java and libsnappy, likely via your OS's package
brew install snappy on MacOS).
If you're receiving mysterious errors, try removing cached files via: