Skip to content
Antenna is a breakpad crash collector for Socorro
Python Shell Makefile Dockerfile
Branch: master
Clone or download
willkg Merge pull request #373 from willkg/1601455-crashpublish
bug 1601455: add AWS SQS crashpublish support
Latest commit 0721f99 Dec 6, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
antenna bug 1601455: add AWS SQS crashpublish support Dec 6, 2019
bin bug 1601455: Fix queue creation in local dev env Dec 6, 2019
docker
docs bug 1601455: add AWS SQS crashpublish support Dec 6, 2019
requirements Update dependencies Dec 5, 2019
testlib Add documentation for run_on_error Nov 7, 2019
tests bug 1601455: Fix queue creation in local dev env Dec 6, 2019
.editorconfig Fix max_line_length in .editorconfig Nov 7, 2019
.gitignore bug 1531865: redo scaffolding to match Socorro Mar 4, 2019
.pyup.yml bug 1559151: add pyup and set to monthly updates Jun 14, 2019
CODE_OF_CONDUCT.md See PR for details Apr 2, 2019
CONTRIBUTING.rst
LICENSE Initial pass at project infrastructure May 16, 2016
MANIFEST.in Initial pass at project infrastructure May 16, 2016
Makefile
README.rst bug 1598765: add localstack-sqs Nov 23, 2019
docker-compose.yml bug 1598765: add localstack-sqs Nov 23, 2019
setup.cfg Update release.py and codify configuration Aug 6, 2019
setup.py Split requirements into default and constraints Jul 18, 2018

README.rst

Antenna: Breakpad crash report collector

Breakpad crash collector web app that handles incoming crash reports and saves them to AWS S3.

Uses Python 3, Gunicorn, gevent, Falcon and some other things.

Quickstart

This is a quickstart that uses Docker so you can see how the pieces work. Docker is also used for local development of Antenna.

For more comprehensive documentation or instructions on how to set this up in production, see docs.

  1. Clone the repository:

    $ git clone https://github.com/mozilla-services/antenna
  2. Install docker 1.10.0+ and install docker-compose 1.6.0+ on your machine

  3. Download and build Antenna docker containers:

    $ make build

    Anytime you want to update the containers, you can run make build.

  4. Set up local Pub/Sub and S3 services:

    $ make setup

    Anytime you want to wipe service state and recreate them, you can run make setup.

  5. Run with a prod-like fully-functional configuration.

    1. Running:

      $ make run

      You should see a lot of output. It'll start out with something like this:

      /usr/bin/docker-compose up web
      antenna_statsd_1 is up-to-date
      antenna_localstack-s3_1 is up-to-date
      Recreating antenna_web_1
      Attaching to antenna_web_1
      web_1      | [2016-11-07 15:39:21 +0000] [7] [INFO] Starting gunicorn 19.6.0
      web_1      | [2016-11-07 15:39:21 +0000] [7] [INFO] Listening at: http://0.0.0.0:8000 (7)
      web_1      | [2016-11-07 15:39:21 +0000] [7] [INFO] Using worker: gevent
      web_1      | [2016-11-07 15:39:21 +0000] [10] [INFO] Booting worker with pid: 10
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: Setting up metrics: <class 'antenna.metrics.DogStatsdMetrics'>
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.metrics: DogStatsdMetrics configured: statsd:8125 mcboatface
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: BASEDIR=/app
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: LOGGING_LEVEL=DEBUG
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: METRICS_CLASS=antenna.metrics.DogStatsdMetrics
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: DUMP_FIELD=upload_file_minidump
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: DUMP_ID_PREFIX=bp-
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_CLASS=antenna.ext.s3.crashstorage.S3CrashStorage
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: THROTTLE_RULES=antenna.throttler.mozilla_rules
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_ACCESS_KEY=foo
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_SECRET_ACCESS_KEY=*****
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_REGION=us-east-1
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_ENDPOINT_URL=http://localstack-s3:4572
      web_1      | [2016-11-07 15:39:21 +0000] [INFO] antenna.app: CRASHSTORAGE_BUCKET_NAME=antennabucket
      
    2. Verify things are running:

      In another terminal, you can verify the proper containers are running with:

      $ docker-compose ps

      You should see containers with names web, statsd and localstack-s3.

    3. Send in a crash report:

      You can send a crash report into the system and watch it go through the steps:

      $ ./bin/send_crash_report.sh
      ...
      <curl http output>
      ...
      CrashID=bp-6c43aa7c-7d34-41cf-85aa-55b0d2160622
      *  Closing connection 0

      You should get a CrashID back from the HTTP POST. You'll also see docker logging output something like this:

      web_1      | [2016-11-07 15:48:45 +0000] [INFO] antenna.breakpad_resource: a448814e-16dd-45fb-b7dd-b0b522161010 received with existing crash_id
      web_1      | [2016-11-07 15:48:45 +0000] [INFO] antenna.breakpad_resource: a448814e-16dd-45fb-b7dd-b0b522161010: matched by is_firefox_desktop; returned ACCEPT
      web_1      | [2016-11-07 15:48:45 +0000] [INFO] antenna.breakpad_resource: a448814e-16dd-45fb-b7dd-b0b522161010 accepted
      web_1      | [2016-11-07 15:48:45 +0000] [INFO] antenna.breakpad_resource: a448814e-16dd-45fb-b7dd-b0b522161010 saved
      
    4. See the data in localstack-s3:

      The localstack-s3 container stores data in memory and the data doesn't persist between container restarts.

      You can use the aws-cli to access it. For example:

      AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=foo \
          aws --endpoint-url=http://localhost:5000 \
              --region=us-east-1 \
              s3 ls s3://antennabucket/
      

      If you do this a lot, turn it into a shell script.

    5. Look at runtime metrics with Grafana:

      The statsd container has Grafana. You can view the statsd data via Grafana in your web browser http://localhost:9000.

      To log into Grafana, use username admin and password admin.

      You'll need to set up a Graphite datasource pointed to http://localhost:8000.

      The statsd namespace set in the dev.env file is "mcboatface".

    6. When you're done--stopping Antenna:

      When you're done with the Antenna process, hit CTRL-C to gracefully kill the docker web container.

    If you want to run with a different Antenna configuration in the local dev environment, adjust your my.env file.

    See docs for configuration options.

  6. Run tests:

    $ make test

    If you need to run specific tests or pass in different arguments, you can run bash in the base container and then run py.test with whatever args you want. For example:

    $ make shell
    app@...$ py.test
    
    <pytest output>
    
    app@...$ py.test tests/unittest/test_crashstorage.py

    We're using py.test for a test harness and test discovery.

For more details on running Antenna or hacking on Antenna, see the docs.

You can’t perform that action at this time.