Personal development fork of https://git.torproject.org/bridgedb.git. Don't clone this copy of the repo, or I'll happily rebase branches out from under you. Clone https://git.torproject.org/user/isis/bridgedb.git instead.
Switch branches/tags
bridgedb-database-improvements_1 bug26150 bug26154 develop feature/cleanup-config feature/sphinx-docs feature/trial-coverage feature/update-docstrings feature/2017-update-year feature/7520-social-dist-design_r1 feature/7520-social-dist-design feature/9199-improved-logging-2-r3 feature/9199-improved-logging-2 feature/9317-metrics-contexts feature/9865-test-harness-coverage feature/9865-test-runner-2-r1+nacl feature/9865-test-runner-2-r1 feature/9865-test-runner-2 feature/9865-test-runner feature/9937-desc-gen-r1 feature/9959-pas-danglais feature/10724-collect-timestamps-option feature/22755-stem-descgen feature/23957 feature/25246 fix/make-config.py fix/mako-template-caching fix/remove-dead-code fix/travis-debug fix/update-gsoc-text fix/update-license-year fix/update-translations fix/0.2.3-readme-testing-instructions fix/1606-based-on_bug1606_spec_update_rebased_r1 fix/1839-rotation-periods fix/3015-remove-buckets fix/3573-robots-txt fix/4405-tor-exit-check_stash fix/4405-tor-exit-check fix/4405-tor-exit-check_2_r1 fix/4771-log-tor-exits_r1 fix/5332-deployment-instructions-rebased fix/5463-7547-7550-8241-11475-11753-email-rewrite fix/5463-gpgme-homedir fix/5463-sign-client-email-addr fix/5655-add-gatech-email fix/6127-render_GET-traceback fix/6127-simple-error-page fix/6127-web-server-tracebacks fix/7550-9678-interactive-translations fix/9013_r1 fix/9127-https-interface-ipv6 fix/9199-extra-logging-changes-r2 fix/9199-safelogging fix/9277-config fix/9380-stem_r10 fix/9385-fuzzy-email-matching_r1 fix/9385-fuzzy-email-matching fix/9462-refactor-netstatus-parsers_r9462C_r2 fix/9462-refactor-netstatus-parsers_r9462C fix/9462A-keyid-equals-fingerprint fix/9462B-parse-netstatus-file fix/9462C-ipaddr-portlist-module fix/9873-convert-old-unittests_r1 fix/9873-convert-old-unittests_r2 fix/9873-convert-old-unittests_r3 fix/9873-convert-old-unittests fix/9874-automate-email-tests fix/9874-email fix/9874-https fix/9874-remove-requirements.txt fix/9988-apply-patch fix/9988-rm-hashlibnew fix/10333-index-by-0L_r2 fix/10333-index-by-0L fix/10385-python-gnupg_r1 fix/10446-geoip-module fix/10559-keyid-in-pt-lines fix/10737-post-args-redirects fix/10803-remove-vidalia-update-options fix/10809-gimp-captcha_r1 fix/10811-add-SIGUSR1-handler fix/11127-recaptcha-ssl_10809r1_r1 fix/11139-riseup_r1 fix/11139-riseup fix/11196-use-leekspin_r1 fix/11215-captcha-timeouts fix/11218-deferred-solution-isvalid fix/11219-httpconnpool fix/11231-additional-bug-fixes fix/11231-blank-captcha-field fix/11345-qrcodes_r2 fix/11346-homepage-link fix/11522-exc-in-email-dist fix/11664-email-buffer-interface fix/12029-dist-api_r1 fix/12029-dist-api_r2 fix/12029-distribute-module fix/12029-hashring-refactor fix/12029-refactor-Bridges fix/12030-db-manager_r1 fix/12030-db-manager fix/12031-redis_r1 fix/12031-redis_r2 fix/12031-redis fix/12086-rcptto-other-domain fix/12091-dkim-dunno fix/12122-translator-notes fix/12130-obfs4 fix/12504-config-pts fix/12505-5418-remove-old-block-code fix/12505-11330-hashrings_r1 fix/12505-11330-hashrings fix/12505-12029-11330-hashrings_r1 fix/12505-12029-11330-hashrings_r2 fix/12505-12029-11330-hashrings fix/12505-refactor-hashrings_r6 fix/12505-refactor-hashrings_r7 fix/12506-separate-dist-dirs fix/12507-automate-documentation-builds fix/12627-hotfixes fix/12635-tbb-dl-links fix/12664-bridgeline-newlines_r2 fix/12709-enable-fte fix/12843-cc-spurning_r2 fix/12843-cc-spurning fix/12872-bridge-geoip_r1 fix/12932-pt-args-spaces fix/13123-bridgeline-spaces fix/13202-missing-pt-args fix/14064-bridgeline-wrapping_r2 fix/14065-vanilla-ipv6 fix/14685-disable-obfs2 fix/14865-disable-obfs2 fix/15155-instructions-string fix/15464-ci-directives_r1 fix/15522-ipv6-enumeration fix/15866-broken-bridgeauth fix/15968-16649-csp-and-mobile fix/16273-gitweb-urls fix/16330-eid-sds-2-fields fix/16616-hsdir-flag fix/18237 fix/18949 fix/19691 fix/20087 fix/20088 fix/21162 fix/22871_r1 fix/22871 fix/22998 fix/23033 fix/23034 fix/24432-ignore-loopback fix/24432-json-str fix/24432 fix/24433 fix/24443 fix/24460 fix/24636 fix/24637 fix/24701 fix/24704 fix/26023 gh-pages hotfix/remove-sqlitedict-class hotfix/requirements-travis hotfix/revert-a49ee2a hotfix/ude-persistent hotfix/0.2.1-docstrings hotfix/0.2.3-html-footer-icons hotfix/0.2.3-pyopenssl-0.14 hotfix/0.2.3-travis-mechanize-http hotfix/0.2.4-14797-4405-missing-proxylistfile hotfix/0.2.4-14797-requirements hotfix/0.2.4-14797-var-bulkexitlist hotfix/0.2.4-bump-mako-version hotfix/0.2.4-bump-pygeoip-version hotfix/0.2.4-mechanize-tags hotfix/0.2.4-pypy-support hotfix/0.2.4-travis-matrix-update hotfix/0.2.4-travis-pyeggcache hotfix/0.3.0-twisted-14.0.2_r1 hotfix/0.3.1-9385-levenshtein-distance hotfix/0.3.1-makefile hotfix/0.3.1-pypy-support hotfix/0.3.2-error-pages hotfix/0.3.2-format-plain-bytes hotfix/0.3.2-pep440 hotfix/0.3.2-remove-lib-dir hotfix/0.3.3-error-pages hotfix/3573-robots-txt_add-web-resource hotfix/5332-lib-gitignore hotfix/5463-email-line-recieved-logging hotfix/9156-pt-bridges-still-have-bridge-prefix-string hotfix/9264-descdigest-none hotfix/9264_2_rtl-2step hotfix/9380-duplicate-extrainfo-error hotfix/9462B-netstatus-returns-None hotfix/9872-options-subcommands-relative-rundir hotfix/9872-return-status-codes hotfix/9937-desc-gen_base64-padding-descDigest hotfix/9937-desc-gen_fix-tabs hotfix/9959-pas-danglais_install-data hotfix/10385-init-gpg-ret hotfix/10803-bridge-lines-newlines hotfix/11127-deferred-solution-isvalid hotfix/11215_12147-intervalstart hotfix/11218-async-render hotfix/11522-gpg-signing hotfix/12627-additional-unicode-fix master release/0.5.0 release/0.6.1 release/0.6.3 rransom/fix/12147-schedule-unittest rransom/5463-ui-typos testing/feature/9865-test-harness-coverage testing/fix/9462-refactor-netstatus-parsers_r9462C testing/fix/9462-refactor-netstatus-parsers testing/fix/9462C-ipaddr-portlist-module testing/rransom/2014-04-02-01-fix-timing-leaks-v3 testing/sysrqb/bug1606_spec_update_rebased testing/sysrqb/bug5232_based_on_fix/9462_r9462C_r2_test2 testing/sysrqb/bug5232_based_on_fix/9462_r9462C_r2 testing/sysrqb/bug5232_r2_based_on_fix/9462_r9462C_r2_cherry-pick-7bb36dd0857 testing/sysrqb/bug5232_r2_based_on_fix/9462_r9462C_r2_cherry-pick-44d2bc761c13 testing/sysrqb/bug5232_r2_based_on_fix/9462_r9462C_r2 testing/tpo-sysrqb/bug5232_newdesign_dev_r1-test1 testing/tpo-sysrqb/bug5232_newdesign_dev_r1-unittests-only testing/tpo-sysrqb/bug5232_newdesign_dev_test1 testing/tpo-sysrqb/bug9127 testing/tpo-sysrqb/bug9264_rebased_1-test2 translations/2014-04-24-update-pot-file translations/2014-04-24-update translations/2014-05-07-update translations/2014-05-16-update translations/2014-06-06-update translations/2014-07-25 translations/2015-03-19-update translations/2015-03-20-update translations/2015-07-25-update
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gnupg
bridgedb
captchas
doc
maint
scripts
.coveragerc
.gitattributes
.gitignore
.pylintrc
.test.requirements.txt
.travis.requirements.txt
.travis.yml
AUTHORS
CHANGELOG
LICENSE
MANIFEST.in
Makefile
README.rst
bridgedb.conf
requirements.txt
setup.cfg
setup.py
versioneer.py

README.rst

BridgeDB Latest Version Build Status Coverage Status

BridgeDB is a collection of backend servers used to distribute Tor Bridges. Currently, it mainly consists of a webserver with an HTTPS interface, an email responder, and an SQLite database.

doc/sphinx/source/_static/bay-bridge.jpg

What are Tor Bridges?

Tor Bridges are special Tor relays which are not listed in the public relay directory. They are used to help circumvent censorship by providing users with connections to the public relays in the Tor network.

Tor Bridges are different from normal relays in another important way: they can run what are called Pluggable Transports.

What's a Pluggable Transport?

A Pluggable Transport is a program which is pluggable — meaning that it is meant to work with lots of other anonymity and censorship circumvention software, not just Tor — and is a transport — meaning that it transports your internet traffic, usually in a way which makes it look different. For example, Obfsproxy is a Pluggable Transport which disguises your traffic by adding an obfuscating layer of encryption.

So how do I use this?

Well, probably, you don't. But if you're looking for bridges, you can use the web interface of the BridgeDB instance deployed by the Tor Project, which has instructions on getting the Pluggable Transports-capable Tor Browser Bundle, as well as instructions for getting extra Bridges.

Maintainer Setup

If you'd like to hack on BridgeDB, you might wish to read BridgeDB's developer documentation. The rest of this document mainly concerns mainenance and installation instructions.

Dependencies and installation

BridgeDB requires the following OS-level dependencies:

  • python>=2.7
  • python-dev
  • build-essential
  • gnupg (preferrably, gnupg2)
  • OpenSSL>=1.0.1g
  • SQLite3
  • MaxMind GeoIP
  • libgeoip-dev
  • geoip-database
  • python-setuptools
  • libjpeg-dev
  • flog (only required if bridgedb is invocated with the run-bridgedb script)

As well as any Python dependencies in the requirements.txt file.

Deploying BridgeDB

BridgeDB should work with or without a Python virtualenv.

  • Install Python 2.7, and other OS-level dependencies. On Debian, you can do:

    sudo apt-get install build-essential openssl python python-dev \
      python-setuptools sqlite3 gnupg2 libgeoip-dev geoip-database
    
  • Install Pip 1.3.1 or later. Debian has this version, but if for some reason that or a newer version isn't available, the easiest way to install a newer Pip is to use the Pip development teams's getpip script:

    wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
    sudo python get-pip.py
    
  • (virtualenv installs only) Use Pip to install virtualenv and virtualenvwrapper:

    sudo pip install --upgrade virtualenv virtualenvwrapper
    
  • (virtualenv installs only) Configure virtualenvwrapper and create a virtualenv for BridgeDB:

    WORKON_HOME=${HOME}/.virtualenvs
    export WORKON_HOME
    mkdir -p $WORKON_HOME
    source $(which virtualenvwrapper.sh)
    git clone https://git.torproject.org/bridgedb.git && cd bridgedb
    mkvirtualenv -a $PWD -r requirements.txt --unzip-setuptools --setuptools bridgedb
    

    From now on, to use BridgeDB's virtualenv, just do $ workon bridgedb (after sourcing virtualenvwrapper.sh, as before). To exit the virtualenv without exiting the shell, do $ deactivate.

  • (virtualenv installs only) To install, set PYTHONPATH to include the root directory of the virtualenv:

    export PYTHONPATH=$PYTHONPATH:${VIRTUAL_ENV}/lib/python2.7/site-packages
    
  • Then, proceed as usual:

    python setup.py install --record installed-files.txt
    

Enabling additional features

Translations

Using New Translations:

This should be done when newly completed translations are available in Transifex.

Piece of cake. Running maint/get-completed-translations will take care of cloning only the bridgedb_completed branch of Tor's translations repo and placing all the updated files in their correct locations.


Requesting Translations for Altered/Added Source Code:

This should be done whenever any of the strings requiring translation -- _("the ones inside the weird underscore function, like this") -- are changed, or new ones are added. See lib/bridgedb/strings.py.

Translations for Tor Project repos are kept in a separate repo. You'll need to extract the strings from BridgeDB's source code into .pot templates, and place these .po files into the translation repo in the bridgedb branch. After than the .po files should be put into Transifex (don't ask me how this works…) and translated. After the translations are complete, the finished .po files should be placed into the bridgedb_completed branch.

  • To extract all strings from BridgeDB's source:

    python setup.py extract_messages
    

    A .pot file will be created in ./i18n/templates/bridgedb.pot

  • Initialise catalogs for each desired language:

    python setup.py init_catalog -l LANG
    

    where LANG is the 2 or 4 letter country-code, eg. 'es'. If you've already initialised a particular language, do instead:

    python setup.py update_catalog
    

Enabling HTTPS

Create a self-signed certificate with:

scripts/make-ssl-cert

Or, place an existing certificate in the path specified in bridgedb.conf by the HTTPS_CERT_FILE option, and a private key where HTTPS_KEY_FILE points to. The defaults are 'cert' and 'privkey.pem', respectively.

Enabling CAPTCHA Support

BridgeDB has two ways to use CAPTCHAs on webpages. The first uses reCaptcha, an external Google service (this requires an account with them), which BridgeDB fetches the CAPTCHAs images from for each incoming request from a client. The second method uses a local cache of pre-made CAPTCHAs, created by scripting Gimp using gimp-captcha. The latter cannot easily be run on headless server, unfortunately, because Gimp requires an X server to be installed.

reCaptcha

To enable fetching CAPTCHAs from the reCaptcha API server, set these options in bridgedb.conf:

RECAPTCHA_ENABLED
RECAPTCHA_PUB_KEY
RECAPTCHA_SEC_KEY

gimp-captcha

To enable using a local cache of CAPTCHAs, set the following options:

GIMP_CAPTCHA_ENABLED
GIMP_CAPTCHA_DIR
GIMP_CAPTCHA_HMAC_KEYFILE
GIMP_CAPTCHA_RSA_KEYFILE

GnuPG email signing:

In your bridgedb.conf file, make sure that:

EMAIL_GPG_SIGNING_ENABLED = True

and edit the following option to add the full fingerprint of the GnuPG key that should be used to by BridgeDB to sign outgoing emails:

EMAIL_GPG_PRIMARY_KEY_FINGERPRINT

The key specified by EMAIL_GPG_PRIMARY_KEY_FINGERPRINT can be a master key, or a subkey (with or without the private portions of its corresponding master key), but it must be inside the secring.gpg and pubring.gpg keyrings inside the directory specified in the bridgedb.conf option:

EMAIL_GPG_HOMEDIR

If the key has requires a passphrase for signing, you'll also need to set either of:

EMAIL_GPG_PASSPHRASE
EMAIL_GPG_PASSPHRASE_FILE

Preventing already-blocked bridges from being distributed:

Uncomment or add COUNTRY_BLOCK_FILE to your bridgedb.conf. This file should contain one bridge entry per line, in the format:

fingerprint <bridge fingerprint> country-code <country code>

If the COUNTRY_BLOCK_FILE file is present, bridgedb will filter blocked bridges from the responses it gives to clients requesting bridges.

Testing BridgeDB

Before running to any of BridgeDB's test suites, make sure you have the additional dependencies in the Pip requirements file, .test.requirements.txt installed:

pip install -r .test.requirements.txt

To create a bunch of fake bridge descriptors to test BridgeDB, do:

bridgedb mock [-n NUMBER_OF_DESCRIPTORS]

Note that you will need to install leekspin in order to run the bridgedb mock command. See doc/HACKING.md for details.

And finally, to run the test suites, do:

make coverage

If you just want to run the tests, and don't care about code coverage statistics, see the bridgedb trial and bridgedb test commands.

Running BridgeDB

To run BridgeDB, simply make any necessary changes to bridgedb.conf, and do:

bridgedb

And remember that all files/directories in bridgedb.conf are assumed relative to the runtime directory. By default, BridgeDB uses the current working directory; you can, however specify an a different runtime directory:

bridgedb -r /srv/bridges.torproject.org/run

Make sure that the files and directories referred to in bridgedb.conf exist. However, many of them, if not found, will be touched on disk so that attempts to read/write from/to them will not raise excessive errors.

Reloading Bridges From Their Descriptor Files:

When you have new lists of bridges from the Bridge Authority, replace the old files and do:

bridgedb --reload

Or just give it a SIGHUP:

kill -s SIGHUP `cat .../run/bridgedb.pid`

To extract all bridge assignments:

To dump all bridge assignments to files, send BridgeDB a SIGUSR1 signal by doing:

kill -s SIGUSR1 `cat .../run/bridgedb.pid`

Using a BridgeDB Instance

Obviously, you'll have to feed it bridge descriptor files from a BridgeAuthority. There's currently only one BridgeAuthority in the entire world, but Tor Project is, of course, very interested in adding support for multiple BridgeAuthorities so that we can scale our own network, and make it easier for individual and organisations who wish to run a lot of Tor bridge relays have an easier time distributing those bridges themselves (if they wish to do so). If you'd like to fund our work on this, please contact tor-dev@lists.torproject.org!

Accessing the HTTPS User Interface

Just connect to the appropriate port. (See the HTTPS_PORT and HTTP_UNENCRYPTED_PORT options in the bridgedb.conf file.)

The HTTPS interface for our BridgeDB instance can be found here.

Accessing the Email User Interface

Any mail sent to the EMAIL_PORT with a destination username as defined by the EMAIL_USERNAME configuration option (the default is 'bridge', e.g. bridges@...) and sent from an @riseup.net, @gmail.com, or @yahoo.com address (by default, but configurable with the EMAIL_DOMAINS option).

You can email our BridgeDB instance here.

Accessing the Moat Interface

Moat is a bridge distributor for requesting bridges through Tor Launcher's user interface.

The following describes the Moat API, version 0.1.0.

The client and server both MUST conform to JSON-API.

The client SHOULD direct all requests via the Meek reflector at MEEK_REFECTOR. ..

XXX meek reflector URL

Requesting Bridges

The client MUST send a POST /meek/moat/fetch containing the following JSON:

{
  "data": {
    "version": "0.1.0",
    "type": "client-transports",
    "supported": [ "TRANSPORT", "TRANSPORT", ... ],
  }
}

where:

  • TRANSPORT is a string identifying a transport, e.g. "obfs3" or "obfs4". Currently supported transport identifiers are: - "vanilla" - "fte" - "obfs3" - "obfs4" - "scramblesuit"

Receiving a CAPTCHA challenge

The moat server will respond with 200 OK.

If there is an overlap with what BridgeDB supports, the moat server will select the "best" transport from the list of supported transports, and respond with the following JSON containing a CAPTCHA challenge:

{
  "data": {
    "id": "1",
    "type": "moat-challenge",
    "version": "0.1.0",
    "transport": "TRANSPORT",
    "image": "CAPTCHA",
    "challenge": "CHALLENGE",
  }
}

where:

  • TRANSPORT is the agreed upon transport which will be distributed,
  • CAPTCHA is a base64-encoded, jpeg image that is 400 pixels in length and 125 pixels in height,
  • CHALLENGE is a base64-encoded CAPTCHA challenge which MUST be later passed back to the server along with the proposed solution.

The challenge contains an encrypted-then-HMACed timestamp, and solutions submitted more than 30 minutes after requesting the CAPTCHA are considered invalid.

If there is no overlap with the transports which BridgeDB supports, the moat server will respond with the list of transports which is does support:

{
  "data": {
    "id": "1",
    "type": "moat-challenge",
    "version": "0.1.0",
    "transport": [ "TRANSPORT", "TRANSPORT", ... ],
    "image": "CAPTCHA",
    "challenge": "CHALLENGE",
  }
}

Responding to a CAPTCHA challenge

To propose a solution to a CAPTCHA, the client MUST send a request for POST /meek/moat/check, where the body of the request contains the following JSON:

{
  "data": {
    "id": "2",
    "type": "moat-solution",
    "version": "0.1.0",
    "transport": "TRANSPORT",
    "challenge": "CHALLENGE",
    "solution": "SOLUTION",
    "qrcode": "BOOLEAN",
  }
}

where:

  • TRANSPORT is the agreed upon transport which will be distributed,
  • CHALLENGE is a base64-encoded CAPTCHA challenge which MUST be later passed back to the server along with the proposed solution.
  • SOLUTION is a valid unicode string, up to 20 bytes in length, containing the client's answer (i.e. what characters the CAPTCHA image displayed). The solution is not case-sensitive.
  • BOOLEAN is "true" if the client wants a qrcode containing the bridge lines to be generated and returned; "false" otherwise.

Receiving Bridges

If the CHALLENGE has already timed out, or if the SOLUTION was incorrect, the server SHOULD respond with 419 No You're A Teapot.

If the SOLUTION was successful for the supplied CHALLENGE, the server responds 200 OK with the following JSON:

{
  "data": {
    "id": "3",
    "type": "moat-bridges",
    "version": "0.1.0",
    "bridges": [ "BRIDGE_LINE", ... ],
    "qrcode": "QRCODE",
  }
}

where:

  • BRIDGE_LINE is a bridge line suitable for configuration in a torrc,
  • QRCODE is a base64-encoded jpeg image of a QRCode containing all the BRIDGE_LINE, if one was requested, otherwise this field will be NaN.

If the SOLUTION was successful for the supplied CHALLENGE, but the server is unable to distribute the requested Bridges, the server responds 200 OK with the following JSON:

{
  "errors": [{
    "id": "6",
    "type": "moat-bridges",
    "version": "0.1.0",
    "code": "404",
    "status": "Not Found",
    "detail": "DETAILS",
  }]
}

where:

  • DETAILS is some string describing the detailed nature of the issue.

Other Responses

If the client requested some page other than /meek/moat/fetch, or /meek/moat/check, the server MUST respond with 501 Not Implemented.

If the client attempts any other HTTP method, other than POST, the server MUST respond 403 FORBIDDEN.

Contact & Support

Send your questions, patches, and suggestions to the tor-dev mailing list or isis.