Source code of the recurrent donations platform Liberapay
Permalink
Failed to load latest commit information.
.ebextensions drop VirtualHost hacks Dec 3, 2017
docker Update tests.yml for postgresql 9.4 to 9.6 May 15, 2018
emails add link to email settings in blacklist notification Sep 23, 2018
i18n update translation catalogs Nov 16, 2018
js fix invalid input handling in payout form Nov 5, 2018
liberapay avoid exceptions when leftover is `None` Nov 15, 2018
sql merge branch.sql Nov 10, 2018
style show recent donation changes in Receiving page Nov 5, 2018
templates fix content of donate button in profile page Nov 8, 2018
tests failing test for `payment_providers` on teams Nov 9, 2018
www fix pending donations in the "Giving" page Nov 15, 2018
.babel_extract work around Babel's bad handling of message IDs Sep 18, 2017
.editorconfig rename, clean up, adapt, internationalize May 26, 2015
.flake8 also run flake8 under Tox Oct 1, 2016
.gitattributes also send `environment` and `release` to Sentry Jan 15, 2018
.gitignore commit the new test fixture Mar 4, 2018
.lfsconfig Change lfs url Aug 24, 2016
.travis.yml bypass Git LFS on Travis Jul 24, 2017
Dockerfile Update Debian version from Dockerfile Aug 16, 2018
Makefile improve the `pytest-i18n-browse` Makefile target Oct 23, 2018
README.md add python 3.4 hashes Sep 30, 2018
_i18n_warning.html add a warning at the top of partially translated pages Jan 10, 2018
app.py modify gunicorn logs to include Host header Feb 22, 2016
backup.sh Refactor Shell Scripts based on ShellCheck's suggestion (#828) Dec 30, 2017
defaults.env also send `environment` and `release` to Sentry Jan 15, 2018
docker-compose.yml Update Docker PostgreSQL version to match requirements. Apr 25, 2018
error.spt show sentry event id in error pages Nov 2, 2017
pytest.ini rename, clean up, adapt, internationalize May 26, 2015
recreate-schema.sh postgres example with default port in recreate-schema.sh to avoid con… Jun 25, 2018
release.sh turn off broken schema drift check in `release.sh` Nov 10, 2018
requirements-prod.txt don't install prod requirements locally Feb 7, 2017
requirements_base.txt check MX records when adding a new email address Sep 12, 2018
requirements_dev.txt add python 3.4 hashes Sep 30, 2018
requirements_tests.txt upgrade oauth-related dependencies Aug 12, 2017
tox.ini automatically test docstrings in liberapay module Mar 29, 2018

README.md

Liberapay

Build Status Weblate Gitter Income Donate

Liberapay is a recurrent donations platform. We help you fund the creators and projects you appreciate.

Note: This webapp is not self-hostable.

Table of Contents

Contact

You want to chat? Join us on Gitter. (If you use IRC, Gitter has a gateway, and we're also in the #liberapay channel on Freenode.)

Alternatively you can post a message in our GitHub salon.

Contributing to the translations

You can help translate Liberapay via Weblate. Current status:

global translation status

translation status by language

If you have questions about translating Liberapay, you can ask them in the salon.

Contributing to the code

Introduction

Liberapay was originally forked from Gratipay and inherited its web micro-framework Pando ( Aspen), which is based on filesystem routing and simplates. Don't worry, it's quite simple. For example to make Liberapay return a Hello $user, your id is $userid message for requests to the URL /$user/hello, you only need to create the file www/%username/hello.spt with this inside:

from liberapay.utils import get_participant
[---]
participant = get_participant(state)
[---] text/html
{{ _("Hello {0}, your id is {1}", request.path['username'], participant.id) }}

As illustrated by the last line our default template engine is Jinja.

The _ function attempts to translate the message into the user's language and escapes the variables properly (it knows that it's generating a message for an HTML page).

The python code inside simplates is only for request-specific logic, common backend code is in the liberapay/ directory.

Installation

Firstly, make sure you have the following dependencies installed:

  • python ≥ 2.7.12 (the code is compatible with python 3, but production still runs on python 2 for now)
    • including the C headers of python and libffi, which are packaged separately in many Linux distributions
  • postgresql 9.6 (see the official download & install docs)
  • make

Then run:

make env

Now you need to give yourself superuser postgres powers (if it hasn't been done already), and create two databases:

su postgres -c "createuser --superuser $(whoami)"

createdb liberapay
createdb liberapay_tests

If you need a deeper understanding take a look at the Database Roles and Managing Databases sections of PostgreSQL's documentation.

Then you can set up the DB:

make schema

Configuration

Environment variables are used for configuration, the default values are in defaults.env and tests/test.env. You can override them in local.env and tests/local.env respectively.

Running

Once you've installed everything and set up the database, you can run the app:

make run

It should now be accessible at http://localhost:8339/.

You can create some fake users to make it look more like the real site:

make data

Payday

To run a local payday open http://localhost:8339/admin/payday and click the "Run payday" button. You can add OVERRIDE_PAYDAY_CHECKS=yes in the local.env file to disable the safety checks that prevent running payday at the wrong time.

SQL

The python code interacts with the database by sending raw SQL queries through the postgres.py library.

The official PostgreSQL documentation is your friend when dealing with SQL, especially the sections "The SQL Language" and "SQL Commands".

The DB schema is in sql/schema.sql, but don't modify that file directly, instead put the changes in sql/branch.sql. During deployment that script will be run on the production DB and the changes will be merged into sql/schema.sql. That process is semi-automated by release.sh.

CSS and JavaScript

For our styles we use SASS and Bootstrap 3. Stylesheets are in the style/ directory and our JavaScript code is in js/. Our policy for both is to have as little as possible of them: the website should be almost entirely usable without JS, and our CSS should leverage Bootstrap as much as possible instead of containing lots of custom rules that would become a burden to maintain.

We compile Bootstrap ourselves from the SASS source in the style/bootstrap/ directory. We do that to be able to easily customize it by changing values in style/variables.scss. Modifying the files in style/bootstrap/ is probably not a good idea.

Testing

The easiest way to run the test suite is:

make test

That recreates the test DB's schema and runs all the tests. To speed things up you can also use the following commands:

  • make pytest only runs the python tests without recreating the test DB
  • make pytest-re does the same but only runs the tests that failed in the previous run

Updating test fixtures

Some of our tests include interactions with external services. In order to speed up those tests we record the requests and responses automatically using vcr. The records are in the tests/py/fixtures directory, one per test class.

If you add or modify interactions with external services, then the tests will fail, because VCR will not find the new or modified request in the records, and will refuse to record the new request by default (see Record Modes for more information). When that happens you can either switch the record mode from once to new_episodes (in liberapay/testing/vcr.py) or delete the obsolete fixture files.

If the new interactions are with MangoPay you have to delete the file tests/py/fixtures/MangopayOAuth.yml, otherwise you'll be using an expired authentication token and the requests will be rejected.

Speeding up the tests

PostgreSQL is designed to prevent data loss, so by default it does a lot of synchronous disk writes. To reduce the number of those blocking writes our recreate-schema.sh script automatically switches the synchronous_commit option to off for the test database, however this doesn't completely disable syncing. If your PostgreSQL instance only contains data that you can afford to lose, then you can speed things up further by setting fsync to off in the server's configuration file (postgresql.conf).

Tinkering with payments

We depend on MangoPay for payments. If you want to modify that part of the code you'll need the MangoPay API documentation.

Modifying python dependencies

All new dependencies need to be audited to check that they don't contain malicious code or security vulnerabilities.

We use pip's Hash-Checking Mode to protect ourselves from dependency tampering. Thus when adding or upgrading a dependency the new hashes need to computed and put in the requirements file. For that you can use hashin:

pip install hashin
hashin package==x.y -r requirements_base.txt -p 2.7 -p 3.4 -p 3.6
# note: we have several requirements files, use the right one

If for some reason you need to rehash all requirements, run make rehash-requirements.

Processing personal data

When writing code that handles personal information keep in mind the principles enshrined in the GDPR.

Deploying the app

Note: Liberapay cannot be self-hosted, this section is only meant to document how we deploy new versions.

Liberapay is currently hosted on AWS (Ireland).

To deploy the app simply run release.sh, it'll guide you through it. Of course you need to be given access first.

Setting up a development environment using Docker

If you don't want to install directly dependencies on your machine, you can spin up a development environment easily, assuming you have Docker and docker-compose installed:

# build the local container
docker-compose build

# initialize the database
docker-compose run web bash recreate-schema.sh

# populate the database with fake data
docker-compose run web python -m liberapay.utils.fake_data

# launch the database and the web server
# the application should be available on http://localhost:8339
docker-compose up

You can also run tests in the Docker environment:

docker-compose -f docker/tests.yml run tests

All arguments are passed to the underlying py.test command, so you can use -x for failing fast or --ff to retry failed tests first:

docker-compose -f docker/tests.yml run tests -x --ff

License

CC0 Public Domain Dedication (See this discussion for details.)