Source code for the email app running on the Lokole edge devices
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docker/app
opwen_email_client
setup
tests
travis
.dockerignore
.env
.gitignore
.pyup.yml
.travis.yml
Gruntfile.js
LICENSE.txt
MANIFEST.in
README.rst
babel.cfg
makefile
manage.py
package.json
requirements-dev.txt
requirements.txt
setup.py
version.txt
yarn.lock

README.rst

Opwen webapp

https://travis-ci.org/ascoderu/opwen-webapp.svg?branch=master

What's this?

This repository contains the source code for the Lokole email application. Lokole is a project by the Canadian-Congolese non-profit Ascoderu.

The Lokole is a simple email client that offers functionality like:

  1. Self-service creation of user accounts
  2. Read emails sent to the account
  3. Write emails including rich formatting
  4. Send attachments

All emails are stored in a local SQLite database. Once per day, the emails that were written during the past 24 hours get exported from the database, stored in a JSON file, compressed and uploaded to a location on Azure Blob Storage. The Lokole Server picks up these JSON files, manages the actual mailboxes for the users on the Lokole and sends new emails back to the Lokole by using the same compressed file exchange format.

The Lokole email application is intended to run on low-spec Raspberry Pi 3 hardware (or similar). Read the "Production setup" section below for further information on how to set up the client devices.

Why is this useful?

Email is at the core of our modern life, letting us keep in touch with friends and family, connecting us to our businesses partners and fostering innovation through exchange of information.

However, in many parts of the developing world, email access is not very wide-spread, usually because bandwidth costs are prohibitively high compared to local purchasing power. For example, in the Democratic Republic of the Congo (DRC) only 3% of the population have access to emails which leaves 75 million people unconnected.

The Lokole is a project by the Canadian-Congolese non-profit Ascoderu that aims to address this problem by tackling it from three perspectives:

  1. The Lokole is an email client that only uses bandwidth on a schedule. This reduces the cost of service as bandwidth can now be purchased when the cost is lowest. For example, in the DRC, $1 purchases only 65 MB of data during peak hours. At night, however, the same amount of money buys 1 GB of data.
  2. The Lokole uses an efficient data exchange format plus compression so that it uses minimal amounts of bandwidth, reducing the cost of service. All expensive operations (e.g. creating and sending of emails with headers, managing mailboxes, etc.) are performed on a server in a country where bandwidth is cheap.
  3. The Lokole only uses bandwidth in batches. This means that the cost of service can be spread over many people and higher savings from increased compression ratios can be achieved. For example, individually purchasing bandwidth for $1 to check emails is economically un-viable for most people in the DRC. However, the same $1 can buy enough bandwidth to provide email for hundreds of people via the Lokole. Spreading the cost in this way makes email access sustainable for local communities.

System overview & Data exchange format

Can be found in the opwen-cloudserver README.

Development setup

First, get the source code.

git clone https://github.com/ascoderu/opwen-webapp.git
cd opwen-webapp

Second, install the system-level dependencies using your package manager, e.g. on Ubuntu:

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update
sudo apt-get install -y yarn make python3 python3-venv shellcheck

Third, use the makefile to verify your installation by running the tests and starting up the server. The makefile will automatically install all required dependencies into a virtual environment and set up some dummy environment variables for local development. The server will automatically reload whenever any of the Flask code or Jinja templates are changed.

make tests
make server

The routes of the app are defined in views.py so take a look there for an overview of the entrypoints into the code.

For local development, you can set the following additional environment variables:

OPWEN_ENABLE_DEBUG="True"
LOKOLE_STORAGE_PROVIDER="LOCAL"
OPWEN_REMOTE_ACCOUNT_NAME="./tests/files/opwen_email_client"

With these environment variables set, when the Lokole exchanges data with the server, it will not make any calls to Azure and instead depend on the files in the ./tests/files/opwen_email_client directory. Any files uploaded to the server will be written to the compressedpackages subdirectory so that they can be inspected. To test sending emails from the server to the Lokole, a sample email batch file is included in the downloads directory. This file will be ingested by the client when the /sync endpoint is called.

Production setup

There is a script to set up a new Lokole device. The script will install the email app in this repository as well as standard infrastructure like nginx and gunicorn. The script will also make ready peripherals like the USB modem used for data exchange, and set up any required background jobs such as the email synchronization cron job.

The setup script assumes that you have already set up:

  • 3 Azure Storage Accounts, general purpose: for the cloudserver to manage its queues, tables and blobs.
  • 1 Azure Storage Account, blob storage: for the cloudserver and email app to exchange email packages.
  • 1 Application Insights account: to collect logs from the cloudserver and monitor its operations.
  • 1 SendGrid account: to send and receive emails in the cloudserver.

The setup script is tested with hardware:

The setup script is also tested with USB modems:

The setup script installs the latest version of the email app published to PyPI. New versions get automatically published to PyPI (via Travis) whenever a new release is created on Github.

You can run the script on your client device like so:

curl https://raw.githubusercontent.com/ascoderu/opwen-webapp/master/setup/setup-lokole.sh \
| bash -s \
    <client-name> <sim-type> <local-password> \
    <storage-account-name> <storage-account-key> <email-key> <server-tables-name> <server-tables-key> \
    <cloudflare-user> <cloudflare-key> <cloudflare-zone> <sync-schedule>

Adding a new language

export LANG=ln
make prepare-translations
poedit "opwen_email_client/webapp/translations/$LANG/LC_MESSAGES/messages.po"
make compile-translations

Or via Transifex.