- Linux or other UNIX-like operating system
- ElasticSearch 7.x or higher installed
- Python 3.7.3 or higher
- PipEnv program (typically
apt install pipenv
oryum install pipenv
etc) - A web server of your choice with proxy capabilities
Warning: the ElasticSearch client libraries from 7.14 have strict server version checking and will not run with the wrong version. You may need to update the requirements.txt files in tools/ and server/ accordingly.
- Clone the Foal git repository to your machine:
git clone https://github.com/apache/incubator-ponymail-foal.git foal
- Install the Python requirements for the setup:
cd foal/tools
pipenv install -r requirements.txt
- Install any desired optional dependencies, for example:
- html2text (GNU GPL 3) - convert HTML to plain text
- mailman (GPLv3+) - interface with Mailman lists
- zope (Zope Public Licence 2.1) - required for Mailman integration
- Run the setup process:
pipenv run python3 setup.py
cd ..
- Import any mailboxes you need to, using
tools/import-mbox.py
OR migrate your old Pony Mail database using the Foal migrator. - Install the server requirements:
cd server/
pipenv install -r requirements.txt
- start the server:
pipenv run python3 main.py
- use a web server like httpd or nginx to serve UI and proxy to the API. A sample httpd configuration could be:
# Load required modules
LoadModule proxy_module ...
LoadModule proxy_http_module ...
...
<VirtualHost *:80>
ServerName ponymail.example.org
DocumentRoot /var/www/ponymail-foal/webui/
ProxyPass /api/ http://localhost:8080/api/
# PathInfo is needed for threads
AcceptPathInfo On
# Also needed for threads to be able to handle message-ids with embedded /:
AllowEncodedSlashes On # (or NoDecode)
</VirtualHost>
A sample nginx config could be:
server {
root /var/www/ponymail-foal/webui/;
index index.html index.htm;
location /api/ {
proxy_pass http://127.0.0.1:8080/api/;
}
}
To set up archiving, the easiest path is to edit your /etc/aliases
file on the machine
that receives email. If your receiving address for email is inbox@yourmachine.tld
, your
alias entry should look like this:
inbox: "| /path/to/foal/tools/archiver.py"
For privately archived emails, use the --private
flag for your command:
inbox: "| /path/to/foal/tools/archiver.py --private"
Once you have added the entry, be sure to run the newaliases
command to update the compiled alias list.
archiver.py
will automatically sort out which list the email is for, if there is a List-ID header.
You can override or manually set a list using the --lid
flag:
inbox-somealias: "| /path/to/foal/tools/archiver.py --lid somealias@mydomain.tld"
By default, OAuth is enabled for the following providers:
- GitHub
- Generic (like oauth.apache.org)
None of these are marked as authoritative by default. Authoritative OAuth domains allow users to compose emails via the UI and see private emails (unless you reshape the AAA plugin). Non-authoritative domains only allows the user to log in, nothing more.
To set an OAuth provider as authoritative, you need to add or uncomment the
authoritative_domains
section of the oauth
configuration in server/ponymail.yaml
:
oauth:
authoritative_domains:
- googleapis.com
- myoauthprovider.tld
For administrative access to certain features, such as deleting/moving email via the UI, you can set a list of people who, via an authoritative oauth provider, will have access to this, as such:
oauth:
authoritative_domains:
- googleapis.com
admins:
- humbedooh@gmail.com
- example@gmail.com
Currently, you will also need to enable or tweak your webui/js/config.js
file to match your
choice of OAuth providers, though that is subject to change.
To begin using Google OAuth, you must procure an OAUth2 client id from the Gooogle Developers Console. Callback URL must be oauth.html in your webui installation.
Once you have a Client ID
, you should set it in server/ponymail.yaml
in the
google_client_id
directive. You will also need to currently set it in webui/js/config.js
.
After this is done, OAuth should work with Google, and you may enable authoritativeness by adding
googleapis.com
to the authoritative_domains
section in server/ponymail.yaml
.
To begin using GitHub OAuth, create an OAuth app at the GitHub Developer Console. Callback URL must be oauth.html in your webui installation.
Once you have created your OAuth app, copy the client ID and client secret to your
server/ponymail.yaml
oauth section, as github_client_id
and github_client_secret
respectively.
When you've done that, you must currently also edit webui/js/config.js
and set the
client_id
for GitHub to the correct value.
It is possible to reply to emails on a list via the web interface, provided you have enabled
this feature. To do so, you must set a mailhost
option as well as designate certain
domains and subdomains as allowed recipients of email. This is done in the server/ponymail.yaml
file, under ui:
ui:
wordcloud: true
mailhost: smtp.mydomain.tld:25
sender_domains: mydomain.tld *.mydomain.tld
You may use wildcards in your domain names, as per standard GLOB/fnmatch rules, separating each entry with a single space:
- to allow email for list@mydomain.tld, sender_domains must include
mydomain.tld
- to allow email for list@sub.mydomain.tld, you should add
sub.mydomain.tld
- to allow email for all lists at any subdomain under mydomain.tld, you should add
*.mydomain.tld
Only users logged in via authoritative OAuth will be able to compose replies via the web interface.
the ui
paragraph of the server configuration allows for enabling an administrative interface
for editing or removing emails from the archives. To enable this, set mgmtconsole
to true
.
For GDPR compliance (deleting an email deletes from disk), set allow_delete
to true
.
If left out or set to false, deleted emails are merely hidden, and can be recovered at a later
stage by an administrator.
ui:
mgmtconsole: true
allow_delete: true
The administrative interface can be accessed by clicking on the yellow cog in the context menu of an email. Admins are defined in the OAuth configuration.
By default, API errors will include a full traceback for debugging purposes. If you wish to
instead have this be printed to the system journal (stderr
), you can set the traceback
option to false
in server/ponymail.yaml
. This will instead print an error ID to the user,
corresponding to a traceback in stderr.
If the error ID is, for instance, a06f7d4b-3a82-4ecf
, you can find the corresponding traceback
by grepping your programs output. If you are running Foal as a systemd service, you could find
the traceback with: journalctl --no-pager -u yourservicename | grep a06f7d4b-3a82-4ecf
To enable the storage in elasticsearch of extra properties related to
threading, the following configuation snippet can be used in the
server/ponymail.yaml
file:
archiver:
threadinfo: yes
threadparents: 10
threadtimeout: 5
The threadparents
value limits the number of existing messages that
will be queried for thread information at archive time when a new
message is received. The threadtimeout
value limits the duration of
each query to elasticsearch.
Enabling threadinfo
means that top
, thread
, and previous
properties will be added to each stored message. The top
property is
a boolean, indicating whether or not the message is the start of a new
thread. The thread
property gives the generated Foal ID of the top
of the current thread; this will be the same as the ID of the current
message if top
is true. The previous
property gives the generated
Foal ID of either the most recent parent message if the message is not
the top of a thread, or the top of the most recent thread otherwise.