IMAP4v1 Download Client

2014, Georg Sauthoff mail@georg.so

This repository contains a IMAP4v1 client for fetching messages from an IMAP server and storing them in a local maildir directory. It is also able to delete them on the server if they are successfully transferred.

Besides the actual client, the repository may be of interest as a base for other IMAP clients/servers because it includes Ragel based parsers for the client and server side of the IMAP protocol. For such use cases it is recommended to include this repository as Git submodule.

(last change to this readme file: 2017-04-29)

Features

• Maildir support - client makes sure to fsync at the right times to minimize data loss in the case of power failure
• SSL (i.e. TLS) enabled by default
• In case the connection is interrupted during a fetch operation, the UIDs of completely fetched messages are written to a journal and expunged on next program start before the remaining messages are fetched. Useful, when e.g. retrieving a large mailbox over an unreliable mobile network (think: UMTS when travelling in a high speed train).
• display From/Subject/Date headers during fetching (when INFO severity level is turned on)
• Workarounds for some IMAP server bugs (deviations from the RFC)
• Uses the UIDPLUS extension if available - thus, excluding side effects with concurrently established server connections when purging messages
• Plain tilde expansion in local mailbox paths
• Configuration via JSON run control file
• Written in C++ with some C++11 features
• Asynchronous IO using Boost ASIO
• Robust IMAP protocol parser implemented in Ragel
• Before commands are send to the server they are locally parsed with a Ragel grammar that implements the server side of the IMAP spec - thus, the client verifies that it doesn't violate RFC3501 in any obvious way.
• High test coverage with unittests

Design Choices

• Use sane and secure defaults (e.g. TLS encryption and certificate validation are enabled by default).
• Don't send commands to the server that are expected to fail (e.g. the LOGIN command when it is disabled or the UID expunge command when the server has no UIDPLUS extension).
• Verify commands before sending them to the server.
• Use asynchronous IO without threads.
• Use state machines where it makes the code more robust, compact, easier to reason about etc.
• Support IPv4 and IPv6.
• Use layering where it reduces complexity (e.g. in the download client the differenes between the Boost ASIO TCP and SSL APIs are abstracted away by a layer, on top of that are the IMAP parsers layered, etc.)
• Give SASL support a low priority, because IMAP over TLS generally makes SASL redundant (cf. SASL notes section).
• Make it portable (e.g. via using open standards like ISO C++, POSIX, portable tools like CMake, Ragel and portable libraries like Boost).

Compile

This project includes some Git submodules, thus, make sure that they are correctly updated after git clone (i.e. verify that the subdirectories libbuffer, libixxx and cmake/ragel are there. Perhaps you have to adjust the submodule URLs, in case you don't have your Github ssh account correctly setup.

Out-of-source builds are recommended, thus, do something like this:

$ git clone git@github.com:gsauthof/imap.git
$ cd imap
$ git submodule update --init
$ mkdir build
$ cd build
$ cmake ..
$ make imapdl ut

If can also use an alternative build generator with cmake, of course, e.g.:

$ cmake -G Ninja ..
$ ninja-build imapdl ut

In case you have Boost installed in a custom location you can do something like:

$ BOOST_ROOT=/home/juser/local/boost-1.55 cmake -G Ninja ..

For setting some CMake variables:

$ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=true -G Ninja ..

Or supply a custom cache initialization file via cmake -C.

Meson

Alternatively, this project can be built with Meson. The Meson build configuration is located in meson.build. The Meson build system is younger than CMake, it has more useful defaults, a saner syntax and some nice features.

Example:

$ meson --buildtype=release build
$ cd build
$ ninja-build imapdl

Dependencies

• C++11 Compiler (e.g. GCC >= 4.8)
• POSIX like system
• CMake (>= 2.8)
• Boost (>= 1.55)
• Ragel (>= 6.6)
• OpenSSL (tested with 1.0.1e)
• Crypto++ or Botan (only for unittests)

Basic usage

On the first run, imapdl generates a run controle file template under:

$HOME/.config/imapdl/rc.json

You can copy the 'example' section and rename it to 'default' for the default account. For multiple accounts you can use a section name of your choice and supply --account somename to the imapdl command line.

The default cipher list for SSL is somewhat progressive (i.e. it basically only include forward secrecy providing TLSv1.2 ciphers). You can use a more relaxed list via setting cipher_preset to 2 or 3 (higher means more relaxed) - or you can configure your own cipher list.

For verifying the authenticity of the certificate provided by the server you can either specify

• the certificate authority (CA) root certificate file,
• a directory with CA root certificates, or
• the expected fingerprint of the server certificate

Tested platforms

• Linux (Fedora 19/20 x86_64)

IMAP Servers

• Dovecot (1.2.9)
• Cyrus (Cyrus IMAP v2.3.13)
• Google GMail (Gimap)

Licence

GPLv3+

Appendix

Example Subdirectory

A collection of small example programs for testing some related features:

• length.rl - lexing a token based on the preceding length value
• client.cc - exploring ASIO features, simple example ASIO client, also used for unittesting replay feature
• server.cc - exploring ASIO features, also used for replaying IMAP sessions in unittests
• replay.cc - for dumping serialized network sessions
• hash.cc - implement sha256sum using the Botan C++ library

SASL Notes

When securing the connection with TLS, SASL doesn't increase your security. On the other hand, on unsecured connections SASL only protects your password somewhat (depending on the used method) but it doesn't help against man-in-the-middle attacks or eavesdropping on the content of your mails. SASL is perhaps of interest for certain methods, e.g. to support Kerberos authentication with IMAP servers, but I haven't come across such setups, yet.

Links

• RFC 3501 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1, Proposed STD, 2003
• RFC 2683 - IMAP4 Implementation Recommendations, Informational, 1999
• RFC 2180 - IMAP4 Multi-Accessed Mailbox Practice, Informational, 1997
• Internet Message Access Protocol (IMAP) Capabilities Registry
• RFC 4315 - Internet Message Access Protocol (IMAP) - UIDPLUS extension, Proposed STD, 2005
• RFC 6855 - IMAP Support for UTF-8, Proposed STD, 2013
• RFC 3629 - UTF-8, a transformation format of ISO 10646, Section 4 Syntax of UTF-8 Byte Sequences, STD, 2003
• RFC 2971 - IMAP4 ID extension, Proposed STD, 2000
• RFC 4466 - Collected Extensions to IMAP4 ABNF, Proposed STD, 2006
• RFC 2595 - Using TLS with IMAP, POP3 and ACAP, Proposed STD, 1999
• RFC 2234 - Augmented BNF for Syntax Specifications: ABNF, Proposed STD, 1997
• RFC 822 - Standard for the Format of ARPA Internet Text Messages, Section 2 Notational Conventions - as a reference for pre-RFC2234 ABNF constructs, STD, 1982
• RFC 5234 - Augmented BNF for Syntax Specifications: ABNF, STD, 2008
• RFC 2822 - Internet Message Format, Proposed STD, 2001
• RFC 2047 - MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text, Section 4 Encodings, Draft STD, 1996
• RFC 2045 - Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies, Section 6.8 Base64 Content-Transfer-Encoding, Draft STD, 1996
• RFC 2231 - MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations, Proposed STD, 1997
• RFC 1766 - Tags for Identifying Language, Proposed STD, 1995
• RFC 5646 - Tags for Identifying Language, BCP 47, 2009
• RFC 3348 - The Internet Message Action Protocol (IMAP4) Child Mailbox Extension, Informational, 2002