Implement support for PROXY Protocol (#246)

* Implement Logic & Test for PROXY protocol * Take PROXY protocol detection out of main _handle_client loop * Extract PROXY handling to its own module * Remove unused class & constant * Add constants for protocol * Implement testing for v2 * Implement testing for ProxyData class itself * Make sure all copyrightable files have license snippet * Catch exception during parsing * Successful use of attrs * Add random test for TCP4 * Make alias for hairy-looking markers * Combine some IPv6 tests into a parametrized one * Implement TLV parser * V2_ constants are now Enums * Add tlv prop to ProxyData * Add tests for tlv prop of ProxyData * Change import * to explicit * Record proxy data in session * Add annotation for SMTP.session and SMTP.envelope * Change _proxy_result to session.proxy_data * Docstring for Session.proxy_data * Fix typo in __slots__ * Change equality test to None * Remove special env flags for Python Version * Remove unused try..except * Add small test for coverage 100% * Add test for tlv property in ProxyData * Defensive assert to ensure PROXY protocol activated * Workaround for ConnectionAborted not raised in Linux * Add test that server is still connectable after timeout * Add some asyncio.sleep() in smtp.py to ensure all transport.close() related processing have taken place * Add test for streaming hiccup * Extract test contants to module * Add test abt extraneous non PROXYv2 data * Add some PROXYv2 tests * Add timeout test for v2 * Update NEWS.rst * Also okay if ConnectionResetError * Tune CSS * Title Case * Change :boldital: to full-fledged py:attribute:: * Update concepts.rst with new attribs in Session * Deprecate Session.login_data * Move _static into docs * More CSS Tuning * Now uses a lot of Sphinx's py domain roles * This reactivates modindex * The whole AUTH system moved to its own page because it's getting unwieldy * Public classes/functions added to several pages * Fetch master only if not in master * Make sure that "run" with "if" is by bash * Redo ProxyTLV class * Now inherits from dict * parse() extracted from from_raw() * parse() can now do partial parsing * New name_to_num() classmethod * TestProxyProtocol{V1,V2}Controller are merged into a common Test Class with parametrized test cases * ... and use a better name for the class * Add simple test for Accept/Reject based on handle_PROXY return value * Also add test for new name_to_map() classmethod * Add logging to PROXY Protocol handling in smtp.py * Add checks on proxy_protocol_timeout values * Add documentation * Fix test for log entry * Add ability to raise exception * General Improvement on ProxyTLV * Rewrite parse() to use Accumulator Pattern. This is needed to properly implement "partial parsing" ability * parse() now also generates "TLV Location". This is to support CRC32C calculation (even though CRC32C calculation will NOT be implemented in proxy_protocol.py due to deps restriction) * parse() now gains "strict" parameter * New UnknownTypeTLV exception to support parse(strict=True) * from_raw() adapted to new parse() retval + gains "strict" param, too * __init__() now requires _tlv_loc * __slots__ corrected * Add test cases for "partial parsing" and "strict" * Add whole_raw attrib to ProxyData * Add test to ensure no log.warning for timeout >= 3.0 * Add reflink to standard doc * Slight tidying up of test data * Fix qa complaint * Increase Timeout Multiplier to 1.5 * Expect raising of ConnectionError * Fix pytype complaints * Fix LGTM Alert * Fix duplicate rst ref in docs * Fix NEWS.rst formatting * Add tlv_start attribute to ProxyData * Add doc about CRC32C Calculation / Validation * Use the desired CaseStyling for products * Use v2.3.0 tag instead of SHA hash * Also record whole_raw for PROXYv1 * Normalize AF and PROTO for v1 & v2 * Add _raises to ProxyData.same_attribs * Add test of public PROXYv1 examples * Reword about HAProxy v2.3.0 * Use internal func to get a field piecemeal using read() instead of readexactly(). This allows rapid response to Connection Lost instead of waiting for timeout. Also, introduces more points where cooperative concurrency context switching can take place. * Pepper the func with lots of log.debug to help troubleshooting. * Rename "rest" to "tail_part" * Totally rewrite _get_v1 * Add option to skip adding prefix to log message * A bunch of documentation/annotation improvements * Extend the "hiccup" delay * Remove a pragma * Use log.warning instead of warnings.warn * Remove unused f * Implement 4th arg for handle_PROXY * Add test for get_proxy unrecognized signature * Rename some methods and classes * Up version to 1.4.0a1 * Remove last vestiges of aiosmtpd.smtp.__version__ * Add Unicode accoutrements to housekeep.py
aio-libs · Feb 21, 2021 · 4336a05 · 4336a05
1 parent cfd7cf6
commit 4336a05
Show file tree

Hide file tree

Showing 18 changed files with 2,253 additions and 44 deletions.
diff --git a/aiosmtpd/__init__.py b/aiosmtpd/__init__.py
@@ -1,4 +1,4 @@
 # Copyright 2014-2021 The aiosmtpd Developers
 # SPDX-License-Identifier: Apache-2.0
 
-__version__ = "1.3.2"
+__version__ = "1.4.0a1"
diff --git a/aiosmtpd/__main__.py b/aiosmtpd/__main__.py
@@ -1,3 +1,6 @@
+# Copyright 2014-2021 The aiosmtpd Developers
+# SPDX-License-Identifier: Apache-2.0
+
 from aiosmtpd.main import main
 
 if __name__ == '__main__':

diff --git a/aiosmtpd/docs/NEWS.rst b/aiosmtpd/docs/NEWS.rst
@@ -3,6 +3,17 @@
 ###################
 
 
+1.4.0 (aiosmtpd-next)
+=====================
+
+Added
+-----
+* Support for |PROXY Protocol|_ (Closes #174)
+
+.. _`PROXY Protocol`: https://www.haproxy.com/blog/using-haproxy-with-the-proxy-protocol-to-better-secure-your-database/
+.. |PROXY Protocol| replace:: **PROXY Protocol**
+
+
 1.3.2 (2021-02-20)
 ==================
 
@@ -18,7 +29,6 @@ Fixed/Improved
 * Update PyPI Long Description
 
 
-
 1.3.1 (2021-02-18)
 ==================
 
@@ -51,7 +61,7 @@ Fixed/Improved
   (It gets reimported to ``aiosmtpd.smtp``,
   so programs relying on ``aiosmtpd.smtp.__version__`` should still work.)
   (Closes #241)
-* Uses pure ``pytest`` for all test cases
+* Uses pure ``pytest`` for all test cases (Closes #198)
 
 
 1.2.4 (2021-01-24)

diff --git a/aiosmtpd/docs/conf.py b/aiosmtpd/docs/conf.py
@@ -13,8 +13,7 @@
 # serve to show the default.
 
 import datetime
-import re
-import sphinx_rtd_theme  # noqa: F401
+import sphinx_rtd_theme  # noqa: F401 # pytype: disable=import-error
 import sys
 
 from pathlib import Path
@@ -28,9 +27,6 @@
     colorama_init = None
 
 
-RE__VERSION = re.compile(r"""__version__ = (['"])(?P<ver>[^'"]+)(\1)""")
-
-
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.

diff --git a/aiosmtpd/docs/handlers.rst b/aiosmtpd/docs/handlers.rst
@@ -184,6 +184,19 @@ The following hooks are currently supported (in alphabetical order):
 
    Called during ``NOOP``.
 
+.. method:: handle_PROXY(server, session, envelope, proxy_data)
+   :noindex:
+
+   :param SMTP server: The :class:`SMTP` instance invoking the hook.
+   :param Session session: The Session data *so far* (see Important note below)
+   :param Envelope envelope: The Envelope data *so far* (see Important note below)
+   :param ProxyData proxy_data: The result of parsing the PROXY Header
+   :return: Truthy or Falsey, indicating if the connection may continue or not, respectively
+
+   Called during PROXY Protocol Handshake.
+
+   See :ref:`ProxyProtocol` for more information.
+
 .. py:method:: handle_QUIT(server, session, envelope) -> str
    :async:
 

diff --git a/aiosmtpd/docs/index.rst b/aiosmtpd/docs/index.rst
@@ -15,6 +15,7 @@ Contents
    lmtp
    handlers
    auth
+   proxyprotocol
    migrating
    testing
    manpage