Skip to content

Commit

Permalink
Merge branch 'master' into quic-latest
Browse files Browse the repository at this point in the history
* master:
  Fix a crash on active timeout on QUIC connections (apache#7059)
  Don't make an error on receiving retransmitted handshake data (apache#7061)
  Document proxy.config.http.cache.post_method. (apache#7060)
  Quote out lists of servers and domains in splitdns.config example (apache#7057)
  Fix proxy.process.http.current_client_connections (apache#7056)
  Fixed CLIENT-URL to use the pristine client URL (apache#7050)
  Removes FIXME that is unlikely to be fixed at this point in the project history (apache#7058)
  Move to denylists and allowlists (apache#7034)
  Avoid unnecessary copying of STL map for QUICTPConfigQCP class. (apache#7039)
  • Loading branch information
maskit committed Jul 31, 2020
2 parents 50937c0 + 3087f16 commit 84e4c8e
Show file tree
Hide file tree
Showing 42 changed files with 352 additions and 232 deletions.
36 changes: 18 additions & 18 deletions doc/.tx/config
Original file line number Diff line number Diff line change
Expand Up @@ -1922,34 +1922,34 @@ file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/basic-authorization/working-with-http-headers.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--accessing-the-transaction-being-processed_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/accessing-the-transaction-being-processed.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/accessing-the-transaction-being-processed.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--accessing-the-transaction-being-processed_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/accessing-the-transaction-being-processed.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/accessing-the-transaction-being-processed.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--index_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/index.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/index.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--index_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/index.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/index.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--setting-a-global-hook_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/setting-a-global-hook.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/setting-a-global-hook.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--setting-a-global-hook_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/setting-a-global-hook.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/setting-a-global-hook.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--setting-up-a-transaction-hook_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/setting-up-a-transaction-hook.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/setting-up-a-transaction-hook.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--setting-up-a-transaction-hook_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/setting-up-a-transaction-hook.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/setting-up-a-transaction-hook.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--source-code_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/source-code.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/source-code.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--source-code_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/source-code.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/source-code.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--blacklist--working-with-http-header-functions_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/blacklist/working-with-http-header-functions.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/blacklist/working-with-http-header-functions.en.pot
[apache-traffic-server-6x.developer-guide--plugins--example-plugins--denylist--working-with-http-header-functions_en]
file_filter = locale/<lang>/LC_MESSAGES/developer-guide/plugins/example-plugins/denylist/working-with-http-header-functions.en.po
source_file = _build/locale/pot/developer-guide/plugins/example-plugins/denylist/working-with-http-header-functions.en.pot
source_lang = en

[apache-traffic-server-6x.developer-guide--plugins--example-plugins--query-remap--example-query-remap_en]
Expand Down
4 changes: 2 additions & 2 deletions doc/admin-guide/configuration/proxy-protocol.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ Proxy Protocol on a port. Once enabled, all incoming requests must be prefaced
with the PROXY v1 header. Any request not preface by this header will be
dropped.

As a security measure, an optional whitelist of trusted IP addresses may be
As a security measure, an optional list of trusted IP addresses may be
configured with :ts:cv:`proxy.config.http.proxy_protocol_allowlist`.

.. important::

If the whitelist is configured, requests will only be accepted from these
If the allowlist is configured, requests will only be accepted from these
IP addresses and must be prefaced with the PROXY v1 header.

See :ts:cv:`proxy.config.http.insert_forwarded` for configuration information.
Expand Down
6 changes: 6 additions & 0 deletions doc/admin-guide/files/records.config.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1945,6 +1945,12 @@ Cache Control

Enables (``1``) or disables (``0``) caching of HTTP requests.

.. ts:cv:: CONFIG proxy.config.http.cache.post_method INT 0
:reloadable:
:overridable:

Enables (``1``) or disables (``0``) caching of HTTP POST requests.

.. ts:cv:: CONFIG proxy.config.http.cache.generation INT -1
:reloadable:
:overridable:
Expand Down
2 changes: 1 addition & 1 deletion doc/admin-guide/files/splitdns.config.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Examples

Consider the following DNS server selection specifications: ::

dest_domain=internal.company.com named=255.255.255.255:212 255.255.255.254 def_domain=company.com search_list=company.com company1.com
dest_domain=internal.company.com named="255.255.255.255:212 255.255.255.254" def_domain=company.com search_list="company.com company1.com"
dest_domain=!internal.company.com named=255.255.255.253

Now consider the following two requests: ::
Expand Down
22 changes: 11 additions & 11 deletions doc/admin-guide/plugins/cachekey.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ Cache key structure and related plugin parameters

* ``User-Agent`` classification
* ``--ua-allowlist=<classname>:<filename>`` (default: empty string) - loads a regex patterns list from a file ``<filename>``, the patterns are matched against the ``User-Agent`` header and if matched ``<classname>`` is added it to the key.
* ``--ua-blocklist=<classname>:<filename>`` (default: empty string) - loads a regex patterns list from a file ``<filename>``, the patterns are matched against the ``User-Agent`` header and if **not** matched ``<classname>`` is added it to the key.
* Multiple ``--ua-allowlist`` and ``--ua-blocklist`` can be used and the result will be defined by their order in the plugin configuration.
* ``--ua-denylist=<classname>:<filename>`` (default: empty string) - loads a regex patterns list from a file ``<filename>``, the patterns are matched against the ``User-Agent`` header and if **not** matched ``<classname>`` is added it to the key.
* Multiple ``--ua-allowlist`` and ``--ua-denylist`` can be used and the result will be defined by their order in the plugin configuration.
* ``User-Agent`` regex capturing and replacement
* ``--ua-capture=<capture_definition>`` (default: empty string) - if specified and not empty then strings are captured from the ``User-Agent`` header based on ``<capture_definition>`` (see below) and are added to the `cache key`.
* If any ``User-Agent`` classification and regex capturing and replacement plugin parameters are used together they are added to the `cache key` in the order shown in the diagram.
Expand Down Expand Up @@ -179,10 +179,10 @@ Cache key structure and related plugin parameters
^^^^^^^^^^^^^^^

* If no query related plugin parameters are used, the query string is included in the `cache key`.
* ``--exclude-params`` (default: empty list) - comma-separated list of query params to be black-listed in the `cache key`. If the list is empty then no black-list is applied (no query parameters will be excluded from the `cache key`). The exclude list overrides the include list.
* ``--include-params`` (default: empty list) - comma-separated list of query params to be white-listed in the `cache key`. If the list is empty then no white-list is applied (all query parameters will be included in the `cache key`).
* ``--include-match-params`` (default: empty list) - regular expression matching query parameter names which will be white-listed in the `cache key`.
* ``--exclude-match-params`` (default: empty list) - regular expression matching query parameter names which will be black-listed in the `cache key`.
* ``--exclude-params`` (default: empty list) - comma-separated list of query params to be excluded in the `cache key`. If the list is empty then no exlusions are applied (no query parameters will be excluded from the `cache key`). The exclude list overrides the include list.
* ``--include-params`` (default: empty list) - comma-separated list of query params to be allow-listed in the `cache key`. If the list is empty then no allow-list is applied (all query parameters will be included in the `cache key`).
* ``--include-match-params`` (default: empty list) - regular expression matching query parameter names which will be allowed in the `cache key`.
* ``--exclude-match-params`` (default: empty list) - regular expression matching query parameter names which will be excluded in the `cache key`.
* ``--remove-all-params`` (boolean:``true|false``, ``0|1``, ``yes|no``, default: ``false``) - if equals ``true`` then all query parameters are removed (the whole query string) and all other URI query parameter related settings (if used) will have no effect.
* ``--sort-params`` (boolean:``true|false``, ``0|1``, ``yes|no``, default: ``false``) - if equals ``true`` then all query parameters are sorted in an increasing case-sensitive order

Expand Down Expand Up @@ -377,7 +377,7 @@ The following will make sure only query parameters ``a`` and ``c`` will be used

If the URI has the following query string ``c=1&a=1&b=2&x=1&k=1&u=1&y=1`` the `cache key` will use ``c=1&a=1``

White-list + black-list certain parameters using multiple parameters in the same remap rule.
Include and exclude certain parameters using multiple parameters in the same remap rule.
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
If the plugin is used with the following plugin parameters in the remap rule: ::

Expand All @@ -390,7 +390,7 @@ If the plugin is used with the following plugin parameters in the remap rule: ::

and if the URI has the following query string ``c=1&a=1&b=2&x=1&k=1&u=1&y=1`` the `cache key` will use ``c=1&b=1``

White-list + black-list certain parameters using multiple parameters in the same remap rule and regular expressions (PCRE).
Include and exclude certain parameters using multiple parameters in the same remap rule and regular expressions (PCRE).
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
If the plugin is used with the following plugin parameters in the remap rule: ::

Expand Down Expand Up @@ -570,7 +570,7 @@ If the plugin is used with the following plugin parameter::

then ``Mozilla/5.0_AppleWebKit/537.75.14`` will be used when constructing the key.

User-Agent white-list classifier
User-Agent allow-list classifier
""""""""""""""""""""""""""""""""
If the plugin is used with the following plugin parameter::

Expand All @@ -585,12 +585,12 @@ and if ``browser_agents.config`` contains: ::

then ``browser`` will be used when constructing the key.

User-Agent black-list classifier
User-Agent deny-list classifier
""""""""""""""""""""""""""""""""
If the plugin is used with the following plugin parameter::

@plugin=cachekey.so \
@pparam=--ua-blacklist=browser:tool_agents.config
@pparam=--ua-denylist=browser:tool_agents.config

and if ``tool_agents.config`` contains: ::

Expand Down
6 changes: 3 additions & 3 deletions doc/admin-guide/plugins/stats_over_http.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,16 +88,16 @@ This sets the path value for stats

.. option:: allow_ip=

A comma separated white list of ipv4 addresses allowed to access the endpoint
A comma separated list of IPv4 addresses allowed to access the endpoint

.. option:: allow_ip6=

A comma separated white list of ipv6 addresses allowed to access the endpoint
A comma separated list of IPv6 addresses allowed to access the endpoint

Output Format
=============

By default stats_over_http.so will output all the stats in json format. However
By default stats_over_http.so will output all the stats in JSON format. However
if you wish to have it in CSV format you can do so by passing an ``Accept`` header:

.. option:: Accept: text/csv
Expand Down
4 changes: 2 additions & 2 deletions doc/developer-guide/api/functions/TSAPI.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,11 @@ type.

Possible uses for plugins include the following:

* HTTP processing plugins can filter, blacklist, authorize users or transform content.
* HTTP processing plugins can filter, denylist, authorize users or transform content.

* Protocol plugins can enable Traffic Server to proxy-cache new protocol content.

* A blacklisting plugin denies attempts to access web sites that are off-limits.
* A denylisting plugin denies attempts to access web sites that are off-limits.

* Append transform plugins add data to HTTP response content.

Expand Down
4 changes: 2 additions & 2 deletions doc/developer-guide/introduction/index.en.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,15 +53,15 @@ Below is a section-by-section breakdown of this guide:
How to compile and load plugins. Walks through a simple "hello world"
example; explains how to initialize and register plugins. Basic structures
that all plugins use: events, continuations, and how to hook on to |TS|
processes. Detailed explication of a sample blacklisting plugin.
processes. Detailed explication of a sample denylisting plugin.

:ref:`developer-plugins-examples-query-remap`
This chapter demonstrates on a practical example how you can
exploit the Traffic Server remap API for your plugins.

:ref:`developer-plugins-header-based-examples`
Detailed explanation about writing plugins that work on HTTP
headers; discusses sample blacklisting and basic authorization
headers; discusses sample denylisting and basic authorization
plugins.

:ref:`developer-plugins-http-transformations`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
.. include:: ../../../../common.defs

.. _developer-plugins-blacklist-access-process-txn:
.. _developer-plugins-denylist-access-process-txn:

Accessing the Transaction Being Processed
*****************************************
Expand All @@ -38,12 +38,12 @@ The key here is that if the event is an HTTP transaction event, then the
data passed to the continuation's handler is of type ``TSHttpTxn`` (a
data type that represents HTTP transactions). Your plugin can then do
things with the transaction. Here's how it looks in the code for the
Blacklist plugin's handler:
Denylist plugin's handler:

.. code-block:: c
static int
blacklist_plugin (TSCont contp, TSEvent event, void *edata)
denylist_plugin (TSCont contp, TSEvent event, void *edata)
{
TSHttpTxn txnp = (TSHttpTxn) edata;
switch (event) {
Expand All @@ -60,5 +60,5 @@ Blacklist plugin's handler:
}
For example: when the origin server DNS lookup event is sent,
``blacklist_plugin`` can call ``handle_dns``\ and pass ``txnp`` as an
``denylist_plugin`` can call ``handle_dns``\ and pass ``txnp`` as an
argument.
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,21 @@
.. include:: ../../../../common.defs

.. _developer-plugins-examples-blacklist:
.. _developer-plugins-examples-denylist:

Blacklist Plugin
Denylist Plugin
****************

The sample blacklisting plugin included in the Traffic Server SDK is
``blacklist_1.c``. This plugin checks every incoming HTTP client request
against a list of blacklisted web sites. If the client requests a
blacklisted site, then the plugin returns an ``Access forbidden``
The sample denylisting plugin included in the Traffic Server SDK is
``denylist_1.c``. This plugin checks every incoming HTTP client request
against a list of listed web sites. If the client requests a
listed site, then the plugin returns an ``Access forbidden``
message to the client.

The flow of HTTP processing with the blacklist plugin is illustrated in
the figure titled :ref:`BlackListPlugin`.
The flow of HTTP processing with the denylist plugin is illustrated in
the figure titled :ref:`DenyListPlugin`.
This example also contains a simple configuration management interface.
It can read a list of blacklisted sites from a file (``blacklist.txt``)
It can read a list of sites from a file (``denylist.txt``)
that can be updated by a Traffic Server administrator. When the
configuration file is updated, Traffic Server sends an event to the
plugin that wakes it up to do some work.
Expand All @@ -52,7 +52,7 @@ Traffic Server has a multi-threaded design, race conditions can occur if
several threads try to access the same continuation's data.

Here is how the static parent continuation is created in
``blacklist_1.c``:
``denylist_1.c``:

.. code-block:: c
Expand All @@ -62,20 +62,20 @@ Here is how the static parent continuation is created in
// ...
TSCont contp;
contp = TSContCreate (blacklist_plugin, NULL);
contp = TSContCreate (denylist_plugin, NULL);
// ...
}
The handler function for the plugin is ``blacklist_plugin``, and the
The handler function for the plugin is ``denylist_plugin``, and the
mutex is null. The continuation handler function's job is to handle the
events that are sent to it; accordingly, the ``blacklist_plugin``
events that are sent to it; accordingly, the ``denylist_plugin``
routine consists of a switch statement that covers each of the events
that might be sent to it:

.. code-block:: c
static int
blacklist_plugin (TSCont contp, TSEvent event, void *edata)
denylist_plugin (TSCont contp, TSEvent event, void *edata)
{
TSHttpTxn txnp = (TSHttpTxn) edata;
switch (event) {
Expand All @@ -86,18 +86,18 @@ that might be sent to it:
handle_response (txnp);
return 0;
default:
TSDebug ("blacklist_plugin", "This event was unexpected: %d", );
TSDebug ("denylist_plugin", "This event was unexpected: %d", );
break;
}
return 0;
}
When you write handler functions, you have to anticipate any events that
might be sent to the handler by hooks or by other functions. In the
Blacklist plugin, ``TS_EVENT_OS_DNS`` is sent because of the global hook
Denylist plugin, ``TS_EVENT_OS_DNS`` is sent because of the global hook
established in ``TSPluginInit``, ``TS_EVENT_HTTP_SEND_RESPONSE_HDR`` is
sent because the plugin contains a transaction hook
(see :ref:`developer-plugins-examples-blacklist-txn-hook`).
(see :ref:`developer-plugins-examples-denylist-txn-hook`).
It is good practice to have a default case in your switch statements.

.. toctree::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Setting a Global Hook
Global hooks are always added in ``TSPluginInit`` using
``TSHttpHookAdd``. The two arguments of ``TSHttpHookAdd`` are the hook
ID and the continuation to call when processing the event corresponding
to the hook. In ``blacklist_1.c``, the global hook is added as follows:
to the hook. In ``denylist_1.c``, the global hook is added as follows:

.. code-block:: c
Expand All @@ -32,7 +32,7 @@ to the hook. In ``blacklist_1.c``, the global hook is added as follows:
Above, ``TS_HTTP_OS_DNS_HOOK`` is the ID for the origin server DNS
lookup hook and ``contp`` is the parent continuation created earlier.

This means that the Blacklist plugin is called at every origin server
DNS lookup. When it is called, the handler functio ``blacklist_plugin``
This means that the Denylist plugin is called at every origin server
DNS lookup. When it is called, the handler functio ``denylist_plugin``
receives ``TS_EVENT_HTTP_OS_DNS`` and calls ``handle_dns`` to see if the
request is forbidden.

0 comments on commit 84e4c8e

Please sign in to comment.