HTTP tables

Tempesta FW's HTTP tables is an extension of standard Linux iptables, nftables and/or bpfilter for the network application layer, HTTP(S) protocol in particular. The network layers, layer 7 for the HTTP tables and layers 3 & 4 for iptables/nftables/bpfilter, are integrated by netfilter packets marking. Using HTTP tables glued with iptables/nftables/bpfilter by mark, you can define complex and efficient multi-layer rules for load balancing, Web security, and DDoS mitigation.

HTTP tables look inside of an HTTP request and examines it's contents such as URI and headers to find out the target virtual host. The work of HTTP tables is represented by a list of linked HTTP chains. Each chain contains pattern-matching rules that map certain header field values to virtual host.

The syntax for rule chains

The full syntax is as follows:

http_chain NAME {
    [ FIELD [FIELD_NAME] == (!=) ARG ] -> ACTION [ = VAL];
    ...
}

NAME: Unique identifier of HTTP chain

FIELD is an HTTP request field such as uri, host, etc. Supported FIELD keywords:

• uri - only a part of URI is looked at that contains the path and the query string if any (e.g. "/abs/path.html?query&key=val#fragment").
• host - the host part from URI in HTTP request line, or the value of "Host" header field or value of "host" parameter from "Forwarded" header field. The Host part in URI takes priority over the Host header field value and "host" parameter from "Forwarded" header.
• method - HTTP request method. Supported ARG values for this field are: "copy", "delete", "get", "head", "lock", "mkcol", "move", "options", "patch", "post", "propfind", "proppatch", "put", "trace", "unlock", "purge".
• hdr - the content of specific HTTP header. In this case FIELD_NAME field is used to specify the name of header; the value of header should be specified in ARG. Matching of special headers: "X-Forwarded-For", "If-None-Match", "Referer", "Cookie", "User-Agent", "Content-Type", "Connection", "Host" - is accelerated; processing of other headers may be slow as it requires walking over all headers of an HTTP request. Also, suffix OP is not supported for not special headers.
• mark - the value of netfilter mark of request's skb.
• cookie - name or value of specific cookie. In this case FIELD_NAME files is used to specify cookie name or pattern; the value of the cookie or value pattern should be specified in ARG. Supported strict matching (no pattern), prefix or suffix patterns.

In configuration example below

http_chain {
    mark == 1 -> app;
    ...

means that all requests with netfilter MARK set to 1 must be passed to virtual host app. Netfilter MARK can be set via iptables command line interface:

iptables -t mangle -A PREROUTING -p tcp -s 192.168.1.1 -j MARK --set-mark 1

Command specified above means that all tcp packets arrived from 192.168.1.1 address, must be marked with 1 value (MARK is set in PREROUTING netfilter chain).

ARG is an argument (such as "/foo/bar.html", "example.com", etc.) for comparison with value extracted for FIELD_NAME field. The type of comparison for FIELD and ARG depends on == (!=) sign and on wildcard existence in ARG:

• ==: "ARG" => eq, "ARG*" => eq_prefix, "*ARG" => eq_suffix.
• !=: "ARG" => non_eq, "ARG*" => non_eq_prefix, "*ARG" => non_eq_suffix.

Types of comparison operations:

• eq - FIELD is fully equal to the string specified in ARG.
• non_eq - FIELD is not equal to the string specified in ARG.
• eq_prefix - FIELD starts with the string specified in ARG.
• non_eq_prefix - FIELD doesn't starts with the string specified in ARG.
• eq_suffix - FIELD ends with the string specified in ARG.
• non_eq_suffix - FIELD doesn't ends with the string specified in ARG.

Wildcard inside of ARG has no any special meaning and is a regular symbol. Backslash can be used in ARG - during configuration processing it is just dropped, leaving escaped symbol in ARG (e.g. \\ means regular simple backslash symbol \). If wildcard (at the beginning or end of ARG) is escaped, then it will not have any special prefix/suffix meaning and will be used as a regular symbol. Also HTTP tables cannot match double-wildcard patterns *ARG*, so prefix pattern automatically applied in such cases.

ACTION is a rule action with appropriate type; possible types are:

• vhost reference - rule with such action pass the request to specified virtual host (must be defined earlier in configuration file).
• chain reference - rule redirects request to other HTTP chain (must be defined earlier and must not be the same as current).
• mark - rule set netfilter marks into all skbs for all matched requests.
• block - rule blocks all matched requests.
• cache_disable - rule set caching policy for matching request.
• cache_ttl - rule to modify default TTL for a reponse in the cachedb.
• a redirection status code (301, 302, 303, 307, 308).

VAL is possible value for specified action. Supported:

• mark action is allowed to have unsigned integer value.
• cache_disable action is allowed to be 1 (do not cache) or 0 (honour all other configured caching policies, efficiently cancelling out effect of previously matched cache_disable directive). For details on caching look at Caching-Responses wiki page. Note: cache_disable = 1 can be abbreviated to just cache_disable.
• cache_ttl default cache ttl for the matched rule in seconds (overrides effect of the global cache_ttl directive) ONLY when upstream hasn't explicitly specified lifetime of an item.
• redirection action must have a target URI as a value.

An example of a redirection rule (note usage of $host and $request_uri variables):

http_chain redirects {
    # Absolute path
    uri == "/company.html" -> 301 = https://tempesta-tech.com/company;

    # Use variables
    uri == "/blog/" -> 308 = https://$host/$request_uri/;

    # Relative path
    uri == "*/services.html" -> 303 = /services;

    # Temporal redirection for the default index page only to a temporal landing page.
    uri == "/" -> 307 = /c++-services;
}

Note: all http_chain directives must be defined after definition of all virtual hosts; configuration order srv_groups/servers -> vhosts -> http_chains is mandatory.

Rule entry is a single instruction for HTTP table that says: take the FIELD of http request, compare it with ARG. If they match, then apply rule ACTION (with possible VAL) to that request.

The chains and rules

The rules processing

For every HTTP request, HTTP table executes all rule instructions in all linked HTTP chains (beginning from the main chain) sequentially until it finds a match.

Note: rules with mark conditions (in which FIELD == mark) are processed before any other rules in each configured HTTP chain. So, even if user placed such rules in the end of HTTP chain - they nevertheless will be processed at the beginning of that chain.

Default rules and chains

If during a chain processing no matching rule is found for a request, then the request is dropped. For example, any request passing through the chain

http_chain strict_match {
    hdr "referer" == "http://good.com" -> some_host;
}

and having Referer header not equal to good.com or not having the header at all, will be blocked.

A default match rule can be specified. Its syntax looks like this:

    -> ACTION;

This rule works as last resort option, and, if specified, it applies designated action to requests that didn't match any more specific rule. As all match rules are processed in sequential order, this rule must come last to serve the intended role.

One main HTTP chain (without name) must be specified after all other chains in configuration file, e.g.

http_chain l7_rules {
    hdr "Referer" == "http://badhost.com*" -> block;
    -> my_hostname;
}

http_chain {
    mark == 1 -> l7_rules;
    -> my_hostname;
}

The example of a wrong configuration:

http_chain {
    mark == 1 -> l7_rules;
    -> my_hostname;
}

http_chain l7_rules {
    hdr "Referer" == "http://badhost.com*" -> block;
    -> my_hostname;
}

If no main chain is specified, it is created implicitly. In this case one default match rule pointing to default virtual host will be created in implicit main chain if default virtual host is present in configuration and if such default rule (with default virtual host) have not been specified explicitly in any chain in configuration. An example of a configuration with an implicit main chain:

listen 192.168.100.4:80;

srv_group default {
    server 127.0.0.1:9090;
}

vhost default {
    proxy_pass default;
}

In this example the implicitly created the main chain looks as

http_chain {
    -> default;
}

User can explicitly create the main HTTP chain with empty list of rules, which means the complete absence of rules - all incoming requests will be dropped in such configuration, i.e. in this configuration:

listen 192.168.100.4:80;

srv_group default {
    server 127.0.0.1:9090;
}

vhost default {
    proxy_pass default;
}

http_chain {
}

Tempesta FW will drop all the requests.

Below are examples of HTTP table configuration:

srv_group static { ... }
srv_group nts_app { ... }
srv_group foo_app { ... }
srv_group bar_app { ... }
srv_group default { ... }

vhost base {
    proxy_pass static;
    ...
}
vhost app {
    proxy_pass default;
    location prefix "?" {
        proxy_pass foo_app;
        ...
    }
    location prefix "/static/" {
        proxy_pass static;
        ...
    }
    ...
}
vhost heavy {
    proxy_pass bar_app backup=foo_app;
    ...
}
vhost nts {
    proxy_pass nts_app backup=bar_app;
    ...
}

http_chain {
    -> mark = 7;
    -> heavy;
}
http_chain stat {
    -> mark = 6;
    -> base;
}
http_chain natsys {
    host == "static.*"    -> stat;
    host == "*.example.com" -> example;
    -> mark = 5;
    -> nts;
}
http_chain {
    mark                    == 1                     -> app;
    mark                    == 2                     -> block;
    hdr "Connection"        == "keep-alive"          -> heavy;
    hdr "Host"              == "bar.*"               -> heavy;
    hdr "Host"              == "*natsys-lab.com"     -> natsys;
    hdr "Host"              == "bar.natsys-lab.com"  -> mark = 3;
    hdr "X-Custom-Bar-Hdr"  == "*"                   -> mark = 4;
    uri                     == "*.php"               -> app;
    host                    == "static.*"            -> app;
    host                    == "*tempesta-tech.com"  -> base;
    host                    == "foo.example.com"     -> base;
                                                     -> app;
}

Reaction to server unavailability. HTTP tables doesn't work with servers, so the reaction depends only on load balancing schedulers attached to the groups.