Skip to content

SPOE: Stream Processing Offloading Engine

Jump to bottom
William Lallemand edited this page Apr 19, 2024 · 28 revisions

What is SPOE ?

HAProxy's Stream Processing Offloading Engine is a technic which allows expanding HAProxy capabilities by calling Agents at different step of processing the traffic. Agents are standalone software and can be written in any language, as long as there is a SPOE library for it. HAProxy and Agents will use TCP connections to exchange information.

Why would I want to use SPOE?

First, HAProxy's internal way of working (high performance event driven model) is not compatible with some processing, such as "blocking" syscalls or libraries or heavy computation. These two types of processing will block or drastically slow down HAProxy event loop, hence slow down the whole traffic passing through the thread where said processing is happening. SPOE helps in both situations because it can be used to perform this blocking call or heavy processing in a third party software without impacting HAProxy performance at all.

How does it work exactly?

HAProxy proposes different "hooks" in the traffic processing, allowing you to:

  • hang the traffic
  • collect data
  • call the agent
  • [process other traffic]
  • read agent's response
  • carry on processing this request

These hooks can happen at the request or response phase.

In a picture:

+--------------+
|   Client     |
+------+-------+
       ^
       |
       |
       v
+------+-------+                    +---------+
|   HAProxy   SPOE<---------------->+  Agent  |
+------+-------+                    +---------+
       ^
       |
       |
       |
       v
+--------------+
|   Server     |
+--------------+

Supported language

Languages with a SPOE library:

Language Url Status
.Net haproxytech/haproxy-spoa-dotnet beta
.Net inulogic/HAProxy.StreamProcessingOffload.AgentFramework 1.0.1
C haproxy/spoa-example beta
Golang criteo/haproxy-spoe-go unmaintained
Golang go-spop/spoe (fork of criteo's)
Golang negasus/haproxy-spoe-go 1.0.3
Lua haproxy/spoa-server beta
Python haproxy/spoa-server beta
Rust vkill/haproxy-spoa-example

Available SPOE Agents for HAProxy

Name Description Status
haproxytech/spoa-mod_defender whitelist based Web Application Firewall (WAF) that uses NAXSI compatible rules configuration format. Due to current SPOE limitation, the analyse is limited to one tune.bufsize and on the request side only not production ready
haproxy/spoa-modsecurity Modsecurity v2 Web Application Firewall (WAF). Due to current SPOE limitation, the analysis is limited to one tune.bufsize and on the request side only not production ready
rikatz/spoa-modsecurity-python Modsecurity v3 Web Application Firewall (WAF). Due to current SPOE limitation, the analyse is limited to one tune.bufsize and on the request side only beta
haproxytech/opentracing send data to distributed tracing systems via the OpenTracing API beta
haproxytech/spoa-mirror Traffic mirroring beta
haproxytech/consul-connect Consul connect "intention" validation beta

Links

SPOE history

Clone this wiki locally