Implement full streaming parsing

[ivanr]

Request and response body processing is currently fully streaming, meaning data callbacks are called as soon as data is available, and there's no buffering. However, there is buffering when it comes to request/response line and header processing.

In most cases, there's no practical difference between the two approaches, because request headers arrive in a single buffer. However, an enterprising attacker could split request data into small packets, and sneak by an IPS that relies on buffering, and which we will see the attack only once all the headers are assembled. By that time it will be too late.

Inspection of buffered data is easy to implement because the desired chunk of data is available at once. Implementing streaming increases the burden on the LibHTP user. Further, it is questionable whether it is possible to prevent such fragmented attacks reliably. In the worst case, for example, the attacker could be using one byte per packet and several hundred packets for all request headers.

We need to interview Suricata developers to determine if we should pursue full streaming.

[ivanr]

Removing enhancements from the issue tracker.