Response body data buffering and callback bug

[cccs-sadugas]

We found a bug where the response body data callback is getting called repeatedly with the same (ever growing) data.

Internally, I think this occurs when RES_LINE state treats incoming data as a response body because it can't find a valid response line.

Steps to reproduce

<<<
1
2
<<<
3
4

Debug output

htp_connp_res_data(connp->out_status 1)

htp_connp_res_data: ptr 0xFFFFFFFF2664 offset 0 len 3
0lx  31 0a 32                                          |1.2|

htp_connp_res_data: out state=RES_IDLE, progress=NULL
[LOG] Unable to match response to request
picked up response w/o requesthtp_connp_res_data: out state=RES_LINE, progress=RES_LINE

htp_connp_RES_LINE: ptr 0xFFFFFFFF2664 offset 0 len 2
0lx  31 0a                                             |1.|


htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2664 offset 0 len 2
0lx  31 0a                                             |1.|

htp_connp_res_data: out state=RES_LINE, progress=RES_LINE
htp_connp_res_data: returning HTP_STREAM_DATA
htp_connp_res_data(connp->out_status 9)

htp_connp_res_data: ptr 0xFFFFFFFF266c offset 0 len 3
0lx  33 0a 34                                          |3.4|

htp_connp_res_data: out state=RES_LINE, progress=RES_LINE

htp_connp_RES_LINE: ptr 0xFFFFFFFF2f80 offset 0 len 3
0lx  32 33 0a                                          |23.|


htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 3
0lx  32 33 0a                                          |23.|

htp_connp_res_data: out state=RES_LINE, progress=RES_LINE
htp_connp_res_data: returning HTP_STREAM_DATA
htp_connp_req_data(connp->in_status 2)

htp_connp_req_data: ptr NULL len 0
htp_connp_req_data: in state=REQ_FINALIZE, progress=NOT_STARTED

PROBING request finalize: ptr NULL len 0
htp_connp_req_data: in state=REQ_IDLE, progress=NULL
htp_connp_req_data: returning HTP_STREAM_DATA
htp_connp_res_data(connp->out_status 2)

htp_connp_res_data: ptr NULL len 0
htp_connp_res_data: out state=RES_LINE, progress=RES_LINE

htp_connp_RES_LINE: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|


htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|

htp_connp_res_data: out state=RES_BODY_FINALIZE, progress=RES_BODY

PROBING response finalize: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|

[LOG] Unexpected response body

htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|

htp_connp_res_data: out state=RES_BODY_FINALIZE, progress=RES_BODY

PROBING response finalize: ptr NULL len 0

htp_tx_res_process_body_data_ex: ptr NULL len 0
htp_connp_res_data: out state=RES_IDLE, progress=NULL
htp_connp_res_data: returning HTP_STREAM_DATA

In short, notice these lines:

htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2664 offset 0 len 2
0lx  31 0a                                             |1.|
htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 3
0lx  32 33 0a                                          |23.|
htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|
htp_tx_res_process_body_data_ex: ptr 0xFFFFFFFF2f80 offset 0 len 4
0lx  32 33 0a 34                                       |23.4|
htp_tx_res_process_body_data_ex: ptr NULL len 0

[cccs-sadugas]

Opening PR for this soon.

[catenacyber]

@cccs-sadugas could you close this issues since the PR has been merged ? Thanks