Fuzzed requests not being sent

[lobax]

I think there might be some issues with malformed requests not being sent through the request library, there are a bunch of reports without response info and I can't find those outgoing requests at all with wireshark.

If you run the test application for instance, you will get the following report for test case 80:

{
  "status": "ZmFpbGVk", 
  "name": "dGFyZ2V0", 
  "sub_reports": [], 
  "test_number": 80, 
  "state": "Q09NUExFVEVE", 
  "reason": "dGFyZ2V0"
}

As you can see, no response. It could be that this request causes some error on the server such that it doesn't reply, but I cannot even find outgoing traffic for these requests when inspecting the traffic with WireShark.

This is the output from the fuzzer while it runs:

[INFO    ][base._test_info] Current test: 80
25790 [INFO] kitty: Current test: 80
25790 [DEBUG] root: Transmit: {'url': b'get', 'method': b'GET', 'headers': {'get|get|headerparam': 'Lm2Ҁ'}}
25790 [INFO] root: Request URL : http://127.0.0.1:5000/get
25790 [DEBUG] root: Request kwargs:{'headers': {'get|get|headerparam': 'Lm2Ҁ'}}, url: http://127.0.0.1:5000/get, method: GET
25790 [DEBUG] urllib3.connectionpool: Starting new HTTP connection (1): 127.0.0.1:5000
25790 [WARNING] root: Failed to parse http response code, exception occurred
25790 [WARNING] root: Test 80 status: failed
25790 [WARNING] root: Reason: target
[WARNING ][base._post_test] !! Failure detected !!
25790 [WARNING] kitty: !! Failure detected !!

[KissPeter]

This is the issue:
25790 [WARNING] root: Failed to parse http response code, exception occurred
The requests are sent, but the tested application just fails. If you use the test application in the development branch, that is a more robust one which works as it should be ( report internal error if fuzz string is received instead of integer).
Once I fixed the test application I started getting such logs:

29138 [WARNING] root: Return code 500 is not in the expected list
29138 [ERROR] root: Failed to save report "<kitty.data.report.Report object at 0x7f563edf9160>" to /tmp/reports/ because: byte must be in range(0, 256)

This is where we are at the moment.

[KissPeter]

Sorry, you were right.
During investigating the cause of the missing report fields I've found this:
4871 [INFO] root: Request URL : http://127.0.0.1:5000/other_methods
4871 [DEBUG] urllib3.connectionpool: Starting new HTTP connection (1): 127.0.0.1
4871 [ERROR] root: Request failed, reason: 'latin-1' codec can't encode character '\u0480' in position 3: ordinal not in range(256)

[lobax]

Simple solution is to catch these errors and just report this as a failure of the request library? It makes sense that the fuzzer might sometimes generate output that the request library can't handle.

[KissPeter]

I did exactly what you described. My plan on this issue is to check werther pycurl can handle is better. If so I can change using that.

[lobax]

I'm currently getting empty reports except when there is an issue with sending the request.

[KissPeter]

works now, tested

[KissPeter]

Hi,

I managed to change to pycurl where it is easier to compile requiest which then will be sent. It should be OK now.

[KissPeter]

it is resolved now