Body Filter: Also match on URL-encoded contents.

[typoon]

Problem Description

When filtering requests (~bq) I would expect that the request body would be urldecoded prior to applying the filter to it. This would allow the filter to find the expression "id" in a request body that looks like myjsonvar={%22id%22:%22123%22}.

Steps to reproduce the behavior:

1. Open mitmproxy on port 12345
2. Send the following two requests using curl:

curl -x 'http://localhost:12345' -d '{"id":"123"}' -H'Content-type: application/json' http://www.example.com/?test=1
curl -x 'http://localhost:12345' -d 'myjsonvar={%22id%22:%22123%22}' http://www.example.com/?test=2

3. Open the request that ends in ?test=2 and notice that the body is displayed urldecoded (you can see the double quotes between id)
4. Go back to the flow list and set the following filter '~bq \'"id"\''
5. Notice that only the request that ends in ?test=1 is displayed

System Information

$ mitmproxy --version
Mitmproxy: 5.0.0.dev (+444, commit bbb7eb6)
Python: 3.7.3
OpenSSL: OpenSSL 1.1.1d 10 Sep 2019
Platform: Linux-5.0.0-31-generic-x86_64-with-Ubuntu-19.04-disco

[mhils]

Hm, tricky. We currently just match on raw bytes, which at least has the benefit of being somewhat predictable. I think it would be fine to look at the request headers and possibly run the request content through urllib.parse.unquote_to_bytes.

[typoon]

I think so too. I will try to do this soon and see how it works.