Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
First class support for HAR #1477
Once we get HAR dumping support (#1464) I think we should make it an integral part of mitm rather than keeping it as a external example:
Paraphrasing some discussion I had on this on Slack with @mhils:
The best thing would be to have HAR support in FlowReader:
FlowReader/FlowWriter should be ABCs that define the reading/writing API and then MitmFlowReader/HarFlowReader as implementations.
Then we could have a
For now we just want to focus on HAR & not PCAP (though that is something nice too) but pcap2mitmproxy is GPLed so we can't include it right now.
I think we should keep our core FlowReader and FlowWriter concepts separate from exporters and importers.
Exporters are the more complicated case, and there are many semantic ways in which they don't fit with our generalised FlowReader/Writer scheme. HAR, for example, has to be terminated after the last flow has been written to make it valid JSON. I don't think we should build the termination concept into our core, because there streaming flows one by one independently is the key attribute. I can also imagine a range of exporters that do things like, for instance, calculate summaries over multiple flows (in fact, I'm working on beasts like this right now) that only write data after all flows are consumed. A further issue is that many exporters will only support exporting a single flow - i.e. curl command export. This again doesn't quite fit with flowreader/writer.
What I'd really like is a way to register exporters and importers a la contentviews, as Max described. Addons should easily be able to add custom exporters/importers, and, in fact, it might be a useful discipline to choose to implement the built-in exporters using the addons mechanism. For exporters, we should have an API that allows the exporter to indicate if it supports multiple-flow export, or only single-flow export.