Updating mitmproxy allow-list on the fly

[chrhansen]

Hi!

Thanks for all the hard work and for the quick response time. I emailed @mhils about this question today, but also wanted to put it here, for anyone else with a similar question.

Question
I'm trying to figure out how to update the allow-hosts (scroll to the bottom) without restarting mitmproxy (as that would drop connections/momentarily block new connections) and found this place in the code where the allow-list seems to be read for each flow.

My question is: What would be the best option to update the allow-list without restarting mitmproxy/-dump? Should I modify the code so that the "allow_hosts"-iterable can be edited?

[mhils]

Updating allow-hosts during execution should generally work fine:

• From within mitmproxy, you can press O to update options interactively.
• If you want to do it programmatically, you can update mitmproxy.ctx.options.allow_hosts from within your addon (https://docs.mitmproxy.org/stable/addons-overview/).
  Here's a somewhat related example from a few days ago: https://gist.github.com/mhils/70b088bf320e6c249270d6a3e3c4feb8

[chrhansen]

There's something I don't quite get and I see now that my initial question

I'm trying to figure out how to update the allow-hosts (scroll to the bottom) without restarting mitmproxy

wasn't clear on that. I should have said "without restarting mitmdump".

allow-hosts can be

• set on the fly/interactively from mitmproxy: https://docs.mitmproxy.org/stable/addons-options/#options
• set on start of mitmdump: https://docs.mitmproxy.org/stable/addons-options/#handling-configuration-updates, but not on the fly as far as I can see.

Also, I can update the allow-hosts from within my addon, but I don't see how my addon is triggered outside of the built-in hooks. I.e. the equivalent of updating an option interactively in mitmproxy.

I've been staring at your example, but can't quite wrap my head around how that helps me. By this example, did you mean, that I could send a "special request" through my running mitmproxy that triggers an addon and thereby modifies an option-value? I guess that could work.

[mhils]

By this example, did you mean, that I could send a "special request" through my running mitmproxy that triggers an addon and thereby modifies an option-value? I guess that could work.

Yes, this is one way of doing it. You could also use UNIX signals, spawn a separate HTTP API on another port, check for the existence of a file, ... I guess there are many ways to accomplish this. What's your use case here, and how/on which event would you like to update options ideally?

[chrhansen]

My use case is

• People will be using the proxy (in transparent mode) and some of the hosts they visit should be intercepted/mitm'ed. Likely more than half of the traffic/hosts should actually be ignored.
• The hosts that must be intercepted change, say once every couple of hours. Some hosts get added, some are removed.
• I would like the users to not notice anything. Why I want the proxy to ideally be always-on.

So I'd like to avoid restarting mitmdump in order to update the allow-hosts list. People using the proxy shouldn't notice any change (except if they care to check the root CA to see if their traffic is intercepted or not).

The event is coming from the outside world: Basically I will decide every couple of hours which hosts should be intercepted and I want mitmdump to be updated on the fly. There's no exact time interval, could be as often as every couple of minutes to no updates over the weekend – but mitmdump must still be running and intercepting the latest host-list. So my challenge is really to instruct mitmdump from the outside. I like your idea with UNIX signals, that may be cleaner than a special crafted request.