Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Inbuilt Map Remote Functionality #1454
Charles Proxy has a Map Remote feature which maps one URL to another.
This can be achieved in mitm by using a script similar to:
def request(flow): if "google.com" in flow.url: flow.request.url = flow.request.url.replace("google.com", "google.jp")
I think we should have this feature inbuilt (just like content replacements) rather than expecting users to write custom scripts.
@daveyarwood Try setting up the dev environment first. See if all tests pass to ensure you have stuff working. Then come discuss this feature at our Slack channel.
I haven't used Charles much, but I remember using it's Map Remote once, and really liked it. Having this in mitm would be great! So it's nice you're picking this up.
After playing around with this a bit, I was able to create a simple script that replicates an example of one thing I've been using Map Remote for with Charles:
def request(flow): if flow.request.pretty_host == "mynetwork.adzerk.com": flow.request.host = "127.0.0.1" flow.request.port = 8080
mynetwork.adzerk.com is my login to the (hypothetical) "mynetwork" account in the Adzerk UI. When running the Adzerk UI locally on port 8080, I can go to any URL in my browser where the URL is yarwood.adzerk.com, and mitmproxy successfully redirects all requests from
Before I start hacking on this further, I could use some input about what this experience should be like for mitmproxy.
We could do a simple reproduction of the Charles Map Remote feature, which would be something like:
A simpler, possibly cleaner alternative is to do this in a more general way, using regex to search and replace over the entire URL. The replacement process would also parse the replaced URL and put each of the URL components where it needs to go. So for example, I might map
I think the first way (let's call it "the Charles Way") makes sense for Charles because it's a GUI application, and it's nice to work with fields in this way in a GUI. But for a more low-level CLI tool like mitmproxy, I think having a way to do regex replacement in the URL might be the more useful thing, and for convenience, we could always build more specific Map Remote-like behavior on top of that.
By the way, it seems like
Thoughts on all this?
@daveyarwood: Thanks for writing that up (and sorry for the slow-ish response)!
There's a certain feature overlap with
(One could also use an enum instead of three boolean flags)
I like this approach for being super flexible, but the UI is getting too complicated. It's also a bit of an open question if/how we can integrate map to local and file replacements with this. We could implement a special case that when a URL starts with
Another approach would be to simply have a dedicated Map Remote/Local editor:
(if these were regexes you could still enter a plain domain name and it'd work)
That is partially duplicating the replace feature (with the possible addition of map to local), but it would make a considerably simpler UI.
@mitmproxy/devs, this feature request came up a couple of times now and I think it'd be super useful. Any opinions?
@ujjwal96: I think there are still lots of improvements to be made regarding the UX. Consider the following user story:
I think this is something that's not viable with replacements right now. I think it would be very useful to have a dedicated "map remote" functionality that can be used to map URLs to other URLs or local files. It would be interesting to see how other tools are solving this (@dufferzafar mentions Charles Proxy), I haven't looked into that yet. To make the distinction a bit more clear, we may want to have the replacements addon operate on the request path only, not the full URL, and handle the URL in the map remote editor.
I tried like below,
but I got error response like this.
From the documentation, I couldn't find any related information.
@eu81273 Hello! Yes, you are right, unfortunately, our documentation doesn't describe some moments very clearly. I hope we will change it in the nearest future. But nevertheless there is something in the documentation you may be interested in:
Shortly speaking, here
The thing is
Is there any way to replace url with port number with command line?