Allow scripting of mock rule behaviour

[pimterry]

As of just now, you can mock & rewrite traffic, using the set of matchers & handlers available on the mock page. You can do a little dynamic logic there by forwarding matching requests to your own local server, which can include whatever logic it likes, but that is still a quite limited form and makes rewriting of remote responses, among other things, very challenging.

Unfortunately, embedding a scripting language inside HTTP Toolkit is quite limiting. Do you have to edit it inside the app? Can you use external packages and data stores? Do you have to use a specific language, that a significant subset of users probably won't already know?

It would be good to find a way to allow scripting, whilst avoiding some or (in a perfect world) all of these limitations.

If you're keen on this feature, context of how you'd use it would be super helpful. Please comment below to describe the kind of scripts you'd like to write!

---

Does this affect you too? Click below and add a 👍 to vote for this and help decide where HTTP Toolkit goes next, or go vote on the other most popular ideas so far.

[shirshak55]

It would be nice we can use wasm to modify or do something with request/ response. As a workaround, I am currently using mockttp.

[pimterry]

I've thought about WASM, it'd definitely be interesting! It's clearly a good fit for that kind of very flexible plugin architecture. I'm not sure exactly how the function interfaces for WASM like that work though, or really how WASI works in cases like this (we'd need scripts to be able to write files to disk and make network connections, for example, to handle logging use cases & custom proxying etc). Do you know much about WASM and this stuff?

I think because we already have Mockttp which has working JS APIs for this, I'd probably start with a pure JS interface anyway (that would also allow quick scripts to be written in JS inline inside HTTP Toolkit) but WASM would definitely be a sensible extension of that later on. In theory it would all work the same way: basically just loading a file (JS or WASM) and running a function that's exported (like customScript.handleRequest(request, response)) and expecting the file to do whatever it wants from there (logging request content, forwarding the request elsewhere, writing values back to the response connection, whatever).

[leumasme]

The lack of this feature and a request composer/replay feature is the only reason I still start up Fiddler4 sometimes.

For comparison with some alternatives:

• Fiddler Everywhere does not have scripting
• Fiddler4 (Legacy pre-electron fiddler) has scripting in "FiddlerScript" which is based on JScript.NET, an ugly child of JavaScript and C#/.NET with the intention of being easy for developers of either language but arguably hard for developers who know just one of the two languages. Despite this, it's quite powerful and is not just bound to mocking/editing requests
• Burp has Extensions written in Java and an official extension store. There are extensions that for example add Python scripting which would be more equivalent of Mock rule scripts

@pimterry I think you're pretty spot-on with starting with a JS interface first since it's likely easy to implement and easy to use, and later adding WASM support if demand for supporting other languages exists (which is also not covered by the ability to transpile to JS, as for example Unity used to do it).
I'm not sure what scope you are intending for scripting, (only request/response handlers? let scripts add custom context menu entries? let scripts filter shown requests/act as an alternative to the existing search?), but in any case, this issue is 4 years old now, and seeing any form scripting rather sooner than later, even if basic, would be great!

[shirshak55]

Yea, it would be pretty nice if we could add a plugin system. However, I find most of my needs can be easily fulfilled by using mockttp package.

It would be nice if we can add an official plugin system though. It can make many things ergonomic.

[pimterry]

I'm not sure what scope you are intending for scripting, (only request/response handlers? let scripts add custom context menu entries? let scripts filter shown requests/act as an alternative to the existing search?), but in any case, this issue is 4 years old now, and seeing any form scripting rather sooner than later, even if basic, would be great!

I'm focused in this issue on scripting for HTTP rules specifically, so that when a rule is triggered, your own code handles the rule, to define custom behaviours or configure other interesting side effects.

The UX for this will need significant work, but the plumbing is actually already in place - as @shirshak55 mentions it's quite possible to do custom logic with Mockttp today in most use cases, and Mockttp is already powered all of HTTP Toolkit's network traffic, so it's just a matter of exposing an interface to configure the script, and then passing it through to those existing APIs.

This should be relatively easy and I agree it would be a huge boost! The problem is just that I have limited time, and it's not yet at the top of my list unfortunately (currently, that's iOS support).

a request composer/replay feature

I'm very keen on adding this too! That's tracked in #48

If you're keen in helping to make either of these happen, pull requests for anything toward this are very welcome, and I'm happy to discuss details and give you pointers in the right direction, just let me know if you're interested.

It would be nice if we can add an official plugin system though. It can make many things ergonomic.

In terms of a more general plugin system, to extend the search or add context menu items etc, I'm more cautious. This would be possible, but designing a good plugin interface is a much harder problem. Also, I think many tools support this because they're not open-source - in most cases, if there's something you'd like to add to HTTP Toolkit, it would be better to just open a PR and add it directly!

I'm sure there are counter examples possible, but in practice I'm yet to see specific suggestions that somebody personally wants to add that wouldn't be better if it was just integrated into the tool for everybody.

[krishnaraj]

Having a scripting console similar to OWASP ZAP would be a killer feature. Lack of this feature alone is making me fallback to MITM Proxy / ZAP often.

[pimterry]

@krishnaraj I can see that ZAP has various very different kinds of scripting - from UI graphical additions to proxy rules to various others, many of which wouldn't apply.

Can you share some specific examples about which kinds of scripts you're using, what you use them for, and how you manage & run them? Any more info on your workflow & use cases would be very helpful.

[krishnaraj]

I primarily use ZAP to change parts of a response/request. The scripting console makes it easy to write custom logic and I can also version control the scripts.

[pimterry]

Thanks @krishnaraj! That's super helpful, I'll use this example once development starts here - those kinds of request/response modification hooks are exactly what I'm aiming to support initially 👍