Add support for slow responses

[waldekmastykarz]

Extend the proxy with ability to test slow API responses.

Rationale

One aspect of testing how applications use an API is how they respond to slow API responses. Is it obvious that content is loading? Is the whole application unresponsive? Is the application failing because it expects a response in a certain timeframe? Graph Developer Proxy can help developers validate the user experience when the API responds slowly.

Spec

• introduce app setting named minDelayMs, that defines the lower range of the delay in milliseconds added to API response. Default value 500
• introduce app setting named maxDelayMs, that defines the upper range of the delay in milliseconds added to API response. Default value 10000
• introduce option named --min-delay that allows users to override the value of minDelayMs as configured in appSettings. When not specified, uses the value from appSettings.
• introduce option named --max-delay that allows users to override the value of maxDelayMs as configured in appSettings. When not specified, uses the value from appSettings.
• introduce option named --slow-responses which is a flag. When used, it will cause the proxy to add a random delay from range minDelayMs <= x <= maxDelayMs to each API response (mocked or not)

[waldekmastykarz]

@gavinbarron @sebastienlevert @garrytrinder I'd appreciate your feedback on this feature

[garrytrinder]

Great idea @waldekmastykarz I like this idea a lot, when working with asynchronous code we cannot assume that calls are always timely, or consistent.

Maybe we could provide some preset values here, similar in the way that Dev Tools does to mimic different network conditions, where they provide Slow 3G or Fast 4G profiles (https://developer.chrome.com/docs/devtools/network/reference/#throttling).

I could see this help with scenarios where apps are used on mobile devices away from the office and developers want to recreate those conditions.

[waldekmastykarz]

This isn't really about the network speed, which developers can simulate using browser tooling. This is more about the delayed response from the server, which would occur when the server is under load and it takes a while to process and respond to a request, before it's sent over the network.

[sebastienlevert]

I feel this should be a little bit more "randomized". Do we have specifics on the endpoints / use-cases that this could deliver? I don't know if we could fake slow responses as it could be slow 2 times (the artificial delay we would add, and then the workload taking extra time). I'd love to understand some of the developers pains you see with this option.

[waldekmastykarz]

The main scenario for this feature is to help developers understand how their app will look like, when the response from the API isn't instantaneous: will the app be blocked and show an empty page until all requests complete, will there be content shifts that have a negative impact on the UX, etc. I also wonder if there are apps that assume that a request completes within a given timeframe and fail the request when it doesn't to avoid being locked for too long.

[gavinbarron]

Could we also add an optional minimum delay to the parameter e.g. --slow-responses and --slow-responses 2000 are valid but the second ensures a minimum of 2000ms delay is added to responses?

[waldekmastykarz]

Could we also add an optional minimum delay to the parameter e.g. --slow-responses and --slow-responses 2000 are valid but the second ensures a minimum of 2000ms delay is added to responses?

It's a good idea to offer the ability to specify a minimum value @gavinbarron. How could make it less ambiguous if the specified value is the exact delay, the minimum or the maximum?

[gavinbarron]

--min-delay and --max-delay seem like good candidates with --slow-responses using the min and max delay defaults that are in the appsettings.json file?
providing either --min-delay or --max-delay will enable slow response mode and use the default for the unspecified value

[waldekmastykarz]

Are the min and max values something you'd be likely to change on each proxy run and hence something we should expose as options, or it's rather something you'd configure once in appSettings and then use as your preferred values? Trying to think if we have to add two more options or if a config setting be sufficient.

[gavinbarron]

In my mind they're likely something that I'd put in config as defaults and then be able to override as a command line option.
However being able to override from the command line is a nice to have.

[waldekmastykarz]

Updated spec. Have I captured everything correctly @gavinbarron?

[sebastienlevert]

Is there a reason where we specify the minDelayMs? I don't think it's required and should be documented (Maybe using JsonC in the config file, and in our docs). I don't feel it adds value.

[gavinbarron]

Updated spec. Have I captured everything correctly @gavinbarron?

That captures everything correctly from my point of view.

Is there a reason where we specify the minDelayMs?

I assume you're meaning the Ms suffix on these settings? IMO when reading the code inside the proxy that uses this setting it would also have the Ms suffix and make it more apparent what units the delay is in without the need to change to using jsonc and it's would also be visible in the C# code in this form.
I'm all for eliminating things that don't have value, but keeping the affordance of settings high through convention based naming is a nice thing to me, I hate to have to go look for documentation to for the units of time being used in these scenarios.