Implement Network Latency Simulation

[gfausagullo]

Overview

The purpose of this issue is to implement network latency simulation in the MockAI server to enhance testing capabilities by mimicking real-world network conditions.

Goals

• Introduce a way to simulate minimum and maximum network delays.
• Ensure that all API endpoints can receive and process the delay parameters.
• Test the functionality to ensure it accurately reflects the specified network latencies.

Tasks

• Update the README.md to include documentation on how to use the network latency simulation.
• Modify the chat.js, embeddings.js, image.js, and text.js files to process the new minMsDelay and maxMsDelay parameters.
• Implement a function to randomly generate delay times within the specified range.

Expected Impact

This feature will allow developers to better understand how their applications behave under various network conditions, leading to more robust and resilient systems.

[rastogiji]

This should be rather be enforced at runtime instead of a configuration parameter at setup using a header such asx-set-response-delay-ms to give downstream applications have more control.

[ctrahey]

@rastogiji I like that, but also it can be valuable to use a mock that doesn't depend on changes in the client to utilize it's features. How do you feel about reading the header if it is present, but drawing from an env var if the header is not present? (with a final default of zero of course, if the env var is also not present)

[rastogiji]

@ctrahey This makes a lot of sense. Let me add that.

[rastogiji]

#12