Pluggable Simulator Design

Author: Anna Pavlenko

Overview

The goal is to make the existing Python - based simulator pluggable, allowing users to implement their own scaling algorithms in any programming language. Users will then provide the path to their executable algorithm, and the simulator will handle the integration seamlessly.

Objectives

• Language - Agnostic: Allow users to implement their scaling algorithms in any programming language.
• Simple Integration: Users should only need to provide the path to their executable, with no need for extensive integration work.
• Flexible Communication: Use stdin / stdout or other methods(like HTTP or gRPC) for communication between the simulator and the algorithm.
• Extensible Design: The simulator should be able to support various communication protocols and easily extend to new ones.

Design Components

1. Simulator Interface

• The simulator should define a clear interface that users' algorithms need to implement. This interface will consist of:
  • Input: The simulator will pass input data(e.g., metrics) to the algorithm. Config can be read from the json file
  • Output: The simulator will expect the algorithm to return processed data(e.g., scaled metrics).

2. External Algorithm Implementation

• Users will write their scaling algorithms as standalone executables that:
  • Accept input via stdin, files, or network requests.
  • Process the input data according to the specified logic.
  • Output the results to stdout, files, or send them back over the network.
• These algorithms can be written in any programming language.

3. Wrapper/Adapter in Simulator

• The simulator will include a wrapper or adapter that:
  • Launches the external algorithm as a separate process or service.
  • Passes the necessary input data to the algorithm.
  • Captures and processes the output data returned by the algorithm.
• This component will handle errors, such as process failures or incorrect data formats.

4. Configuration and Execution

• Users will specify the path to their algorithm executable via a command - line argument or configuration file.
• Example command - line usage: ```bash python simulator.py - -algorithm / path / to / external / algorithm - -config data / metadata.json

• The simulator will invoke the provided executable, passing data and retrieving results as part of the simulation.

5. Error Handling and Validation

• The simulator will validate the output from the external algorithm to ensure it matches the expected format.
• Robust error handling will be implemented to manage issues like:
  • Failed process execution.
  • Incorrect output formats.
  • Timeouts or unresponsive algorithms.

6. Extensibility

• The design should allow for easy addition of new communication protocols(e.g., gRPC, REST API).
• Users and developers should be able to extend the simulator to support different types of external algorithms without modifying the core logic.

Advantages

• Language Independence: Users are free to implement algorithms in the language of their choice.
• Ease of Use: Simple interface and minimal setup required for users to integrate their algorithms.
• Scalability: The simulator can be extended to support more complex integration patterns, including distributed algorithms or web services.

Next Steps

1. ** Define the input / output format ** that the simulator and external algorithms will use.
2. ** Implement the wrapper / adapter ** in the simulator for process management and data handling.
3. ** Create example implementations ** of scaling algorithms in different languages to demonstrate the pluggability.
4. ** Develop documentation ** to guide users in implementing their algorithms and integrating them with the simulator.