Experiment Manager Design Discussion #227
Replies: 11 comments 2 replies
-
|
As per our discussion and the RM design as seen here, here is the latest INPUT JSON |
Beta Was this translation helpful? Give feedback.
-
|
please correct me if i'm wrong, The |
Beta Was this translation helpful? Give feedback.
-
|
Deployment Usecases that Exp Mgr needs to cater to Deployment options for a single microservice part of an App that has many microservices.
|
Beta Was this translation helpful? Give feedback.
-
|
Running in a Dev/QA Cluster, use EM to do a K8s rolling update
Reference: |
Beta Was this translation helpful? Give feedback.
-
|
Would like to get views on the input addon in JSON which is sent to EM (Will be added between trail info and training info)
|
Beta Was this translation helpful? Give feedback.
-
|
Any other settings which will be required later will be added as |
Beta Was this translation helpful? Give feedback.
-
|
I was thinking about a config store and validation object for storing the parsed entities from JSON and supply it on request to the ETC (Experiment Trial Controller) currently we have the
Validations : (Some validations which i can think of but not limited to)
I feel this can solve the storage and validation problem of EM config, I do accept the storage part and validation part can be separate but I feel keeping them accesible in one object avoids creating a separate validation flow for config which involves new objects and new function calls and also we can keep it config specific validation rather than a generic validation so it will be easy in a maintainance perspective. Please correct me if I'm wrong |
Beta Was this translation helpful? Give feedback.
-
|
For ease of adding further config json changes would like to propose a section design in config json which has sections like Note: list of deployments which we have in Can I please get a review on this input JSON structure ? @dinogun @chandrams @kusumachalasani @shruacha1234 Example JSON: |
Beta Was this translation helpful? Give feedback.
-
|
NOTE: This comment is added to make things clear about the data transfer between DA and EM. I'm just adding my views below, please correct me if I'm wrong Requirement of AutotuneDTO dependency in EM: Class: I feel we need to add/return the I would like to suggest a change in this mechanism which makes ot one step: Now in EM we write a validator to check if the received JSON Object is of type config object (validate the required keys and structure) and then we save the config as config object and link it in Pros:
Cons:
Please add any pros or cons which you feel that the new mechanism could bring in. I feel it's a very tough way to go forward as minimal change in JSON structure could make drastic change in the Config Object design and functionality but I feel it's really useful to make this change coz its an PS: I'm expecting lot of changes in input JSON going forward so keeping it limited to a change in class and reflecting changes in it's callers can be a first good step to make EM design + code mapping errors free. @dinogun @chandrams @kusumachalasani @shruacha1234 Can I please have your views on it ? |
Beta Was this translation helpful? Give feedback.
-
|
Adding Unscheduled stage merge feature to EM: In the current EM implementation, we are having each stage separated and being executed by different threads, Some stages can be very minimal (Eg: deploying the config) which can be in-lined with the previous stage. We keep the stages separate but provide an in-lining feature for unscheduled stages. The stage which is continuous without any scheduling delay can be in-lined and we could take that decision based on a setting whether to in-line stages. This can be a performance tuning for the EM itself. PROS:
CONS:
@dinogun @chandrams @kusumachalasani @shruacha1234 Can I have your views on this? |
Beta Was this translation helpful? Give feedback.
-
|
updated JSON: |
Beta Was this translation helpful? Give feedback.
-
|
Expected Output JSON - for |
Beta Was this translation helpful? Give feedback.
-
|
Need for datasource availability monitor: The current EM design expects the datasource to be available all time, what if the datasource is not up and running? EM shouldn't wait for metric collection to fail getting unreachable endpoint or etc. We need to have datasource monitor service which takes in trial id as input and constantly monitors a datasource given in the trial input json, it holds a map of datasources and the trial id's using those sources so once it finds that data source is unavailable it gets the ETD (Experiment Trial Data) object and updates a field (Lets say is datasource available - mapped to each data source of in trail input JSON) so based on this status when CollectMetrics transistion is invoked it checks the status (we make it synchronous so only after update we read it) and then proceeds to metric collection if not waits for a certain timeout and even after the datasource is not up we report it to the user saying datasource is compromised. |
Beta Was this translation helpful? Give feedback.
-
This discussion talks about the changes we expect in the input, output, flow etc of the Experiment Manager.
Here are the expected input and output JSON.
Expected Input JSON:
Expected Output JSON:
Beta Was this translation helpful? Give feedback.
All reactions