New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
馃摉 Add instructions to start a RTV Experiment #29008
Conversation
|
||
We understand that experimenting a new feature or change on the client side is not always practial. For example, if you rewrite a big chunk of the code base, or include add extra step to optimize code in the transpiling process, you want to run the experiment at the server level, where different bianries will be returned. | ||
|
||
This doc explains how to create a server side experiment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe add one sentence how it is different from the other "client side diverted experiments"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
|
||
Once the experimental bianries are shipped, once can access those with different RTV prefix. | ||
|
||
For Experiment with `environment = AMP`. The Beta version that has prefix (03) will be used as control build for all experiments. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The control build (which is binary equivalent to stable) is used, so it's 02
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We do want to use RC (03) as the control group because RC and the experimental build are promoted at the same time, while control build(02) is promoted a week after.
I renamed the control build
to control group
, hope that clarify.
The doc should help whoever wants to create an experiment. The inabox part will be documented separately and internally.