Proposal: Chorify NewDot Performance
Background
For the last few months, #migrate has been (by choice) blocked by NewDot performance improvement efforts. We track metrics in Sentry, and the most important "vital" metrics are tracked in the P90 of vital metrics Sentry dashboard. Each metric has it's own, more detailed, dedicated dashboard as well (example). Each metric has a pre-defined constant threshold which we discussed and agreed upon. We define the metric as "good" if its P90 is under the threshold, or "bad" if its P90 is over the threshold.
To track and coordinate this effort, @tgolen organized the AdHoc NewDot team - with internal engineers assigned to own each metric and work to get it below its agreed-upon threshold. 4/7 metrics are currently below their thresholds, and we are gearing up to unblock #migrate.
Those of us who have been focused on these metrics are looking forward to a future where we can work on other projects while maintaining confidence that the app will continue to perform according to our collectively agreed-upon performance goals.
Problem
When new features are added or existing features experience performance regressions which affect our vital app metrics, if only a small group of individuals are responsible for monitoring and addressing those vital metrics, then those metrics risk backsliding or degrading over time, which prevents us from delivering a high-quality product experience to our user base while allowing the members of the current NewDot Performance Team to work on other projects.
Solution
- Set up the GitHub+Sentry Integration.
- Create a new Alert in Sentry for each of the vital metrics, which creates a GitHub Issue if the alert drops below the threshold.
- Auto-assign that GitHub issue to a member of the engineering team using FairSelection™.
a. Once assigned to the issue, the assignee will be responsible for "owning" the metric until it drops back down below the threshold - they can use whatever resources available to them to achieve that goal (external contributors, AI, other internal contributors, etc...).
b. Once the metric drops below the threshold for a week, they can close the issue. Subsequent alerts will create a new issue and assign it to a new member of the team.
- Update the design doc template with a new section - Sentry metrics. If a project adds a new product feature, design doc authors and reviewers should:
a. Define any relevant performance metrics for that new feature
b. Agree upon and set a threshold for the feature
c. Capture the metric in the app (i.e: create a new span)
d. Create a Sentry dashboard for that metric
e. Add the metric to P90 of Vital Metrics
f. Configure a Sentry threshold and alert for the new metric
Issue Owner
Current Issue Owner: @roryabraham 
Design Doc
Proposal: Chorify NewDot Performance
Background
For the last few months, #migrate has been (by choice) blocked by NewDot performance improvement efforts. We track metrics in Sentry, and the most important "vital" metrics are tracked in the P90 of vital metrics Sentry dashboard. Each metric has it's own, more detailed, dedicated dashboard as well (example). Each metric has a pre-defined constant threshold which we discussed and agreed upon. We define the metric as "good" if its P90 is under the threshold, or "bad" if its P90 is over the threshold.
To track and coordinate this effort, @tgolen organized the AdHoc NewDot team - with internal engineers assigned to own each metric and work to get it below its agreed-upon threshold. 4/7 metrics are currently below their thresholds, and we are gearing up to unblock #migrate.
Those of us who have been focused on these metrics are looking forward to a future where we can work on other projects while maintaining confidence that the app will continue to perform according to our collectively agreed-upon performance goals.
Problem
When new features are added or existing features experience performance regressions which affect our vital app metrics, if only a small group of individuals are responsible for monitoring and addressing those vital metrics, then those metrics risk backsliding or degrading over time, which prevents us from delivering a high-quality product experience to our user base while allowing the members of the current NewDot Performance Team to work on other projects.
Solution
a. Once assigned to the issue, the assignee will be responsible for "owning" the metric until it drops back down below the threshold - they can use whatever resources available to them to achieve that goal (external contributors, AI, other internal contributors, etc...).
b. Once the metric drops below the threshold for a week, they can close the issue. Subsequent alerts will create a new issue and assign it to a new member of the team.
a. Define any relevant performance metrics for that new feature
b. Agree upon and set a threshold for the feature
c. Capture the metric in the app (i.e: create a new span)
d. Create a Sentry dashboard for that metric
e. Add the metric to P90 of Vital Metrics
f. Configure a Sentry threshold and alert for the new metric
Issue Owner
Current Issue Owner: @roryabraham