Add minimum traffic threshold to alerts #58
Conversation
The minimum traffic threshold defaults to 1 call per minute across all functions of a SLO, but can be modified in the CLI
autometrics-cli/src/sloth.rs
Outdated
| @@ -10,6 +10,21 @@ pub struct Arguments { | |||
| #[clap(long, default_values = &["90", "95", "99", "99.9"])] | |||
| objectives: Vec<String>, | |||
|
|
|||
| /// Minimum traffic to trigger alerts in events/second. | |||
| /// | |||
| /// If a service (all function sharing the same service name and objective | |||
There was a problem hiding this comment.
I wouldn't use the term "service" because it might not be a full service and we're kind of avoiding the term elsewhere. Maybe say something more like "An alert will only be fired for an objective if the total traffic to the functions comprising it exceeds the given threshold. This is to avoid alerts firing due to a small number of errors or high latency function calls during periods of low traffic."
There was a problem hiding this comment.
I struggled to find a word for it, because the functions really need to share the exact same service name and objectives in order to count in the threshold. So if a user has a SLO for "API" that's 99% success rate, and one SLO for "API" that is 90% success rate in the same app, then the calls to the "API/90%" SLO don't count for the threshold of the "API/99%" SLO, and I don't know how to push that warning in the documentation
There was a problem hiding this comment.
I tried to reword this part
Co-authored-by: Evan Schwartz <3262610+emschwartz@users.noreply.github.com>
The minimum traffic threshold defaults to 1 call per minute across all
functions of a SLO, but can be modified in the CLI
Ref: https://github.com/orgs/autometrics-dev/discussions/9