New feature that user assumpt an arbitrary value of sli when metrics hasn't a increase #242
Conversation
…hasn't a increase, and it is equals to 0. The metric exists but has no changes. Some scenarios we want a sli assumption to 100% or 0%. This feature can be used by adding the option assumption_empty_sli under spec in the slo_config.
|
Hi @casmssm, I just took over the maintenance of the SLO Generator. Could you give us a quick example where this extra setting is relevant, and what the corresponding YAML files look like? |
There was a problem hiding this comment.
I suggest rewriting this in the following way to keep to code clean:
- In the
reportcode, add an if SLI value ==NO_DATAand set the SLI value toDEFAULT_EMPTY_SLI_VALUE
nb: it might even makes sense to specify a default sli value per backend, but I'm not sure about the use cases.
| data = self.run_backend(config, backend, client=client, delete=delete) | ||
| if not self._validate(data): | ||
| # Patch globo-0.3 for three retries | ||
| RETRIES=3 |
There was a problem hiding this comment.
Retries do not belong here, this code should be left as clean as possible.
Retries should go in the providers (backends / exporters) code as they can be different for various backends. Please see https://github.com/google/slo-generator/blob/master/slo_generator/backends/dynatrace.py#L270 for an example.
On top of that, what is globo-0.3 ?
| # Post validation | ||
| if not self._post_validate(): | ||
| self.valid = False | ||
|
|
||
| def build(self, step, data): | ||
| def build(self, step, data, assumption_empty_sli=-1): |
There was a problem hiding this comment.
same here, I don't think we should pass assumption_empty_sli to the build method.
imho the way should be a user configurable global variable to specify the SLI value when no events are found.
Please keep in mind that SLOs with no events cannot be relied on, read https://sre.google/workbook/alerting-on-slos/#low-traffic-services-and-error-budget-alerting for more info on why.
| @@ -246,11 +280,18 @@ def get_sli(self, data): | |||
| if isinstance(data, tuple): # good, bad count | |||
| good_count, bad_count = data | |||
| if good_count == NO_DATA: | |||
| good_count = 0 | |||
| #patch globo-0.3, terminate if not exists good events | |||
| good_count = 'EMPTY' | |||
There was a problem hiding this comment.
good_count should be an int o a float, and NO_DATA already tells us that no events were found.
New feature that user assumpt an arbitrary value of sli when metrics hasn't a increase, and it is equals to 0. The metric exists but has no changes. Some scenarios we want a sli assumption to 100% or 0%. This feature can be used by adding the option assumption_empty_sli under spec in the slo_config.
We use this tool in production and we need this patch, but someone else might need it too.