Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metrics-operator): introduce scoring logic for Analysis evaluations #1872

Merged
merged 11 commits into from
Aug 21, 2023

Conversation

odubajDT
Copy link
Contributor

@odubajDT odubajDT commented Aug 10, 2023

Changes

  • introduce structures for storing results of each layer of SLO scoring system
  • implement scoring logic in multiple layers
  • unit tests

Fixes: #1767

@netlify
Copy link

netlify bot commented Aug 10, 2023

Deploy Preview for keptn-lifecycle-toolkit ready!

Name Link
🔨 Latest commit 31a8ff4
🔍 Latest deploy log https://app.netlify.com/sites/keptn-lifecycle-toolkit/deploys/64e30b71c46645000837f280
😎 Deploy Preview https://deploy-preview-1872--keptn-lifecycle-toolkit.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@codecov
Copy link

codecov bot commented Aug 10, 2023

Codecov Report

Merging #1872 (e66ee7c) into main (df874e2) will increase coverage by 0.33%.
The diff coverage is 100.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1872      +/-   ##
==========================================
+ Coverage   83.53%   83.87%   +0.33%     
==========================================
  Files         107      141      +34     
  Lines        7077     8588    +1511     
==========================================
+ Hits         5912     7203    +1291     
- Misses        957     1128     +171     
- Partials      208      257      +49     
Files Changed Coverage Δ
...-operator/api/v1alpha3/analysisdefinition_types.go 100.00% <100.00%> (ø)
.../controllers/common/analysis/analysis_evaluator.go 100.00% <100.00%> (ø)
...controllers/common/analysis/objective_evaluator.go 100.00% <100.00%> (ø)
.../controllers/common/analysis/operator_evaluator.go 100.00% <100.00%> (ø)
...or/controllers/common/analysis/target_evaluator.go 100.00% <100.00%> (ø)
...perator/controllers/common/analysis/types/types.go 100.00% <100.00%> (ø)

... and 32 files with indirect coverage changes

Flag Coverage Δ
certificate-operator 68.55% <ø> (ø)
component-tests 58.93% <ø> (-0.14%) ⬇️
lifecycle-operator 84.92% <ø> (ø)
metrics-operator 85.37% <100.00%> (?)
scheduler 32.12% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@odubajDT odubajDT force-pushed the feat/1767/slo-scoring branch 3 times, most recently from 30b16c8 to 8c464c3 Compare August 11, 2023 08:00
@odubajDT odubajDT marked this pull request as ready for review August 11, 2023 08:05
@odubajDT odubajDT requested a review from a team as a code owner August 11, 2023 08:05
@odubajDT odubajDT requested a review from bacherfl August 11, 2023 09:04
bacherfl
bacherfl previously approved these changes Aug 16, 2023
Copy link
Member

@bacherfl bacherfl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>
@sonarcloud
Copy link

sonarcloud bot commented Aug 21, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@odubajDT odubajDT merged commit b6f2172 into main Aug 21, 2023
39 checks passed
@odubajDT odubajDT deleted the feat/1767/slo-scoring branch August 21, 2023 11:20
// get the value
floatVal, err := getValueFromMap(values, computeKey(obj.AnalysisValueTemplateRef))
if err != nil {
result.Error = err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this error message makes sense to be stored in the CR 🤔
looking at the code, it seems that this .Error field is not used. Maybe we could propagate the error up so we can log it at the entry point with info about what analysis fails

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the error is filled when the map of values does not contain the required key and value, therefore we store the error that the value for the objective was not retrieved and pass the result one layer above. Therefore in the end, each ObjectiveResult has an error stored in the structure if it occured

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The struct is only internally used and it is not stored in any CR. However, the field it is not used so I am questioning the utility of storing errors. Furthermore, we are going against Go practices to have the error flow as part of the function return type and instead, we hide that into the returning struct :/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we should store the error, when the value cannot be retrieved. It needs to be stored within the objectiveResult. You are right that we do not need to store the whole error, but just the error message.

If we just return the error as a function return type, we do not have any information in the resulting status why this particular objective has score == 0. We might lose information here then

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement sloTarget scoring logic to evaluate AnalysisValues based on sloTargets of AnalysisDefinition
4 participants