Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unset (default) images in TempoStack CR #675

Conversation

andreasgerstmayr
Copy link
Collaborator

@andreasgerstmayr andreasgerstmayr commented Nov 9, 2023

Various tools are capable of automatic rebuilding of images in case of
CVEs (e.g. freshmaker [1]). These tools rebuild all images and update
the RELATED_IMAGE_ environment variables in the CSV accordingly (ref. #638).

However, they do not update the version field (the
github.com/grafana/tempo-operator/internal/version.operatorVersion LDFLAG
when building the operator), they only update the version of the CSV.
Therefore the upgrade process won't run, and nothing will replace the
images field in the TempoStack CRs, i.e. the tempo/tempo-query/gateway
pods are not updated with the new image version.

This PR modifies the handling to use the default images if the image
field of the TempoStack CR is unset. Then the operator will use the image
from RELATED_IMAGE_ env variables, and the containers are always using the
latest images as specified in the RELATED_IMAGE_ env vars. As a bonus,
if a user is manually editing the images field, it won't get reset on every
upgrade.

[1] https://github.com/redhat-exd-rebuilds/freshmaker

Resolves: #674

@codecov-commenter
Copy link

codecov-commenter commented Nov 9, 2023

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (e89fe16) 78.22% compared to head (4ceafa2) 78.22%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #675      +/-   ##
==========================================
- Coverage   78.22%   78.22%   -0.01%     
==========================================
  Files          65       66       +1     
  Lines        5024     5023       -1     
==========================================
- Hits         3930     3929       -1     
  Misses        907      907              
  Partials      187      187              
Flag Coverage Δ
unittests 78.22% <84.44%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
apis/tempo/v1alpha1/tempostack_webhook.go 79.14% <ø> (+2.11%) ⬆️
cmd/generate/main.go 56.64% <100.00%> (+1.24%) ⬆️
controllers/tempo/tempostack_create_or_update.go 58.24% <100.00%> (ø)
internal/manifests/gateway/gateway.go 92.62% <100.00%> (+0.06%) ⬆️
internal/manifests/gateway/openshift.go 93.63% <100.00%> (+0.11%) ⬆️
internal/manifests/queryfrontend/query_frontend.go 91.11% <100.00%> (+0.14%) ⬆️
...nternal/manifests/servicemonitor/servicemonitor.go 100.00% <100.00%> (ø)
internal/tlsprofile/options.go 100.00% <100.00%> (ø)
internal/upgrade/upgrade.go 74.73% <ø> (-3.84%) ⬇️
internal/upgrade/v0_6_0.go 100.00% <100.00%> (ø)
... and 5 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Various tools are capable of automatic rebuilding of images in case of
CVEs (e.g. freshmaker [1]). These tools rebuild all images and update
the RELATED_IMAGE_ environment variables in the CSV accordingly (ref. grafana#638).

However, they do not update the version field (the
github.com/grafana/tempo-operator/internal/version.operatorVersion LDFLAG
when building the operator), they only update the version of the CSV.
Therefore the upgrade process won't run, and nothing will replace the
images field in the TempoStack CRs, i.e. the tempo/tempo-query/gateway
pods are not updated with the new image version.

This PR modifies the handling to use the default images if the image
field of the TempoStack CR is unset. Then the operator will use the image
from RELATED_IMAGE_ env variables, and the containers are always using the
latest images as specified in the RELATED_IMAGE_ env vars. As a bonus,
if a user is manually editing the images field, it won't get reset on every
upgrade.

[1] https://github.com/redhat-exd-rebuilds/freshmaker

Resolves: grafana#674
Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
@andreasgerstmayr andreasgerstmayr force-pushed the unset-images-field-in-crd-if-default branch from 7a67977 to 7e4a74c Compare November 9, 2023 15:15
@andreasgerstmayr andreasgerstmayr marked this pull request as ready for review November 9, 2023 15:16
// This upgrade unsets the image fields in the TempoStack CR.
// From 0.6.0 onwards, the image location is not stored in the CR unless it got changed manually.
func upgrade0_6_0(ctx context.Context, u Upgrade, tempo *v1alpha1.TempoStack) (*v1alpha1.TempoStack, error) {
tempo.Spec.Images.Tempo = ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this correctly, this is going to unset all the images on the CRs, but if an user specified an image and then do the upgrade, he will lose that setting?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Hopefully for the last time though, after that PR the operator should not touch that field anymore.
Currently we also overwrite the image on every upgrade:

updateTempoStackImages(u, &tempo)

Is there a correct way to determine if the setting is the default or was changed by the user?
Looking for the current default docker.io/grafana/tempo:2.2.3 for tempo won't work in two cases:

  • downstream products with different image locations
  • in case the operator gets upgraded from e.g. 0.3.0 to 0.6.0 directly, i.e. skipping a version

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking more about it. If the user configures a custom image, let's say a rebuild with modifications of tempo 2.2.3, and then upgrades the tempo operator and the new operator version requires tempo version 2.4.
This will likely break the installation anyway?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, setting a custom image only makes sense if the TempoStack is set to unmanaged imho (unmanaged also means it won't get upgraded).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is a way, I think the approach here is the right one. May be we need to put in the changelog or in some place that if you upgrade from 0.5.0 to 0.6.0 and you have custom images those changes will be overridden.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that make sense to me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I updated the text in the changelog entry to make it more clear.

Signed-off-by: Andreas Gerstmayr <agerstmayr@redhat.com>
@andreasgerstmayr
Copy link
Collaborator Author

andreasgerstmayr commented Nov 10, 2023

Copy link
Collaborator

@frzifus frzifus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats actually quite nice :)

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would document here the workaround

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which workaround?

@andreasgerstmayr andreasgerstmayr merged commit 88595ff into grafana:main Nov 17, 2023
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automated rebuilds won't pick up new images
5 participants