No alerts for HelmRelease in slack

[ilya-git]

Describe the bug

I have created this Alert:

  eventSources:
    - kind: GitRepository
      name: "*"
    - kind: Kustomization
      name: "*"
    - kind: ImageUpdateAutomation
      name: "*"
    - kind: HelmRelease
      name: "*"

When there is a new image in the docker registry that changes helm release tag, I get this message in slack:

kustomization/kustomization-name.flux-system
HelmRelease/my-namespace/release-name configured
revision
master/(commit-sha)

I do not get a message for HelmRelease though. K8s has events that I was expecting to see:

Normal info 25m (x2 over 27m) helm-controller Helm upgrade has started
Normal info 25m (x2 over 27m) helm-controller Helm upgrade succeeded

So something like that should appear in slack:

HelmRelease/release-name.my-namespace
Helm upgrade has started
chart
my-chart-1.2.3.tgz

Steps to reproduce

1. Add an alert with providers to post to slack similar to the one from description
2. Add a helm release
3. Change a helm release
4. Run kustomization
5. See that only kustomization has posted to slack, but not HelmRelease

Expected behavior

HelmRelease should post to slack as well

Screenshots and recordings

No response

OS / Distro

Ubuntu

Flux version

0.18.3

Flux check

► checking prerequisites
✗ flux 0.18.3 <0.19.1 (new version is available, please upgrade)
✔ Kubernetes 1.21.2 >=1.19.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.12.0
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.15.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.12.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.15.5
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.17.1
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.16.0
✔ all checks passed

Git provider

No response

Container Registry provider

ACR

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[kingdonb]

When you watch for a new HelmRelease revision notification in Slack, do you know if there are changes that should result in a new release?

Helm Controller reconciles every spec.interval but it doesn't create a new revision unless there are changes. You can see the new revisions in helm history <release> -- I am asking I guess whether there is actually an upgrade that occurs, and Notification controller does not broadcast it, or if the issue is that there is actually no new revision generated

(Say, a new version / git rev appears, but it doesn't have any changes in values / template / configmaps that are referenced via valuesFrom -- I would expect this doesn't generate a new revision in the HelmRelease, and you would get only the output that you saw, and no HelmRelease notification, since there was no upgrade performed.)

If there are releases upgrading and there aren't notifications generated from them, then this is a different bug. (If there are new versions resulting in new HelmChart artifacts and they don't get applied, that might still be a different bug.)

I would expect from an end-user perspective that a new chart version always results in a new helmrelease revision, regardless of if there are changes or not, but as an insider I would expect the code to behave the way I think it works, which is to say, a new revision is only reconciled when there are changes from the previous revision. This is done for reasons, but in the presence of an actual new version of the chart I'm not sure it's the correct behavior.

[ellakk]

I'm got bit by the same problem, but for Discord. I'm currently building a new cluster and i recive no notification at all from HelmRelease. I even had a release fail and still nothing. All other eventSources work fine.

  eventSources:
    - kind: GitRepository
      name: '*'
    - kind: Kustomization
      name: '*'
    - kind: HelmRelease
      name: '*'
    - kind: HelmRepository
      name: '*'

[kingdonb]

@ellakk Are you certain that HelmReleases are in the same namespace as the Alert?

https://fluxcd.io/docs/components/notification/alert/#example

According to the Alert CRD docs, if you do not specify a namespace then the same namespace as the Alert will be used. So writing an alert like this does not target HelmRelease resources in any other namespace.

There are changes to Multi-tenancy in Flux coming soon.

Long story short, there is no way today to target all namespaces with a wildcard like you can target all HelmRelease resources in a namespace with a wildcard.

[ellakk]

@kingdonb You're right, I missed that, thanks.

[kingdonb]

Awesome. That is a usability problem that is not currently addressed by any of the open RFCs, but you may want to be aware of these RFCs that are in the vicinity of the issue:

• [RFC] Access control for cross-namespace source refs flux2#2092
• [RFC] Flux Multi-Tenancy Mode flux2#2093
• [RFC] Define Flux tenancy models flux2#2086

We would like to be able to make cross-namespace references but safely permitted. Alerts can reach across namespaces now, but they cannot reach all namespaces. Lots might change about this, as I'm not certain any of the existing RFCs focus on alerts in particular, I think they are all focused on the issue as it pertains to sources for now, because it's where the big challenges are (because the sources may potentially carry access to a read-write git secret.)

This issue below might try to update that behavior about alerts and providers, so you could have ClusterAlerts which reach all namespaces and ClusterProviders similarly which are not tied up inside of any namespace:

• Add ClusterAlert and ClusterProvider resources flux2#2298

Closing, thank you for using Flux!