Helm next release is not picking the latest version

[adejanovski]

Bug Report

Describe the bug
Helm ignores version numbers that use a -**** suffix.

To Reproduce
Steps to reproduce the behavior:

1. Install K8ssandra from the next channel
2. Check the chart version by running helm ls -A
3. The installed version should be k8ssandra-1.0.5 despite the existence of 1.0.5-3 and 1.1.0-20210329062259-fe6b1cc1 in the index
4. Check the available versions using helm search repo k8ssandra-next -l |grep "k8ssandra-next/k8ssandra "
5. The latest available version will be 1.0.5

Expected behavior
The latest version should be 1.1.0-20210329062259-fe6b1cc1, which should be the one picked by Helm when performing the installation.

Additional context

Since Helm can't support such version numbers, I guess the best path forward should be to use version numbers that will match Helm's requirements (although they don't seem to be properly documented). Something such as 1.1.0.20210329062259 would probably work better and should be tested.

[adejanovski]

I stumbled upon an issue that explains it: helm/helm#2797 (comment)

By default, Helm will hide all releases that have a version number which matches a prerelease pattern (such as x.x.x-something).
if you want to include the prereleases, you need to add --devel to your helm command.

So without the --devel flag, here's what I get:

% helm search repo k8ssandra-next -l |grep "k8ssandra-next/k8ssandra " |head -5
k8ssandra-next/k8ssandra        	1.0.5        	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	1.0.4        	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	0.55.0       	3.11.10    	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	0.54.0       	3.11.10    	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	0.53.0       	3.11.10    	Provisions and configures an instance of the en...

And now with the --devel flag:

% helm search repo k8ssandra-next -l --devel |grep "k8ssandra-next/k8ssandra " |head -5
k8ssandra-next/k8ssandra        	1.1.0-20210329062259-fe6b1cc1	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	1.0.5                        	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	1.0.5-3                      	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	1.0.5-2                      	           	Provisions and configures an instance of the en...
k8ssandra-next/k8ssandra        	1.0.5-1                      	           	Provisions and configures an instance of the en...

We can see here also that 1.0.5 is considered a stable release and will take priority over 1.0.5-* releases which are considered pre-releases.

This probably means that we shouldn't separate stable and beta releases since Helm protects users from picking betas by default.

@bradfordcp, did you know about that Helm feature?
What would you think of using a single repo then and dropping the k8ssandra-next one?

[jdonenine]

That's an interesting find @adejanovski , thanks for getting to the details of it.

I have to say I'm a bit torn...

On one hand it simplifies things for us if we consolidate to a single release channel and we play by helms "rules" so to speak - which I do think is important.

On the other hand, it would seem that not a one of us knew about this feature of helm so I worry that we represent a decent chunk of users who also wouldn't know it's a thing and would never find our "next" releases. We'd have to document it well. Which maybe is fine, the majority of users would be using stable anyhow.

From a development perspective it gives us what we wanted, the ability to move forward without hurting the stable releases. But, I'm curious to hear what @bradfordcp thinks about from the user perspective.

[jsanda]

Prior to Helm 3.0, the Helm project itself maintained 3rd party charts. It had stable and incubator chart repos; so there is a precedent for multiple chart repos. I see pros and cons of each way. I do agree with @jdonenine that a lot of users likely won't be aware of this behavior.

[bradfordcp]

If I understand the issue correctly, even if we put our charts in a separate Helm repo we still have to use the --devel flag when installing or it won't try to install them?

If our repo only had 1.0.5-1, 1.0.5-2, and 1.0.5-3 which would it choose (assuming --devel was used)?

[jsanda]

If our repo only had 1.0.5-1, 1.0.5-2, and 1.0.5-3 which would it choose (assuming --devel was used)?

My understanding is that Helm would choose 1.0.5-3.

[adejanovski]

If I understand the issue correctly, even if we put our charts in a separate Helm repo we still have to use the --devel flag when installing or it won't try to install them?

Correct

My understanding is that Helm would choose 1.0.5-3.

Right. If we keep on putting prereleases in the next repo though, folks will get stuck at 1.0.5 unless they are aware of that flag.

Since there's no versioning that would allow us to put version numbers that would be consistent with the stable repo versions while not being prereleases, I'd be in favor of dropping the next repo and documenting this appropriately.
We can't put a 1.1.1, 1.1.2, 1.1.3 in the next repo which would not relate to our 1.1.1, 1.1.2 and 1.1.3 from the stable repo, it would be very confusing. And having a separate repo that requires to use the --devel flag each time seems odd to me.

In any case, I think that installing prereleases should not be the norm and any user doing it would have to be fully aware of how to do it (it would probably just be about testing new stuff).

[jsanda]

I do think there are plenty of situations in which folks will want to use charts that we are currently storing in the next repo. My concern with having to use the --devel option is that you have to opt into it. It is easy enough to forget. I am a forgetful person 😄.

I think I would prefer different version numbers in next vs requiring --devel and we document the mapping strategy. We currently do not use the appVersion field in Chart.yaml. We could use that to indicate whether the chart come from next or stable. This would make it clear when you run helm ls.

[adejanovski]

I do think there are plenty of situations in which folks will want to use charts that we are currently storing in the next repo. My concern with having to use the --devel option is that you have to opt into it. It is easy enough to forget. I am a forgetful person 😄.

Well, you have to opt in either way since you need to be aware of the existence of the next repo 🙂 that means you're actively installing k8ssandra with some special parameters

Alternatively, if we choose to stick with the next repo, we could keep the chart version to x.x.x-SNAPSHOT but when publishing to the next repo we could switch it to yyyymmdd.hh.mm. It would not be seen as a prerelease, which would then not require to use --devel. It would also avoid us from having to update the Helm chart version upon merge, which is a pain.
In this case maybe we could set the appVersion to the commit sha?

I'm still keen to use a single repo and use built in feature from Helm to handle prereleases, but would like to hear from the rest of the team on this matter.

[jeffbanks]

I am 👍🏻 on a solution that avoids doing an update of the Helm chart upon merge. We move too fast to synchronize that among team members. Looking to compare with other standards, so we aren't creating our own path, if not needed.

This is an explanation of how Rancher is aligning app and chart versions, for latest, stable, and alpha with examples of each.

[jsanda]

@jeffbanks Thanks for the Rancher reference. Very insightful!

[adejanovski]

As discussed during our meeting we will remove the Next repo and keep a single Helm repo which will contain both the stable releases and pre-releases.
I'll update CI accordingly.