appVersion unmarshalling error - yaml package version issue

[karuppiah7890]

Currently checking go.mod and go.sum shows that internally different yaml packages are being used

go.mod

github.com/ghodss/yaml v1.0.0

go.sum

github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
....
....
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
....
....
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=

Recently there's been lot of issues around this - helm/helm#6849

I noticed an issue in chartmuseum too. Here's how you can reproduce the issue:

1. Get latest version of chartmuseum
2. Run it in local

$ chartmuseum --storage local --storage-local-rootdir charts

3. Now create a dummy chart with any helm v2 version and push it to chartmuseum

$ helm repo add local http://localhost:8080
$ helm create old-chart
$ helm push old-chart local
$ curl http://localhost:8080/index.yaml | less
apiVersion: v1
entries:
  old-chart:
  - apiVersion: v1
    appVersion: 722225e2
    created: "2019-12-03T23:57:47.191526985+05:30"
    description: A Helm chart for Kubernetes
    digest: 504af607d12a44c4a3f6a05fe7e4b246cc84df3e55a3bce06a7b00dedd1e71cf
    name: old-chart
    urls:
    - charts/old-chart-0.1.0.tgz
    version: 0.1.0
generated: "2019-12-03T23:57:50+05:30"
serverInfo: {}

4. Use the latest version of helm v2 - v2.16.1, and try removing the repository and adding it back again and you will see this error

$ helm repo rm local
$ helm repo add local http://localhost:8080
Error: Looks like "http://localhost:8080" is not a valid chart repository or cannot be reached: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal number into Go struct field ChartVersion.entries.appVersion of type string

Now, this error is based on the discussions here - helm/helm#6849 . The gist is,

1. this commit fixes a float parsing - go-yaml/yaml@7b8349a
2. This new fix is available only in v2.2.3 - https://github.com/go-yaml/yaml/commits/v2.2.3 but not in v2.2.2 - https://github.com/go-yaml/yaml/commits/v2.2.2
3. Helm v2.16.1 recently moved to using newer version of gokpg.yaml.v2 - v2.2.4 here helm/helm@712c90f#diff-37aff102a57d3d7b797f152915a6dc16R62 . We need to use the new one too, for bug fixes
4. Actually there are multiple versions of a given yaml package being used here and there internally. And multiple yaml packages are also present in github/helm/helm . Same in chartmuseum. Here's the gist based on what chartmuseum has

https://github.com/helm/chartmuseum -> https://github.com/ghodss/yaml -> https://gopkg.in/yaml.v2 -> https://github.com/go-yaml/yaml

Now https://github.com/go-yaml/yaml is the main package and gopkg.in/yaml.v2 is just an alias like thing. Everyone else depends on this go-yaml/yaml library underneath. For example sigs.k8s.io/yaml too, which is a fork of https://github.com/ghodss/yaml . And these wrapper packages import go-yaml but with lower versions - https://github.com/ghodss/yaml/blob/master/go.mod .

My recommendation - just go ahead and use the latest version of gopkg.in/yaml.v2 , which is v2.2.7 , using replace directive in go.mod

This will fix the above given error, but will lead to this kind of index.yaml :

apiVersion: v1
entries:
  old-chart:
  - apiVersion: v1
    appVersion: "7.22225e+07"
    created: "2019-12-03T23:57:47.191526985+05:30"
    description: A Helm chart for Kubernetes
    digest: 504af607d12a44c4a3f6a05fe7e4b246cc84df3e55a3bce06a7b00dedd1e71cf
    name: old-chart
    urls:
    - charts/old-chart-0.1.0.tgz
    version: 0.1.0
generated: "2019-12-04T00:02:58+05:30"
serverInfo: {}

The appVersion becomes a scientific notation and will show up wrong in the helm search and might lead to any issues related to appVersion. You can see here - helm/helm#6921 for more info

[jdolitsky]

@karuppiah7890 thanks for detailed report

use the latest version of gopkg.in/yaml.v2 , which is v2.2.7 , using replace directive in go.mod

This sounds good. Looks like the appVersion as created by helm is still in weird format, but at least it is surrounded by quotes and shouldnt prevent the repo being added. Correct?

[karuppiah7890]

Yes. Repo add will at least work to start with 😄 we can tackle the appVersion scientific notation separately!

[willzgli]

Yes. Repo add will at least work to start with 😄 we can tackle the appVersion scientific notation separately!

Why same error happens with helm v3.6.0 ? I thought it was repaired about 1 year ago.