Spegel don't start in version >=0.0.25

[morsik]

Spegel version

>=0.0.25

Kubernetes distribution

kubeadm

Kubernetes version

1.30.8

CNI

flannel

Describe the bug

Hi, I've had Spegel running for few months without any issues on version 0.0.23. Today I decided to upgrade it, I upgraded Helm Chart to 0.0.24 and everything was fine. So then I tried update to 0.0.25 (or any hgher) and pods are crashing with very weird message I wasn't even able to find in Spegel's source code. No version since 0.0.25 works up until latest.

Update method:

$ helm upgrade --create-namespace -n spegel -i --version v0.0.25 spegel oci://ghcr.io/spegel-org/helm-charts/spegel -f spegel/spegel.values.yaml

just in case, this is what I have in my valuess:

resources:
  requests:
    cpu: 10m
    memory: 24Mi
  limits:
    cpu: 1
    memory: 64Mi

And pod logs:

$ k -n spegel logs -f pod/spegel-wj7d8
Defaulted container "registry" out of: registry, configuration (init)
{"time":"2025-02-07T22:32:06.08283206Z","level":"ERROR","source":{"function":"main.main","file":"/build/main.go","line":89},"msg":"run exit with error","err":"Invalid Semantic Version"}

My Helm version is:

version.BuildInfo{Version:"v3.16.4", GitCommit:"7877b45b63f95635153b29a42c0c2f4273ec45ca", GitTreeState:"dirty", GoVersion:"go1.23.4"}

[phillebaba]

The error is coming from the version check of Containerd. There is something going on with the reported version of Containerd. Could you check and share what version is being reported by the node?

[morsik]

@phillebaba

$ k get nodes -o wide
NAME     STATUS   ROLES           AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION   CONTAINER-RUNTIME
dhc-01   Ready    control-plane   41d   v1.30.8   10.19.11.21   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-28-amd64   containerd://1.6.20
dhc-02   Ready    control-plane   41d   v1.30.8   10.19.11.22   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-28-amd64   containerd://1.6.20
dhc-03   Ready    control-plane   41d   v1.30.8   10.19.11.23   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-28-amd64   containerd://1.6.20

But just to clarify: versions 0.0.23 and 0.0.24 works perfectly fine.

[morsik]

@phillebaba, @jfroy I found this commit that adds semver verification: e2ccf67 - and that commit is indeed added since 0.0.25 release of Spegel.

And my containerd is installed from Debian repository and reports indeed non-semver-compliant number with Debian addons...

root@dhc-01 ~ # containerd --version
containerd github.com/containerd/containerd 1.6.20~ds1 1.6.20~ds1-1+b1

This shouldn't break stuff, as package names in Debian (and possibly other distros) has their own standards to specifically distingiush original software version from package release in their versioning scheme.

[phillebaba]

This explains why things are not working, and it is a bit frustrating that versions would be different based on the dist flavor. Let me see if there is some way of fixing this without having to disable verification in certain cases.

[morsik]

@phillebaba the reason for that is different distributions may add their own patches to software (even if that is not patch in the software itself, but just systemd service file for example), so they have to specifically say, that it's still the same version of software (1.6.20, but different release of package for that version of software (ds1 in this case). Red Hat also uses something very similar (just don't use ~ character, but slightly other scheme - but still not compatible with semver if you want to take that version fully and parse it)

[jfroy]

I did t think about that when I wrote the patch. Something like npm semver coerce parsing might help here.

[jfroy]

Unfortunately Masterminds/semver’s NewVersion does not parse a Debian version.

https://go.dev/play/p/gQQAXRLYGcm

[morsik]

And it won't be parsed, because those distro-specific version numbers are intentionally "invalid", so they are intentionally not parsed as part of actual software version :D

But they are also intentionally there, to give user a hint, that his software is packaged by Debian/RedHat/whatever maintainers, and it's not provided by original author/built from original code in 100%.

[phillebaba]

Thought about it and the best option is to add special handling for these version suffixes. Does anyone know if the version suffix format for Debian/Ubuntu is documented anywhere?

[morsik]

@phillebaba let me tell you from experience as someone who created packages for Red Hat distros, ArchLinux, Gentoo and even once for Debian over last 15 years... you don't want to do this :) I mean - special handling for specific distro - there is a lot of special cases, and in a year, you'll get identical Issue for another distro, and another year later for yet another distro.

Just don't do this, for your and our own sake :)

It would be better and easier to just verify beginning of version string if you really need to. But I see, you don't even need, as you're just checking major version.

This is for example standard for Debian: https://www.debian.org/doc/debian-policy/ch-controlfields.html#special-version-conventions

And oh... SUSE while it is using also RPMs like RedHat, they only just use RPM format for packages, but they have their own another standards for package naming and other packaging conventions...

[phillebaba]

I guess the option would be to check the start of the string. I know that I will need to do more version verification in the future, as I am planning on adding new features to Containerd. I will need to check the version to see if I can use the new features or fallback to the old behavior.

On the other hand I do not expect to have a bunch of users using a wide variety of distros, as a lot of people are using Kubernetes from one of the cloud providers. So if I had special handling for Debian now, it will most likely not open the floodgates for more issues. And if I am wrong feel free to quote me in future issues 😆