deployed components should have fixed version

[phoracek]

We should use images with specific version instead of :latest. One reason is that we cannot be sure that our manifests will work with whatever is shipped in the image. Another reason is that we make sure all components work well together. Finally it would make upgrades more obvious - upgrade from components:x to components:y.

@SchSeba @booxter does it make sense and is it doable?

• SR-IOV DP
• SR-IOV CNI
• Linux bridge CNI
• Linux bridge marker
• Multus
• kubemacpool
• nmstate

TODO: Keep table of image/component versions matched for image release in README

[booxter]

@phoracek my feeling is SR-IOV upstream is not particularly interested in maintaining release tags / branches. There are several tags (notably, release-v1, latest, and snapshot) for some of them but I don't think there is plan to properly tag any of that upstream. I am also not sure whether docker / quay maintain all uploaded versions, or just latest; so when a new version is build, how long does the registry keep the older versions?

Is this a suggestion to go back to upstream and push for a more clear release versioning / branch management? Or is it e.g. a suggestion to build our own builds in kubevirt repos and use those instead of upstream builds?

[phoracek]

@booxter I'm not aware of them removing old tags, never heard of it. We need to stick to a specific version, otherwise there is no guarantee that after :latest image upgrade, manifest we keep in the operator will be still working. We can fix to a specific image hash, but that sounds risky.

Having proper tags on u/s would be ideal, but I'm not sure we could make it. Having image copies under kubevirt sounds like a blasphemy, but easier to handle.

There is a request/pattern in HCO. They want every operator to expose ImageRegistry and ImageTag in operator config. That is impossible while pulling various third-party images.

If we copy set of third-party images under our organization, we can test given set with the operator (and together), we can set them with a tag matching operator version and keep it all nice and tidy.

@booxter @dankenigsberg what do you think? Do we copy third-party images under our organization and make it simple and easily tested or do we fight u/s projects to tag their images (if they have u/s image, we still miss repo for kubemacpool and we need to build our own bridge CNI with VLAN support).

edit: also, if we keep our custom set of images on u/s, it will be way easier to mirror them to d/s, since we keep them all together with known version

[dankenigsberg]

An upstream versioned image is best.

For projects that have upstream versions, a versioned kubevirt image is a close second.

If we need d/s only patches (eg no recent upstream versions) it would be cool if we had RPM-style release number that includes the githash, so the operator always takes the latest of those.

"latest" sounds fragile, and should be used only if we are too lazy/busy to get a clear tag.

Please fight with u/s until they have versions and formal releases. But until they do, we must have our own images.

If we can avoid maintaining our own image by exposing SriovImageRegistry that defaults to ImageRegistry, I think we should.

HTH.

[phoracek]

Thanks @dankenigsberg

Could we do this?

• All images will be kept in operator config as (alternative would be SriovDPRegistry, SriovDPImageName, SriovDPTag; not sure which would be better for HCO integration Express ContainerRegistry, ContainerTag, and PullPolicy on the operator's CR #43, what do you think @rthallisey ):

SriovDPImage: registry/image_name:image_tag
...

• Note that if someone overrides the default image (e.g. on d/s), they need to make sure that their version is ~same as on u/s (so operator manifests will work)

• All images must be versioned

• If there is good enough tag on u/s, use it

• If there is none (or we need newer version), make temporary kubevirt/cluster-network-addon-<component_name>:<operator_version>, once we get u/s tagged, we use it

Would you agree?

[rthallisey]

I think that's fine @phoracek. The HCO only needs a mapping from version, containerRegistry, and imagePullPolicy to whatever you expose. You'll just need to add something for imagePullPolicy.

• version -> image_tag
• containerRegistry -> registry
• imagePullPolicy -> ?

[phoracek]

Thanks. We already have single imagePullPolicy parameter in operator's CR applied to all deployed images. Will start with adding images to the CR, making it possible to switch between them live. That should bring us closer to possible upgrades.

[phoracek]

kubemacpool resolved in #83

[phoracek]

Linux bridge CNI resolved in #85

[booxter]

@phoracek do we just pick initial version at random? Like v0.1.0? Or is there some guidance?

[SchSeba]

@booxter I think v0.1.0 should be fine

[phoracek]

@SchSeba @booxter I'm not sure about the versions. We cannot use u/s version, since they do not version it, that's fine. However, if we introduce our own versioning we would need to maintain a table saying from which u/s version we built it (otherwise it would get pretty chaotic). Do you think we could use just short commit hash (first 7 characters from the commit id) instead? There is nothing forcing us to use semver AFAIK.

[dankenigsberg]

I think we could still use semver: use the latest upstream version, then add release, which may include the downstream githash.

This would give us a chance to re-sync with upstream if it wakes up and releases a new version.

e.g for
https://github.com/intel/sriov-network-device-plugin/releases
v2.0.0-8.gitdeadd00d

[SchSeba]

@phoracek Sure no problem.

@dankenigsberg before my change for multus and sriov-cni we pull from latest(not a specific version) and for the device plugins we pull from @booxter repo.

PR #92

Can we leave the version tag (v0.1.0) with the commit hash as requested by petr?

[phoracek]

@SchSeba if I understood @dankenigsberg properly, for multus, we should do v3.2-1.gitbf61002, where v3.2 is the last release, shows this is our first release and commit id bf61002 is the top of master. If we were to ship released version (not latest), we would simply drop the git commit ID part.

[SchSeba]

@booxter what version should I tag the sriov-device-plugin?

[booxter]

@SchSeba (answered the other place but) since they have release-v1 branches for both CNI and DP, I assume anything below v1 is no-go. I wouldn't suggest going far from v1 either, nor introducing our own minor / patch versions w/o their involvement.

So in essence, v1. is probably the way to go. (Unless we have some constraints in operator upgrade that would require something more semantic.)

[rthallisey]

I'd expect that these vars are defaulted by your operator if they are not explicitly set on the CR. You probably already have that, but in case you don't, just providing a reminder.

[SchSeba]

@rthallisey right now the configuration about the version of every component is describe in the operator itself not int the CR.

Operator - https://github.com/kubevirt/cluster-network-addons-operator/blob/master/manifests/cluster-network-addons/0.5.0/operator.yaml#L131
CR - https://github.com/kubevirt/cluster-network-addons-operator/blob/master/manifests/cluster-network-addons/0.5.0/network-addons-config-example.cr.yaml

[phoracek]

All components are now in their fixed versions.