feat(plugins) Syncing plugin version to the Kong version

[tyler-ball]

Summary

With almost all Kong plugins bundled inside Kong itself we want to
remove customer confusion by syncing all plugin versions to the Kong
version. This allows users to:

1. Refer to the Kong version when reporting plugin issues, instead of
   having to look up the plugin version.
2. Update our plugin documentation to be Kong version centric instead
   of plugin version centric starting in Kong 3.0
3. Developers no longer need to remember to update the plugin version
   when making bundled plugin changes.
4. Simplify changelog management by only having 1 changelog for Kong

I suggest looking at and reviewing each commit individually as quite a few files were touched. I also deleted old unecessary files (rockspec, changelog, license).

Full changelog

• Sync every plugin version to the Kong version

Issue reference

FT-2775

[flrgh]

Wonderful!

Just to double check: are there any unreleased plugin-specific changelog entries (i.e. post 2.8) that need to get copy-pasted into the top-level changelog so they aren't lost?

[aboudreault]

to discuss: currently, this will break in kong-ee because _VERSION has the following format x.y.z-enterprise-edition. see: https://github.com/Kong/kong-ee/blob/master/kong/meta.lua#L26

[tyler-ball]

I'm going to update this to kong_meta.version and introduce the version key, matching what is in https://github.com/Kong/kong-ee/pull/3191/files#diff-f2919e234cadaf71fe36827f6b1e90f6fd8babd16cb39f6a71fee5fccc85e3acR29

[aboudreault]

done.

[tyler-ball]

NTS: check the / endpoint and see if it includes plugin versions

[tyler-ball]

We do not return plugin version info on the / endpoint, just loaded_plugins list and available_on_server plugins

[Tieske]

this looks like an extremely dangerous thing to do. The same plugin will report a different version on the next version of Kong. I'm not the expert on this but I think it will break all compatibilty checks between CP and DP when upgrading.

The version belongs to the plugin (it's plugin meta-data) and hence should be set there, never depend dynamically on the Kong version.

I think the better approach would be to update the plugin versions, hardcoded, not dynamic!, to the version of Kong in which they were shipped. Albeit that might defeat the purpose of this PR to reduce customer confusion.

my 2cts.

Ping @mikefero @hbagdi @RobSerafini

[mikefero]

this looks like an extremely dangerous thing to do. The same plugin will report a different version on the next version of Kong. I'm not the expert on this but I think it will break all compatibility checks between CP and DP when upgrading.

Version compatibility between the CP and DP currently use the Kong Gateway version and disregard plugin version; we went that route because we knew what features were bundled in each version of the gateway which simplified the logic quite a bit. With that stated we really should clean up the compatibility layer @tyler-ball to remove unnecessary checks that this PR creates in that layer (see check_configuration_version as one example).

I think the better approach would be to update the plugin versions, hardcoded, not dynamic!, to the version of Kong in which they were shipped. Albeit that might defeat the purpose of this PR to reduce customer confusion.

Are you suggesting doing this at release time? That is doable as well, but I think the end result would be the same. From the version compatibility standpoint this shouldn't cause any issues there, but does need to be thoroughly tested before merging this in.

[Tieske]

Are you suggesting doing this at release time?

I'd suggest that at release time we update all UPDATED plugins to match the new Kong version, so anything that didn't change, doesn't get updated. Especially the dynamic link to the Kong version makes me very uncomfortable, because why wouldn't we simply remove the plugin version field from the code in that case?

Version compatibility between the CP and DP currently use the Kong Gateway version and disregard plugin version

That is ignoring the custom plugins? We have folks running dozens of custom plugins on a single dataplane...

[mikefero]

because why wouldn't we simply remove the plugin version field from the code in that case?

I agree with you there, doesn't make much sense for sure. It does make sense to keep this in there for now and maybe deprecate in some future major; WDYT?

That is ignoring the custom plugins? We have folks running dozens of custom plugins on a single dataplane...

There isn't any version compatibility layer for custom plugins though; unless I am mistaken. I remember our conversation around moving all the VC logic into the plugins or out of the control_plane layer (which would be ideal), but we could still build that functionality into the gateway itself so custom plugin developers could add their own features for VC,

[tyler-ball]

because why wouldn't we simply remove the plugin version field from the code in that case?

I'm also in favor of this. I had the same conclusion and investigated doing it at one point. I don't remember exactly why I didn't go that route - I don't remember what reads the plugin version. I think I also wanted to introduce less risk. But I am in favor of removing the version entirely in the future

[Tieske]

Discussed this with @RobSerafini today. And by now I'm convinced we cannot remove the plugin versions. There will always be custom plugins and we need the version checks there anyway. Whether we like it or not we'll have to deal with compatibility logic.

There isn't any version compatibility layer for custom plugins though; unless I am mistaken. I remember our conversation around moving all the VC logic into the plugins or out of the control_plane layer (which would be ideal)

That would be the preferred way, since only the plugin itself has the contextual information to know how it should convert its config into an older-version-of-self config. But in that case; the plugin needs to carry a version, simply to know what the target version of the plugin is it needs to convert to.

We can take a shortcut to this by syncing all Kong plugin to the Kong core version. But in the end that's not enough because users will be running custom plugins. So imho we better have a single approach in this.

slightly related; #8810

Brain farts;

• the VERSION constant in plugins should move into schema.lua I think, so Koko can also access it.
• if the version is in the schema, the schema can also be the place where we add the version-conversion logic (since that is "simple Lua", whereas the handler.lua will probably pull in everything-but-the-kitchen-sink, which won't work in Koko)

[fffonion]

I love the tags cloud on this PR 😄 .

For attendees of this PR, do we have agreed on the path for this PR? Do we want to put this in 3.0?

[aboudreault]

@fffonion yes, we would like this PR in 3.0.

@hbagdi Can you comment about the following points, raised by @Tieske :

1. the VERSION constant in plugins should move into schema.lua I think, so Koko can also access it.
2. if the version is in the schema, the schema can also be the place where we add the version-conversion logic (since that is "simple Lua", whereas the handler.lua will probably pull in everything-but-the-kitchen-sink, which won't work in Koko)

[aboudreault]

As discussed offline, moving the VERSION to another file wont help.