Issue 8558: Upgrade Kubernetes client and remove sundr-codegen (allow Pulsar to run on JDK14+)

[eolivelli]

Fixes #8558

Motivation

sundr-codegen jar is not needed at runtime and in bundles legacy versions of commons-lang and other libraries, this is polluting the runtime classpath of Pulsar Broker,Bookie,Functions...
The depedency comes from Kubernetes java client, used by Pulsar Functions.

sundrio/sundrio#153

Modifications

• remove sundr-codegen from distribution lib
• allow Pulsar bookie to run on JDK14+
• upgrade Kubernetes java client
• upgrade commons-lang3

Verifying this change

I have verified this change manually.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): yes

[eolivelli]

@codelipenghui PTAL to this change as well, otherwise it is not possible to run Pulsar on JDK14+ using the standard package.

at MagNews.com we are already running Pulsar Broker on JDK14 and also JDK15 without issues, but we are not using the ASF package

[sijie]

@eolivelli Have you verified if this change works for running functions with Kubernetes runtime?

[eolivelli]

@sijie no I didn't and I do not have a way to do it right now, I am sorry.

It would be really better to test it

[sijie]

@eolivelli I would like to hold on this until 2.7.0 is released. Because we found upgrading Kubernetes version potentially broke the build in the past (see #8601).

[eolivelli]

@sijie
not a problem from my side.
I think it is safer to postpone

[sijie]

I think we should add an integration test framework to test kubernetes runtime. @wolfstudy Can you help with that?

[eolivelli]

@wolfstudy feel free to pick up this patch
thank you

[eolivelli]

@wolfstudy do you have time to pick up this issue ?
it is very annoying for people running on a recent Java version

[wolfstudy]

@eolivelli Sorry, I missed this message, Sure, I will try to add an integration test to cover this change

[eolivelli]

@wolfstudy if you do not have cycles
Can you please point me to some other kind of k8s tests we have here in Pulsar ?
This way I can try to add the necessary tests

[eolivelli]

@wolfstudy @sijie I am going to try out this brach on a K8s cluster manually.
Any suggestion about where to put an integration tests is very appreciated.

[eolivelli]

@sijie @wolfstudy
I have tested locally this branch (using the helm-chart) and it works very well.
I believe there is no need to add integration tests in order to commit this patch.

We can add system tests, probably the best place would be nearby the helm-chart and not in this repository.

[eolivelli]

/pulsarbot run-failure-checks

[eolivelli]

/pulsarbot run-failure-checks

[sijie]

@eolivelli I think the integration tests should be added in this repo. The helm chart repo is used for testing the helm chart. You can clone the helm chart repo in the CI job to start a KIND cluster for testing.

[eolivelli]

@eolivelli I think the integration tests should be added in this repo. The helm chart repo is used for testing the helm chart. You can clone the helm chart repo in the CI job to start a KIND cluster for testing.

@sijie agreed, we can make it here (or in the new 'release repo' once PIP-62 is completed
but I would keep it out of the scope of this patch

I created a new issue
#9687

[eolivelli]

@rdhabalia @merlimat @codelipenghui PTAL as well

[eolivelli]

@codelipenghui @congbobo184 @wolfstudy can you please take a look ?

[eolivelli]

I have merged with current master in order to bootstrap CI again.

Thank you @sijie for taking a look

[eolivelli]

/pulsarbot run-failure-tests

[eolivelli]

@wolfstudy can you please help merging this patch ?

[eolivelli]

/pulsarbot run-failure-tests

[eolivelli]

/pulsarbot rerun-failure-checks

[eolivelli]

/pulsarbot rerun-failure-checks

[eolivelli]

@wolfstudy @codelipenghui @rdhabalia CI finally passed.
I believe this is a good time to merge this patch