[functions] Default functionAuthProvider when running in k8s #6203

addisonj · 2020-02-04T01:37:13Z

Motivation

In 2.4.x, when running with the KubernetesRuntime, it default to always
using the KubernetesSecretAuthProvider class. With the change in 2.5 to
making this behavior pluggable, there is currently a bug in that it
doesn't keep this behavior and requires a new configuration option to be
passed.

Modifications

This commit changes the config so that it defaults to the correct class
when we are running with a kubernetes runtime. This restores the
behavior match that of earlier versions

Verifying this change

Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

Added a test that ensures that we get the correct default based on other config values

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): n
The public API: n
The schema: n
The default values of configurations: y
The wire protocol:n
The rest endpoints: n
The admin cli options: n
Anything that affects deployment: n

Documentation

Does this pull request introduce a new feature? n
If yes, how is the feature documented? n/a
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

sijie · 2020-02-05T00:43:29Z

@addisonj can you rebase this pull request to include some improvements in the github action files?

addisonj · 2020-02-05T23:49:37Z

@sijie I did the rebase, looks to have still failed some

sijie · 2020-02-07T00:26:14Z

@addisonj can you try to rebase again? I just merge a pull request with bunch of improvements in integration tests.

tuteng · 2020-02-15T01:54:06Z

Please take a look at this test: https://github.com/apache/pulsar/pull/6203/checks?check_run_id=446244014 @addisonj

The exception was thrown with the wrong message: expected "Function language runtime is either not set or cannot be determined" but got "null"
	at org.testng.internal.ExpectedExceptionsHolder.wrongException(ExpectedExceptionsHolder.java:71)
	at org.testng.internal.Invoker.considerExceptions(Invoker.java:1130)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:615)
	at org.testng.internal.Invoker.retryFailed(Invoker.java:839)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1010)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)

tuteng · 2020-02-24T12:38:34Z

ping @addisonj Please try rebase master.

addisonj · 2020-02-25T19:38:34Z

@tuteng rebased and ran the tests locally and they appear to pass

addisonj · 2020-02-25T20:27:33Z

whoops, ignore the last comment, am able to repro now, will fix

In 2.4.x, when running with the KubernetesRuntime, it default to always using the KubernetesSecretAuthProvider class. With the change in 2.5 to making this behavior pluggable, there is currently a bug in that it doesn't keep this behavior and requires a new configuration option to be passed. This commit changes the config so that it defaults to the correct class when we are running with a kubernetes runtime. This restores the behavior match that of earlier versions This also moves the WorkerConfig test to the same package where the workerConfig resides after the refactor and re-arranges the resources files and copied via a maven task

addisonj · 2020-03-05T15:12:52Z

/pulsarbot run-failure-checks

addisonj · 2020-03-05T22:00:55Z

/pulsarbot run-failure-checks

addisonj · 2020-03-06T01:05:13Z

/pulsarbot run-failure-checks

addisonj · 2020-03-06T03:04:50Z

@tuteng @sijie It looks like all the tests passed at least once, notice the CPP tests are on the list twice, one passed? I think this is ready for merge

addisonj · 2020-03-13T17:41:29Z

@sijie you still want this for 2.5.1? All tests passed on this, it just looks like github is showing the checks twice

addisonj · 2020-03-18T22:10:59Z

this looks good to merge @sijie, not sure if you want for 2.5.1?

sijie · 2020-03-18T23:40:39Z

@addisonj thank you for your contribution!

…6203) In 2.4.x, when running with the KubernetesRuntime, it default to always using the KubernetesSecretAuthProvider class. With the change in 2.5 to making this behavior pluggable, there is currently a bug in that it doesn't keep this behavior and requires a new configuration option to be passed. This commit changes the config so that it defaults to the correct class when we are running with a kubernetes runtime. This restores the behavior match that of earlier versions This also moves the WorkerConfig test to the same package where the workerConfig resides after the refactor and re-arranges the resources files and copied via a maven task Co-authored-by: Addison Higham <ahigham@instructure.com> (cherry picked from commit 3a3174b)

In 2.4.x, when running with the KubernetesRuntime, it default to always using the KubernetesSecretAuthProvider class. With the change in 2.5 to making this behavior pluggable, there is currently a bug in that it doesn't keep this behavior and requires a new configuration option to be passed. This commit changes the config so that it defaults to the correct class when we are running with a kubernetes runtime. This restores the behavior match that of earlier versions This also moves the WorkerConfig test to the same package where the workerConfig resides after the refactor and re-arranges the resources files and copied via a maven task Co-authored-by: Addison Higham <ahigham@instructure.com> (cherry picked from commit 3a3174b)

…6203) In 2.4.x, when running with the KubernetesRuntime, it default to always using the KubernetesSecretAuthProvider class. With the change in 2.5 to making this behavior pluggable, there is currently a bug in that it doesn't keep this behavior and requires a new configuration option to be passed. This commit changes the config so that it defaults to the correct class when we are running with a kubernetes runtime. This restores the behavior match that of earlier versions This also moves the WorkerConfig test to the same package where the workerConfig resides after the refactor and re-arranges the resources files and copied via a maven task Co-authored-by: Addison Higham <ahigham@instructure.com>(cherry picked from commit 3a3174b)

…6203) In 2.4.x, when running with the KubernetesRuntime, it default to always using the KubernetesSecretAuthProvider class. With the change in 2.5 to making this behavior pluggable, there is currently a bug in that it doesn't keep this behavior and requires a new configuration option to be passed. This commit changes the config so that it defaults to the correct class when we are running with a kubernetes runtime. This restores the behavior match that of earlier versions This also moves the WorkerConfig test to the same package where the workerConfig resides after the refactor and re-arranges the resources files and copied via a maven task Co-authored-by: Addison Higham <ahigham@instructure.com>

sijie approved these changes Feb 4, 2020

View reviewed changes

sijie requested a review from jerrypeng February 4, 2020 01:45

sijie assigned addisonj Feb 4, 2020

sijie added release/2.5.1 area/function labels Feb 4, 2020

sijie added this to the 2.6.0 milestone Feb 4, 2020

addisonj force-pushed the default_k8s_secrets branch from 27f4b9f to 26d031a Compare February 4, 2020 02:08

sijie closed this Feb 4, 2020

sijie reopened this Feb 4, 2020

addisonj force-pushed the default_k8s_secrets branch from 26d031a to d120650 Compare February 5, 2020 20:42

addisonj closed this Feb 6, 2020

addisonj reopened this Feb 6, 2020

addisonj force-pushed the default_k8s_secrets branch from d120650 to 214c71e Compare February 7, 2020 17:35

addisonj force-pushed the default_k8s_secrets branch from 214c71e to 1dc21b4 Compare February 25, 2020 19:37

addisonj force-pushed the default_k8s_secrets branch 2 times, most recently from 2746814 to baa928c Compare March 2, 2020 22:14

addisonj force-pushed the default_k8s_secrets branch from baa928c to f5c4502 Compare March 3, 2020 00:05

addisonj closed this Mar 5, 2020

addisonj reopened this Mar 5, 2020

sijie approved these changes Mar 6, 2020

View reviewed changes

sijie merged commit 3a3174b into apache:master Mar 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[functions] Default functionAuthProvider when running in k8s #6203

[functions] Default functionAuthProvider when running in k8s #6203

addisonj commented Feb 4, 2020

sijie commented Feb 5, 2020

addisonj commented Feb 5, 2020

sijie commented Feb 7, 2020

tuteng commented Feb 15, 2020

tuteng commented Feb 24, 2020 •

edited

Loading

addisonj commented Feb 25, 2020

addisonj commented Feb 25, 2020

addisonj commented Mar 5, 2020

addisonj commented Mar 5, 2020

addisonj commented Mar 6, 2020

addisonj commented Mar 6, 2020

addisonj commented Mar 13, 2020

addisonj commented Mar 18, 2020

sijie commented Mar 18, 2020

[functions] Default functionAuthProvider when running in k8s #6203

[functions] Default functionAuthProvider when running in k8s #6203

Conversation

addisonj commented Feb 4, 2020

Motivation

Modifications

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

sijie commented Feb 5, 2020

addisonj commented Feb 5, 2020

sijie commented Feb 7, 2020

tuteng commented Feb 15, 2020

tuteng commented Feb 24, 2020 • edited Loading

addisonj commented Feb 25, 2020

addisonj commented Feb 25, 2020

addisonj commented Mar 5, 2020

addisonj commented Mar 5, 2020

addisonj commented Mar 6, 2020

addisonj commented Mar 6, 2020

addisonj commented Mar 13, 2020

addisonj commented Mar 18, 2020

sijie commented Mar 18, 2020

tuteng commented Feb 24, 2020 •

edited

Loading