fix: respect authentication.skip.paths properly #9224

agavra · 2022-06-24T22:25:24Z

Description

fixes #9206 by respecting the authentication.skip.paths in KsqlAuthorizationProviderHandler.java

Testing done

spun up CFK with MTLS and RBAC enabled on the normal endpoints. see relevant portion of configuration:

    authentication.skip.paths=/chc*,/heartbeat,/lag
    bootstrap.servers=kafka.operator-feyrhf.svc.cluster.local:9073
    config.providers=file
    config.providers.file.class=org.apache.kafka.common.config.provider.FileConfigProvider
    confluent.metadata.basic.auth.user.info=${file:/mnt/secrets/ksqldb-mds-client/bearer.txt:username}:${file:/mnt/secrets/ksqldb-mds-client/bearer.txt:password}
    confluent.metadata.bootstrap.server.urls=https://kafka.operator-feyrhf.svc.cluster.local:8090
    confluent.metadata.http.auth.credentials.provider=BASIC
    confluent.metadata.ssl.truststore.location=/mnt/sslcerts/truststore.jks
    confluent.metadata.ssl.truststore.password=${file:/mnt/sslcerts/jksPassword.txt:jksPassword}
    ksql.authentication.plugin.class=io.confluent.ksql.security.VertxBearerOrBasicAuthenticationPlugin
    ksql.heartbeat.enable=true
    ksql.lag.reporting.enable=true
    ksql.query.pull.enable.standby.reads=true
    ksql.security.extension.class=io.confluent.ksql.security.KsqlConfluentSecurityExtension
    ksql.streams.num.standby.replicas=1
    sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required metadataServerUrls="https://kafka.operator-feyrhf.svc.cluster.local:8090" username="${file:/mnt/secrets/ksqldb-mds-client/bearer.txt:username}" password="${file:/mnt/secrets/ksqldb-mds-client/bearer.txt:password}";
    sasl.login.callback.handler.class=io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler
    sasl.mechanism=OAUTHBEARER
    security.protocol=SASL_SSL
    ssl.enabled.protocols=TLSv1.2
    ssl.key.password=${file:/mnt/sslcerts/jksPassword.txt:jksPassword}
    ssl.keystore.location=/mnt/sslcerts/keystore.jks
    ssl.keystore.password=${file:/mnt/sslcerts/jksPassword.txt:jksPassword}
    ssl.truststore.location=/mnt/sslcerts/truststore.jks
    ssl.truststore.password=${file:/mnt/sslcerts/jksPassword.txt:jksPassword}

setting up unit testing for this is nigh impossible and the fix is blocking production users

Reviewer checklist

Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
Ensure relevant issues are linked (description should include text like "Fixes #")

pgaref · 2022-06-24T22:51:45Z

ksqldb-rest-app/src/main/java/io/confluent/ksql/api/auth/KsqlAuthorizationProviderHandler.java

  }

  @Override
  public void handle(final RoutingContext routingContext) {

    final String path = routingContext.normalisedPath();

-    if (KSQL_AUTHENTICATION_SKIP_PATHS.contains(path)) {
+    if (unauthenticatedpaths.matcher(path).matches()) {


Oh wow, so AUTHENTICATION_SKIP_PATHS_CONFIG was not taken into account at all before.

pgaref

Thanks for the FIX @agavra !
Changes LGTM

pgaref · 2022-06-24T22:54:07Z

This should also land to 7.0.x etc right?

cadonna · 2022-06-27T07:08:59Z

@agavra Thanks a lot for the fix!
While I see that unit testing the whole setup is hard, I do not understand why unit testing AuthenticationPluginHandler#getAuthorizationSkipPaths() and KsqlAuthorizationProviderHandler#handle() should be hard. Could you clarify?

cadonna · 2022-06-27T07:14:05Z

This should also land to 7.0.x etc right?

@pgaref Once this PR is merged to 6.0.x, we need to pint merge the commit so that it will bubble up to all <major>.<minor>.x branches including 7.0.x and finally end up on master. See https://confluentinc.atlassian.net/wiki/spaces/QERM/pages/69009670/Release+Branches+and+Tags for more details about CPs branching strategy.

agavra · 2022-06-27T15:02:34Z

@cadonna there are no existing tests for these classes - I will add tests for AuthenticationPluginHandler#getAuthorizationSkipPaths() but tests for KsqlAuthorizationProviderHandler#handle() require a lot of mocking, and to be honest, won't add any meaningful coverage.

EDIT: I see what you mean about testing handle. I'll test that to make sure it picks up the right skip paths.

cadonna

@agavra Thanks for the update!

I have two comments regarding the tests.

cadonna · 2022-06-27T16:34:15Z

ksqldb-rest-app/src/test/java/io/confluent/ksql/api/auth/AuthenticationPluginHandlerTest.java

+    final Pattern skips = AuthenticationPluginHandler.getAuthorizationSkipPaths(configs);
+
+    // Then:
+    assertThat(skips.matcher("/heartbeat").matches(), is(true));


Why not also verify for /lags?
Could you also add a negative case, like /shouldnotbefound?

Why not also verify for /lags?

that doesn't add any coverage, I'm ensuring that what is added in the config is able to skip

Could you also add a negative case, like /shouldnotbefound?

Sure I'll add a negative test

that doesn't add any coverage, I'm ensuring that what is added in the config is able to skip

I cannot follow. Why does verifying for /lag not add any coverage?

adding /lag in the config list makes sure that the regex works when you have multiple components - I'm not sure why checking that it works for /lag adds any coverage beyond checking for /heartbeat (basically it uses the same exact code path to check for /lag as it does to check for /heartbeat)

The additional verification would check that the code works for all entries in the config. The code could not consider the other entry. Unit test should not only test the current code but they should also ensure that future refactorings of the code do not change the intended behavior.

I suppose I can see your point :) I'll add it in.

cadonna · 2022-06-27T16:35:08Z

...-rest-app/src/test/java/io/confluent/ksql/api/auth/KsqlAuthorizationProviderHandlerTest.java

+  private RoutingContext routingContext;
+
+  @Test
+  public void shouldRespectServerAuthSkipPathsConfig() {


Could you also add a case where authentication is not skipped?

agavra requested a review from a team as a code owner June 24, 2022 22:25

agavra changed the base branch from master to 6.2.x June 24, 2022 22:26

fix: respect authentication.skip.paths properly

0d8c4cf

agavra force-pushed the fix-skip-paths branch from 805481b to 0d8c4cf Compare June 24, 2022 22:30

agavra changed the base branch from 6.2.x to 6.0.x June 24, 2022 22:30

nateab approved these changes Jun 24, 2022

View reviewed changes

pgaref reviewed Jun 24, 2022

View reviewed changes

pgaref approved these changes Jun 24, 2022

View reviewed changes

test: add tests

946ae5e

cadonna reviewed Jun 27, 2022

View reviewed changes

test: add more tests

403801a

agavra merged commit 4c33edd into confluentinc:6.0.x Jun 27, 2022

agavra deleted the fix-skip-paths branch June 27, 2022 18:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: respect authentication.skip.paths properly #9224

fix: respect authentication.skip.paths properly #9224

agavra commented Jun 24, 2022 •

edited

pgaref Jun 24, 2022

pgaref left a comment

pgaref commented Jun 24, 2022

cadonna commented Jun 27, 2022

cadonna commented Jun 27, 2022

agavra commented Jun 27, 2022 •

edited

cadonna left a comment

cadonna Jun 27, 2022

agavra Jun 27, 2022

cadonna Jun 27, 2022 •

edited

agavra Jun 27, 2022

cadonna Jun 27, 2022

agavra Jun 27, 2022

agavra Jun 27, 2022

cadonna Jun 27, 2022

fix: respect authentication.skip.paths properly #9224

fix: respect authentication.skip.paths properly #9224

Conversation

agavra commented Jun 24, 2022 • edited

Description

Testing done

Reviewer checklist

Choose a reason for hiding this comment

pgaref left a comment

Choose a reason for hiding this comment

pgaref commented Jun 24, 2022

cadonna commented Jun 27, 2022

cadonna commented Jun 27, 2022

agavra commented Jun 27, 2022 • edited

cadonna left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cadonna Jun 27, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agavra commented Jun 24, 2022 •

edited

agavra commented Jun 27, 2022 •

edited

cadonna Jun 27, 2022 •

edited