branch-4.1: [feature] Add information_schema role mappings table#62417
branch-4.1: [feature] Add information_schema role mappings table#62417CalvinKirs wants to merge 13 commits into
Conversation
…role mapping (apache#61819) apache#60361 This PR integrates the Doris FE authentication extension modules and delivers the end-to-end OIDC login path for MySQL clients. The core changes are: 1. Add MySQL protocol-layer support for OIDC login - bridge OIDC credentials carried in the MySQL authentication packet into Doris authentication requests - allow OIDC-based external login to enter the FE authentication pipeline through the MySQL protocol path 2. Add ROLE MAPPING for built-in authorization - support using external identity attributes/groups to drive internal authorization decisions 3. Integrate the authentication feature modules into FE - wire the authentication plugin/integration modules into the FE runtime - connect external authentication, JIT/external login, and built-in role authorization into one complete flow With these changes, Doris can accept OIDC-based external logins from the MySQL protocol entry, authenticate them through the integrated authentication modules, and apply Doris-native authorization through ROLE MAPPING. (cherry picked from commit c116852)
[[feature](fe) Add information_schema role mappings table](apache@399d8b0) apache#60361 Issue Number: None Related PR: None Problem Summary: Add information_schema.role_mappings so administrators can inspect authentication integration role mapping definitions and audit fields through SQL. Add information_schema.role_mappings for inspecting role mapping metadata. - Test: FE Unit Test; Regression test attempted - FE Unit Test: ./run-fe-ut.sh --run org.apache.doris.service.FrontendServiceImplTest - FE Unit Test: ./run-fe-ut.sh --run org.apache.doris.catalog.SchemaTableTest - Regression test: ./run-regression-test.sh --run -d auth_p0 -s test_role_mapping_system_table (blocked locally because the default 127.0.0.1:9030 cluster did not include CREATE ROLE MAPPING syntax) - Behavior changed: Yes (adds information_schema.role_mappings) - Does this need documentation: No (cherry picked from commit e2e5159)
### What problem does this PR solve? Issue Number: close apache#60361 Related PR: apache#61819, apache#62077, apache#61362 Problem Summary: Backport the missing FE-side authentication chain compatibility pieces required by the branch-4.1 OIDC authentication and role mapping cherry-picks so the FE build succeeds on auth_branch-4.1. ### Release note None ### Check List (For Author) - Test: env DORIS_PARALLELISM=16 BUILD_TYPE=ASAN ./build.sh --fe -j16 - Manual test - Behavior changed: Yes (restores FE-side authentication chain compatibility for the branch-4.1 auth backport) - Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
### What problem does this PR solve? Issue Number: close apache#60361 Related PR: apache#61362 Problem Summary: Restore the missing test authentication plugin factory and SPI registration that were omitted while backporting the authentication chain changes to branch-4.1, so the fe-core auth plugin tests cover the same path as master. ### Release note None ### Check List (For Author) - Test: FE unit test - mvn -f pom.xml -pl fe-core -am -Dtest=org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest,org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest,org.apache.doris.authentication.AuthenticationIntegrationMgrTest test -DfailIfNoTests=false (remaining 1 failure is existing OIDC fallback assertion mismatch in AuthenticatorManagerTest) - Behavior changed: No - Does this need documentation: No
### What problem does this PR solve? Issue Number: None Related PR: apache#62159 Problem Summary: Preserve the original OIDC auth packet when native auth switch runs on branch-4.1, normalize OIDC access-token requests to OAUTH_TOKEN across auth packet extraction and authentication_chain fallback, and create AuthenticationPluginManager under AuthenticationPluginAuthenticator's classloader context so SPI lookup does not depend on the caller thread context classloader. ### Release note None ### Check List (For Author) - Test: FE unit test - mvn -f pom.xml -pl fe-core -am -Dskip.clean=true -Dtest=org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.mysql.authenticate.MysqlAuthPacketCredentialExtractorTest,org.apache.doris.mysql.authenticate.plugin.AuthenticationPluginAuthenticatorTest test -DfailIfNoTests=false - Behavior changed: Yes (branch-4.1 now preserves the original OIDC auth packet across native auth-switch fallback, uses OAUTH_TOKEN for OIDC access-token requests in fe-core auth flows, and resolves auth plugin SPI factories without relying on the caller thread context classloader) - Does this need documentation: No
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
Removed 'constraintManager' from the list of modules in PersistMetaModules.java.
|
run buildall |
FE UT Coverage ReportIncrement line coverage |
FE Regression Coverage ReportIncrement line coverage |
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage |
FE Regression Coverage ReportIncrement line coverage |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
@copilot resolve the merge conflicts in this pull request |
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: Bring auth_branch-4.1 up to date with rich/branch-4.1 and resolve the FrontendServiceImplTest merge conflict.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- ./run-fe-ut.sh --run org.apache.doris.service.FrontendServiceImplTest
- Behavior changed: No
- Does this need documentation: No
|
run buildall |
…IDC access token auth (apache#62159) apache#60361 Problem Summary: FE authentication plugin loading currently assumes a single plugin root in several runtime paths, which makes external plugin deployment less flexible and inconsistent across callers. In addition, OIDC token-based authentication uses inconsistent credential typing across MySQL authentication and authentication_chain fallback flows, and the client-visible error message handling is not aligned with access-token- specific failures. This change makes plugin root parsing reusable across FE runtime paths, allows configuring multiple authentication or authorization plugin roots with a comma-separated list, normalizes OIDC access token requests to use `OAUTH_TOKEN`, broadens OIDC request detection in fallback authentication flow, and aligns access-token-related failure messages. It also marks `authentication_chain` as mutable so the fallback chain can be adjusted dynamically.
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
…lback config (apache#61362) This PR adds the FE-side authentication capability required to integrate `authentication integration` into Doris. Previously, Doris login was still based on the legacy authenticator flow. That made `authentication integration` incomplete in practice: - `authentication_type` could not naturally use an authentication plugin as the primary authenticator - MySQL login could not execute `authentication_chain` as a standard integration fallback path - authentication integration/plugin failures were not classified, so Doris could not decide clearly when to continue the chain and when to stop - authentication integrations did not have an independent runtime path for preparation, activation, and execution during login This PR fills in that missing path, so Doris can use auth integration in the FE MySQL authentication flow. How it works: 1. Doris chooses the primary MySQL authenticator from `authentication_type`. - Existing `default` / `password` / `ldap` behavior is preserved - plugin names are now also supported as primary authenticators 2. If `authentication_type` points to a plugin, Doris uses `AuthenticationPluginAuthenticator` to adapt the authentication plugin to the legacy MySQL `Authenticator` interface. 3. If primary authentication succeeds, Doris maps the authenticated principal to: - an existing Doris user, or - a temporary JIT user when `enable_jit_user` is enabled on the integration/plugin side. 4. If primary authentication fails and `authentication_chain` is configured, Doris falls back to `AuthenticationIntegrationAuthenticator`, which executes the configured integrations in order. 5. Each integration/plugin failure is tagged with a typed failure category. Doris continues the chain only for failures that should be retryable in chain mode (for example `USER_NOT_FOUND` or `SOURCE_UNAVAILABLE`), and stops on terminal failures such as bad credentials or access denied. 6. If one integration in the chain succeeds, login succeeds. Otherwise Doris returns the final authentication failure.
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
FE UT Coverage ReportIncrement line coverage |
FE Regression Coverage ReportIncrement line coverage |
#61819
#62077
#61362