Skip to content

branch-4.1: [feature] Add information_schema role mappings table#62417

Open
CalvinKirs wants to merge 13 commits into
apache:branch-4.1from
CalvinKirs:auth_branch-4.1
Open

branch-4.1: [feature] Add information_schema role mappings table#62417
CalvinKirs wants to merge 13 commits into
apache:branch-4.1from
CalvinKirs:auth_branch-4.1

Conversation

@CalvinKirs
Copy link
Copy Markdown
Member

@CalvinKirs CalvinKirs commented Apr 13, 2026

…role mapping (apache#61819)

apache#60361

This PR integrates the Doris FE authentication extension modules and
delivers the end-to-end OIDC login path for MySQL clients.

  The core changes are:

  1. Add MySQL protocol-layer support for OIDC login
- bridge OIDC credentials carried in the MySQL authentication packet
into Doris authentication requests
- allow OIDC-based external login to enter the FE authentication
pipeline through the MySQL protocol path

  2. Add ROLE MAPPING for built-in authorization
- support using external identity attributes/groups to drive internal
authorization decisions

  3. Integrate the authentication feature modules into FE
- wire the authentication plugin/integration modules into the FE runtime
- connect external authentication, JIT/external login, and built-in role
authorization into one complete flow

With these changes, Doris can accept OIDC-based external logins from the
MySQL protocol entry, authenticate them through the integrated
authentication modules, and apply Doris-native authorization
  through ROLE MAPPING.

(cherry picked from commit c116852)
[[feature](fe) Add information_schema role mappings
table](apache@399d8b0)
apache#60361

Issue Number: None

Related PR: None

Problem Summary: Add information_schema.role_mappings so administrators
can inspect authentication integration role mapping definitions and
audit fields through SQL.

Add information_schema.role_mappings for inspecting role mapping
metadata.

- Test: FE Unit Test; Regression test attempted
- FE Unit Test: ./run-fe-ut.sh --run
org.apache.doris.service.FrontendServiceImplTest
- FE Unit Test: ./run-fe-ut.sh --run
org.apache.doris.catalog.SchemaTableTest
- Regression test: ./run-regression-test.sh --run -d auth_p0 -s
test_role_mapping_system_table (blocked locally because the default
127.0.0.1:9030 cluster did not include CREATE ROLE MAPPING syntax)
- Behavior changed: Yes (adds information_schema.role_mappings)
- Does this need documentation: No

(cherry picked from commit e2e5159)
### What problem does this PR solve?

Issue Number: close apache#60361

Related PR: apache#61819, apache#62077, apache#61362

Problem Summary: Backport the missing FE-side authentication chain compatibility pieces required by the branch-4.1 OIDC authentication and role mapping cherry-picks so the FE build succeeds on auth_branch-4.1.

### Release note

None

### Check List (For Author)

- Test: env DORIS_PARALLELISM=16 BUILD_TYPE=ASAN ./build.sh --fe -j16
    - Manual test
- Behavior changed: Yes (restores FE-side authentication chain compatibility for the branch-4.1 auth backport)
- Does this need documentation: No
@CalvinKirs CalvinKirs requested a review from yiguolei as a code owner April 13, 2026 03:00
@Thearas
Copy link
Copy Markdown
Contributor

Thearas commented Apr 13, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

### What problem does this PR solve?

Issue Number: close apache#60361

Related PR: apache#61362

Problem Summary: Restore the missing test authentication plugin factory and SPI registration that were omitted while backporting the authentication chain changes to branch-4.1, so the fe-core auth plugin tests cover the same path as master.

### Release note

None

### Check List (For Author)

- Test: FE unit test
    - mvn -f pom.xml -pl fe-core -am -Dtest=org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.authentication.AuthenticationIntegrationRuntimeTest,org.apache.doris.mysql.authenticate.integration.AuthenticationIntegrationAuthenticatorTest,org.apache.doris.authentication.AuthenticationIntegrationMgrTest test -DfailIfNoTests=false (remaining 1 failure is existing OIDC fallback assertion mismatch in AuthenticatorManagerTest)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: apache#62159

Problem Summary: Preserve the original OIDC auth packet when native auth switch runs on branch-4.1, normalize OIDC access-token requests to OAUTH_TOKEN across auth packet extraction and authentication_chain fallback, and create AuthenticationPluginManager under AuthenticationPluginAuthenticator's classloader context so SPI lookup does not depend on the caller thread context classloader.

### Release note

None

### Check List (For Author)

- Test: FE unit test
    - mvn -f pom.xml -pl fe-core -am -Dskip.clean=true -Dtest=org.apache.doris.mysql.authenticate.AuthenticatorManagerTest,org.apache.doris.mysql.authenticate.MysqlAuthPacketCredentialExtractorTest,org.apache.doris.mysql.authenticate.plugin.AuthenticationPluginAuthenticatorTest test -DfailIfNoTests=false
- Behavior changed: Yes (branch-4.1 now preserves the original OIDC auth packet across native auth-switch fallback, uses OAUTH_TOKEN for OIDC access-token requests in fe-core auth flows, and resolves auth plugin SPI factories without relying on the caller thread context classloader)
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.47% (1797/2290)
Line Coverage 64.15% (32288/50334)
Region Coverage 65.12% (16249/24952)
Branch Coverage 55.64% (8684/15608)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.41% (758/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.99% (19882/37520)
Line Coverage 36.54% (187262/512496)
Region Coverage 32.89% (145569/442545)
Branch Coverage 34.02% (63750/187410)

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 90.59% (77/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.48% (26253/36726)
Line Coverage 54.41% (277895/510778)
Region Coverage 51.74% (231025/446488)
Branch Coverage 53.09% (99775/187936)

yiguolei
yiguolei previously approved these changes Apr 15, 2026
@CalvinKirs CalvinKirs marked this pull request as draft April 15, 2026 03:28
@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Apr 15, 2026
@github-actions
Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

Removed 'constraintManager' from the list of modules in PersistMetaModules.java.
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@github-actions github-actions Bot removed the approved Indicates a PR has been approved by one committer. label Apr 15, 2026
@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.41% (758/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 37.36% (414/1108) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs CalvinKirs marked this pull request as ready for review April 20, 2026 02:35
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.47% (1797/2290)
Line Coverage 64.13% (32279/50334)
Region Coverage 65.06% (16234/24952)
Branch Coverage 55.60% (8678/15608)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.41% (758/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 35.66% (413/1158) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 90.59% (77/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.34% (26933/36724)
Line Coverage 56.78% (290026/510806)
Region Coverage 54.28% (242397/446530)
Branch Coverage 55.91% (105083/187956)

@CalvinKirs CalvinKirs closed this Apr 23, 2026
@CalvinKirs CalvinKirs reopened this Apr 23, 2026
@CalvinKirs
Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.05% (1842/2360)
Line Coverage 64.76% (32985/50936)
Region Coverage 65.30% (16370/25070)
Branch Coverage 55.89% (8741/15640)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 18.05% (200/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 54.32% (19984/36787)
Line Coverage 35.27% (180533/511928)
Region Coverage 29.50% (132265/448423)
Branch Coverage 31.06% (58962/189861)

@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.05% (1842/2360)
Line Coverage 64.74% (32977/50936)
Region Coverage 65.23% (16354/25070)
Branch Coverage 55.84% (8733/15640)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.41% (758/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 54.30% (19975/36789)
Line Coverage 35.21% (180273/511984)
Region Coverage 29.45% (132088/448513)
Branch Coverage 30.99% (58843/189875)

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 54.28% (19970/36792)
Line Coverage 35.22% (180332/512055)
Region Coverage 29.45% (132129/448585)
Branch Coverage 30.99% (58855/189915)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 36.91% (409/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 54.69% (20122/36792)
Line Coverage 35.70% (182780/512055)
Region Coverage 29.97% (134446/448585)
Branch Coverage 31.63% (60072/189915)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 37.36% (414/1108) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 4.71% (4/85) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.69% (20491/36792)
Line Coverage 36.72% (188036/512055)
Region Coverage 31.28% (140314/448585)
Branch Coverage 32.86% (62410/189915)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 32.24% (414/1284) 🎉
Increment coverage report
Complete coverage report

### What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary: Bring auth_branch-4.1 up to date with rich/branch-4.1 and resolve the FrontendServiceImplTest merge conflict.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - ./run-fe-ut.sh --run org.apache.doris.service.FrontendServiceImplTest
- Behavior changed: No
- Does this need documentation: No
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

…IDC access token auth (apache#62159)

apache#60361

  Problem Summary:
FE authentication plugin loading currently assumes a single plugin root
in several runtime paths, which makes
external plugin deployment less flexible and inconsistent across
callers.

In addition, OIDC token-based authentication uses inconsistent
credential typing across MySQL authentication and
authentication_chain fallback flows, and the client-visible error
message handling is not aligned with access-token-
  specific failures.

This change makes plugin root parsing reusable across FE runtime paths,
allows configuring multiple authentication
or authorization plugin roots with a comma-separated list, normalizes
OIDC access token requests to use
`OAUTH_TOKEN`, broadens OIDC request detection in fallback
authentication flow, and aligns access-token-related
failure messages. It also marks `authentication_chain` as mutable so the
fallback chain can be adjusted dynamically.
@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.11% (1848/2366)
Line Coverage 64.83% (33214/51232)
Region Coverage 65.30% (16435/25167)
Branch Coverage 55.87% (8777/15710)

…lback config (apache#61362)

This PR adds the FE-side authentication capability required to integrate
`authentication integration` into Doris.

Previously, Doris login was still based on the legacy authenticator
flow. That made `authentication integration` incomplete in
  practice:
- `authentication_type` could not naturally use an authentication plugin
as the primary authenticator
- MySQL login could not execute `authentication_chain` as a standard
integration fallback path
- authentication integration/plugin failures were not classified, so
Doris could not decide clearly when to continue the chain and when
  to stop
- authentication integrations did not have an independent runtime path
for preparation, activation, and execution during login

This PR fills in that missing path, so Doris can use auth integration in
the FE MySQL authentication flow.

  How it works:
1. Doris chooses the primary MySQL authenticator from
`authentication_type`.
     - Existing `default` / `password` / `ldap` behavior is preserved
     - plugin names are now also supported as primary authenticators

2. If `authentication_type` points to a plugin, Doris uses
`AuthenticationPluginAuthenticator`
to adapt the authentication plugin to the legacy MySQL `Authenticator`
interface.

3. If primary authentication succeeds, Doris maps the authenticated
principal to:
     - an existing Doris user, or
- a temporary JIT user when `enable_jit_user` is enabled on the
integration/plugin side.

4. If primary authentication fails and `authentication_chain` is
configured, Doris falls back to
`AuthenticationIntegrationAuthenticator`, which executes the configured
integrations in order.

5. Each integration/plugin failure is tagged with a typed failure
category.
Doris continues the chain only for failures that should be retryable in
chain mode
(for example `USER_NOT_FOUND` or `SOURCE_UNAVAILABLE`), and stops on
terminal failures
     such as bad credentials or access denied.

6. If one integration in the chain succeeds, login succeeds. Otherwise
Doris returns the final authentication failure.
@CalvinKirs
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.11% (1848/2366)
Line Coverage 64.82% (33211/51232)
Region Coverage 65.32% (16439/25167)
Branch Coverage 55.90% (8782/15710)

@hello-stephen
Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.29% (786/1151) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 36.52% (428/1172) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants