Skip to content

[#10959] feat(idp-basic): Add Basic authentication for built-in IdP#11226

Merged
mchades merged 97 commits into
apache:mainfrom
lasdf1234:implement-basic-authentication-flow
May 28, 2026
Merged

[#10959] feat(idp-basic): Add Basic authentication for built-in IdP#11226
mchades merged 97 commits into
apache:mainfrom
lasdf1234:implement-basic-authentication-flow

Conversation

@lasdf1234
Copy link
Copy Markdown
Collaborator

@lasdf1234 lasdf1234 commented May 26, 2026

What changes were proposed in this pull request?

Add built-in IdP Basic authentication and wire it into the Gravitino server, Java client, and distribution:

  • plugins:idp-basic: BasicAuthenticator validates HTTP Basic credentials against relational IdP user metadata with Argon2id password hashes; ServiceAdminInitializer creates configured service admins on first startup when GRAVITINO_INITIAL_ADMIN_PASSWORD is set.
  • Server bootstrap: Optional plugins load through ServerPluginBootstrap SPI (ServerPluginBootstrapper in server-common, IdpServerPluginBootstrap in idp-basic) so the server module does not compile-depend on idp-basic.
  • Java client: GravitinoClient.builder(...).withBasicAuth(username, password) and BasicTokenProvider.
  • Docs: Basic mode configuration and client usage in docs/security/how-to-authenticate.md.

Why are the changes needed?

Gravitino needs a built-in authentication path for deployments that do not use an external OAuth provider. This PR implements the basic authenticator mode, service-admin bootstrap for initial login, client support, and packaging so the feature is available in official binaries.

Fix: #10965

Does this PR introduce any user-facing change?

Yes.

  1. Server configuration: New authenticator value basic in gravitino.authenticators (requires gravitino.entity.store=relational).
  2. Environment variable: GRAVITINO_INITIAL_ADMIN_PASSWORD — JSON array of username:password entries for first-time service admin creation.
  3. Java client API: GravitinoClientBase.Builder.withBasicAuth(String username, String password).

How was this patch tested?

Unit tests added/updated in plugins:idp-basic, server-common, and clients:client-java. Locally ran:

  • ./gradlew :plugins:idp-basic:test -PskipITs -PskipDockerTests=true
  • ./gradlew :server-common:test --tests "*BasicAuthentication*" -PskipITs -PskipDockerTests=true
  • ./gradlew :clients:client-java:test --tests "*BasicTokenProvider*" -PskipITs

lasdf1234 and others added 19 commits May 25, 2026 14:34
…dP auth

Implement HTTP Basic authentication in the idp-basic plugin, register the
basic authenticator type, and resolve user groups from IdP metadata.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Wire idp-basic on the server-common test classpath and verify BasicAuthenticator
behavior with AuthenticationFilter.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tests

Trim Base64 credentials before decoding, treat blank passwords as invalid
credentials, and add unit tests for malformed Basic auth headers.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tartup

Add ServiceAdminInitializer in idp-basic to provision missing built-in IdP
accounts from GRAVITINO_INITIAL_ADMIN_PASSWORD when basic auth is enabled.
Wire startup through IdpStorageBootstrap and reflectively load the plugin from
server without a compile-time dependency on idp-basic.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy the plugin and bcprov jars into the Iceberg REST server package and
use implementation dependencies for api and server-common in idp-basic.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document Basic mode alongside simple, OAuth, and Kerberos, including
server setup, initial admin password, curl usage, and a full example.

Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce BasicTokenProvider and GravitinoClient builder support, plus
document Basic mode client usage in how-to-authenticate.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sync with apache/gravitino main via origin/main. Resolve IdpStorageBootstrap
conflict by keeping service-admin initialization only; relational storage and
garbage collection are handled by IdpRelationalStorage on main.

Co-authored-by: Cursor <cursoragent@cursor.com>
…to initializer

Remove ServiceAdminManager and wire ServiceAdminInitializer directly to
IdpUserMetaService, aligning user PO fields with IdpUserGroupManager.

Co-authored-by: Cursor <cursoragent@cursor.com>
…inBootstrap SPI

Replace reflective BuiltInIdpPluginLauncher with ServiceLoader-based
ServerPluginBootstrapper and register idp-basic through META-INF/services.

Co-authored-by: Cursor <cursoragent@cursor.com>
…e admin init

Merge IdpStorageBootstrap into IdpServerPluginBootstrap, make
ServiceAdminInitializer a static utility, and consolidate unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
…AdminInitializer

Remove the private newUserPO helper and build the user PO at the insert call site.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove duplicate curl example and info admonition from Basic mode section;
Simple mode already documents Authorization header usage.

Co-authored-by: Cursor <cursoragent@cursor.com>
Copilot AI review requested due to automatic review settings May 26, 2026 08:00
@lasdf1234 lasdf1234 changed the title [#10959][#10963][#10964][#10965] feat(idp-basic): Basic authentication and service admin bootstrap [#10959] feat(idp-basic): Add Basic authentication for built-in IdP May 26, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds built-in HTTP Basic authentication backed by the IdP relational store, including server-side plugin bootstrapping and Java client support.

Changes:

  • Introduces a ServiceLoader-based server plugin bootstrap mechanism and wires it into server startup.
  • Adds IdP Basic authentication (authenticator + service-admin initialization) plus tests and distribution packaging updates.
  • Updates Java client builder/token provider and documentation to support/configure Basic auth.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
server/src/main/java/org/apache/gravitino/server/GravitinoServer.java Calls plugin bootstrapper during server initialization.
server-common/src/main/java/org/apache/gravitino/server/plugin/ServerPluginBootstrap.java Defines the ServiceLoader-discovered plugin bootstrap SPI.
server-common/src/main/java/org/apache/gravitino/server/plugin/ServerPluginBootstrapper.java Loads and initializes plugin bootstraps from the classpath.
server-common/src/main/java/org/apache/gravitino/server/authentication/AuthenticatorFactory.java Registers BASIC authenticator type mapping.
common/src/main/java/org/apache/gravitino/auth/AuthenticatorType.java Adds BASIC to supported authenticator types.
plugins/idp-basic/src/main/java/org/apache/gravitino/idp/auth/BasicAuthenticator.java Implements HTTP Basic auth against IdP user metadata.
plugins/idp-basic/src/main/java/org/apache/gravitino/idp/auth/ServiceAdminInitializer.java Initializes configured service-admin users on startup via env-provided passwords.
plugins/idp-basic/src/main/java/org/apache/gravitino/idp/IdpServerPluginBootstrap.java Implements server plugin bootstrap to run IdP initialization once per JVM.
plugins/idp-basic/src/main/java/org/apache/gravitino/idp/storage/service/IdpUserMetaService.java Adds existence check API used during initialization.
plugins/idp-basic/src/main/resources/META-INF/services/org.apache.gravitino.server.plugin.ServerPluginBootstrap Registers IdP bootstrap provider for ServiceLoader.
clients/client-java/src/main/java/org/apache/gravitino/client/GravitinoClientBase.java Adds withBasicAuth builder method.
clients/client-java/src/main/java/org/apache/gravitino/client/BasicTokenProvider.java Adds Basic auth header token provider.
docs/security/how-to-authenticate.md Documents Basic auth configuration and examples.
docs/index.md Updates authentication documentation summary to include Basic.
server-common/build.gradle.kts Adds idp-basic as test dependency for server-common tests.
plugins/idp-basic/build.gradle.kts Adds dependencies/resources/copy tasks needed for plugin + standalone packaging.
server-common/src/test/java/org/apache/gravitino/server/authentication/TestBasicAuthentication.java Adds server-side filter tests for Basic auth flow.
plugins/idp-basic/src/test/java/org/apache/gravitino/idp/auth/TestBasicAuthenticator.java Adds unit tests for BasicAuthenticator parsing/authn behavior.
plugins/idp-basic/src/test/java/org/apache/gravitino/idp/auth/TestServiceAdminInitializer.java Adds unit tests for service-admin initialization behavior and validation.
plugins/idp-basic/src/test/java/org/apache/gravitino/idp/TestIdpServerPluginBootstrap.java Verifies ServiceLoader registration for the IdP bootstrap.
clients/client-java/src/test/java/org/apache/gravitino/client/TestBasicTokenProvider.java Tests Basic token construction/encoding.

…avoid IdpUserMetaService changes

Rename ServerPluginBootstrap.initializeOnce to initialize, and check
service admin existence via getIdpUserByUsername instead of adding
idpUserExists to IdpUserMetaService.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234
Copy link
Copy Markdown
Collaborator Author

@roryqi @mchades @yuqi1129 Please review the code for me, thank you.

@mchades mchades requested review from roryqi and yuqi1129 May 26, 2026 08:18
Comment thread docs/security/how-to-authenticate.md Outdated
Comment thread docs/security/how-to-authenticate.md Outdated
Copy link
Copy Markdown
Contributor

@mchades mchades left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing update Python client?

…ginBootstrap SPI

Fix RAT check failure on the META-INF/services registration file.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread plugins/idp-basic/build.gradle.kts Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 27, 2026

Code Coverage Report

Overall Project 66.55% +0.1% 🟢
Files changed 81.41% 🟢

Module Coverage
aliyun 1.72% 🔴
api 46.82% 🟢
authorization-common 85.96% 🟢
aws 3.66% 🔴
azure 2.47% 🔴
catalog-common 10.04% 🔴
catalog-fileset 80.33% 🟢
catalog-glue 66.08% 🟢
catalog-hive 79.55% 🟢
catalog-jdbc-clickhouse 80.02% 🟢
catalog-jdbc-common 45.31% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.29% 🟢
catalog-jdbc-starrocks 78.51% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 44.89% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.65% 🟢
catalog-lakehouse-paimon 79.29% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.91% +0.11% 🟢
common 49.99% 🟢
core 82.37% 🟢
filesystem-hadoop3 76.97% 🟢
flink 0.0% 🔴
flink-common 41.37% 🟢
flink-runtime 0.0% 🔴
gcp 14.12% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 53.26% 🟢
iceberg-common 54.98% 🟢
iceberg-rest-server 70.08% 🟢
idp-basic 85.99% -4.6% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 20.83% 🔴
lance-rest-server 60.27% 🟢
lineage 53.02% 🟢
optimizer 82.95% 🟢
optimizer-api 21.95% 🔴
server 85.1% 🟢
server-common 72.85% 🟢
spark 32.79% 🔴
spark-common 39.61% 🔴
trino-connector 39.44% 🔴
Files
Module File Coverage
client-java BasicTokenProvider.java 100.0% 🟢
GravitinoClientBase.java 79.78% 🟢
idp-basic IdpGroupMetaBaseSQLProvider.java 100.0% 🟢
IdpUserMetaBaseSQLProvider.java 100.0% 🟢
IdpGroupMetaH2Provider.java 100.0% 🟢
IdpUserMetaH2Provider.java 100.0% 🟢
IdpGroupMetaPostgreSQLProvider.java 100.0% 🟢
IdpUserMetaPostgreSQLProvider.java 100.0% 🟢
IdpUserMetaService.java 96.77% 🟢
IdpGroupMetaService.java 95.45% 🟢
IdpUserMetaSQLProviderFactory.java 92.86% 🟢
IdpGroupMetaSQLProviderFactory.java 91.67% 🟢
BasicAuthenticator.java 89.8% 🟢
IdpUserGroupManager.java 88.89% 🟢
IdpRESTUtils.java 80.0% 🟢
IdpSQLExceptionConverter.java 73.68% 🟢
IdpPOConverters.java 69.23% 🟢
IdpUser.java 61.54% 🟢
IdpUserWithGroupsPO.java 42.86% 🔴
IdpGroupWithUsersPO.java 33.33% 🔴
IdpGroupMetaMapper.java 0.0% 🔴
IdpUserMetaMapper.java 0.0% 🔴
IdpBasicBinder.java 0.0% 🔴
IdpRESTFeature.java 0.0% 🔴

Comment thread .github/workflows/idp-basic-test.yml Outdated
./gradlew :plugins:idp-basic:test -PskipITs -PskipDockerTests=false -PskipWeb=true
for backend in h2 mysql postgresql; do
./gradlew :plugins:idp-basic:test \
-PtestMode=embedded \
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you also need to run the deploy testMode

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got deploy mode has been added.

Comment thread .github/workflows/idp-basic-test.yml Outdated
-PjdbcBackend="${backend}" \
-PskipDockerTests=false \
-PskipWeb=true \
--tests "org.apache.gravitino.idp.integration.test.IdpRESTApiIT"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use "org.apache.gravitino.idp.integration.test.**"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got code has been modified.

metalake_name="metalake",
auth_data_provider=BasicAuthProvider("admin", "YourSecureGravitinoPassword"),
)
```
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you plz add the shell example?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

God shell example has been added.

}

private String requireBasicAuthHeader(byte[] tokenData) {
if (tokenData == null) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using StringUtils.isBlank here, and the check below for 'authData ' can be simplified?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got the logical has been simplified

throw new UnauthorizedException("Invalid username or password", BASIC_CHALLENGE);
}
return new BasicCredentials(username, password);
} catch (IllegalArgumentException e) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catch from which method?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got cahtch base64 method, code has been modified.

Comment on lines +36 to +38
AuthConstants.AUTHORIZATION_BASIC_HEADER
+ base64.b64encode(user_information.encode("utf-8")).decode("utf-8")
).encode("utf-8")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please improve the code's readability?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got.Revised the method.

SIMPLE_AUTH_TYPE = "simple"
BASIC_AUTH_TYPE = "basic"
BASIC_USERNAME = "basic_username"
BASIC_PASSWORD = "basic_password"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you also update the Python client config user doc for these config items?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got added.

* @param groupNames The group names the user belongs to.
*/
public IdpUser(String name, List<String> groupNames) {
public IdpUser(String name, String passwordHash, List<String> groupNames) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you check the parameters?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I split the constructor of IdpUser into two parts, one including passwordHash and the other not. The constructor that includes the "passwordHash" parameter will check that "passwordHash" cannot be empty.

Comment on lines +33 to +41
return "SELECT g.group_name as name,"
+ " '['"
+ " || COALESCE(GROUP_CONCAT("
+ " CASE"
+ " WHEN u.user_name IS NOT NULL AND u.user_name <> ''"
+ " THEN '\"' || u.user_name || '\"'"
+ " ELSE NULL"
+ " END), '')"
+ " || ']' as usernames"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this logic be handled within the Mapper?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User and group mapper such logical patterns have been deleted.

return currentProvider().selectIdpGroup(groupName);
}

public static String selectIdpGroupWithUsers(@Param("groupName") String groupName) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should return List<String>

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This factory class returns SQL, it should be String.

…ss-control

Add compileDistribution, deploy tests for MySQL/PostgreSQL, and DRY the
embedded/deploy matrix while matching access-control integration test patterns.

Co-authored-by: Cursor <cursoragent@cursor.com>
* @return The built-in IdP user.
*/
public static IdpUser fromIdpUserWithGroupsPO(IdpUserWithGroupsPO userPO) {
Objects.requireNonNull(userPO, "userPO must not be null");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use Preconditions.checkArgument to avoid throwing NPE

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the methods in the class have been checked and modified.

lasdf1234 and others added 9 commits May 28, 2026 11:05
Document HTTP Basic usage with username and password, aligned with the
existing Simple mode shell example.

Co-authored-by: Cursor <cursoragent@cursor.com>
…thenticator

Use StringUtils.isBlank on authData after normalizing null tokenData to an
empty string, reducing duplicate empty-header checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ator

Catch IllegalArgumentException only around Base64.getDecoder().decode so
credential parsing errors are not conflated with invalid base64 input.

Co-authored-by: Cursor <cursoragent@cursor.com>
… readability

Extract _build_basic_auth_token to separate credential encoding from header
assembly, matching the structure of SimpleAuthProvider.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add basic_username and basic_password to the GVFS config table and include
a Python usage example for built-in IDP Basic auth.

Co-authored-by: Cursor <cursoragent@cursor.com>
…word hash

Add a two-argument constructor for users without a loaded hash, validate
constructor parameters, and update converters and call sites accordingly.

Co-authored-by: Cursor <cursoragent@cursor.com>
…uctors

Remove redundant null or empty checks in H2 and PostgreSQL join queries,
and initialize passwordHash explicitly in the two-argument IdpUser constructor.

Co-authored-by: Cursor <cursoragent@cursor.com>
Replace Objects.requireNonNull with Guava Preconditions.checkNotNull for
consistency with other idp-basic modules.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234 lasdf1234 requested a review from mchades May 28, 2026 06:30
lasdf1234 and others added 3 commits May 28, 2026 14:57
Set GRAVITINO_INITIAL_ADMIN_PASSWORD in the workflow and append it to
gravitino-env.sh during deploy-mode IdpRESTApiIT so the packaged server
can initialize configured service admins. Upload package-all server logs
on failure for easier debugging.

Co-authored-by: Cursor <cursoragent@cursor.com>
Deploy mode starts Gravitino via gravitino.sh without main() args; inheriting
GRAVITINO_TEST from the module test env caused ArrayIndexOutOfBoundsException
in GravitinoServer.main. Embedded IT still sets GRAVITINO_TEST via root build.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread .github/workflows/idp-basic-test.yml Outdated
if: needs.changes.outputs.source_changes == 'true' && needs.changes.outputs.module_exists == 'true'
runs-on: ubuntu-22.04
timeout-minutes: 60
timeout-minutes: 90
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

30 min is enough

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got resolved.

Comment thread .github/workflows/idp-basic-test.yml Outdated
dev/ci/util_free_space.sh

- name: Run idp-basic tests
- name: Run idp-basic unit tests
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unnecessary to seperate tests out

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got resolved.

Merge unit and REST IT into one step, use nested loops for embedded/deploy
across h2/mysql/postgresql, and drop redundant -PskipWeb on test tasks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234 lasdf1234 closed this May 28, 2026
@lasdf1234 lasdf1234 reopened this May 28, 2026
@mchades mchades merged commit 3c60292 into apache:main May 28, 2026
33 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask] Add Basic authentication for built-in IdP

4 participants