Skip to content

Updating metastore documentation with Aurora postgres example #2706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 2, 2025
Merged

Updating metastore documentation with Aurora postgres example #2706

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6b9159f
added Aurora postgres to metastore documentation
fabio-rizzo-01 Sep 29, 2025
a853258
Merge remote-tracking branch 'private/feature/iam-postgres-support-2'…
fabio-rizzo-01 Sep 29, 2025
a27570f
added Aurora postgres to metastore documentation
fabio-rizzo-01 Sep 29, 2025
49c9358
Merge branch 'apache:main' into feature/iam-postgres-support-2
fabio-rizzo-01 Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions site/content/in-dev/unreleased/metastores.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,45 @@ deprecated EclipseLink persistence backends.
This implementation leverages Quarkus for datasource management and supports configuration through
environment variables or JVM -D flags at startup. For more information, refer to the [Quarkus configuration reference](https://quarkus.io/guides/config-reference#env-file).

We have 2 options for configuring the persistence backend:

### 1. Relational JDBC metastore with username and password

using environment variables:
```
POLARIS_PERSISTENCE_TYPE=relational-jdbc

QUARKUS_DATASOURCE_USERNAME=<your-username>
QUARKUS_DATASOURCE_PASSWORD=<your-password>
QUARKUS_DATASOURCE_JDBC_URL=<jdbc-url-of-postgres>
```
using properties file:

```
polaris.persistence.type=relational-jdbc
quarkus.datasource.jdbc.username=<your-username>
quarkus.datasource.jdbc.password=<your-password>
quarkus.datasource.jdbc.jdbc-url=<jdbc-url-of-postgres>
```
Comment on lines +44 to +51
Copy link
Contributor

@singhpk234 singhpk234 Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check this feedback from @dimas-b
https://github.com/apache/polaris/pull/1515/files#r2074243975

IMHO we should be consistent, as far as application.propeties is concerned we reference the quarkus conf page above so that audience can choose what ever way they prefer.

Copy link
Contributor

@dimas-b dimas-b Sep 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to avoid options like env. vs. file in main docs. All of that is essentially a config entry. There are multiple sources for the config (env., system properties, properties file) and there are differences with how you can specify config names.

Also, lower case names with dots do work in the environment (e.g. in docker/k8s) but do not work in bash (and probably other shells).

I think we may need to overhaul the docs WRT user-facing config, but let's keep the existing patterns for this PR (upper case env. variables).

Copy link
Contributor

@dimas-b dimas-b Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the RDS config may not be easy to convert to upper case. So let's use lower case (with dots and dashes) in all cases.

Users have references to Quarkus docs and https://polaris.apache.org/in-dev/unreleased/configuration/

I hope that should be enough input for them to figure out specifics in their deployment env.

Copy link
Contributor Author

@fabio-rizzo-01 fabio-rizzo-01 Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so no changes needed from my side right?

Copy link
Contributor

@dimas-b dimas-b Oct 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that's right 😅 I think the PR is good to merge :)


### 2. AWS Aurora PostgreSQL metastore using IAM AWS authentication

```
polaris.persistence.type=relational-jdbc
quarkus.datasource.jdbc.url=jdbc:postgresql://polaris-cluster.cluster-xyz.us-east-1.rds.amazonaws.com:6160/polaris
quarkus.datasource.jdbc.additional-jdbc-properties.wrapperPlugins=iam
quarkus.datasource.username=dbusername
quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc.additional-jdbc-properties.ssl=true
quarkus.datasource.jdbc.additional-jdbc-properties.sslmode=require
quarkus.datasource.credentials-provider=aws

quarkus.rds.credentials-provider.aws.use-quarkus-client=true
quarkus.rds.credentials-provider.aws.username=dbusername
quarkus.rds.credentials-provider.aws.hostname=polaris-cluster.cluster-xyz.us-east-1.rds.amazonaws.com
quarkus.rds.credentials-provider.aws.port=6160
```
This is the basic configuration. For more details, please refer to the [Quarkus plugin documentation](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-rds.html#_configuration_reference)

The Relational JDBC metastore currently relies on a Quarkus-managed datasource and supports only PostgresSQL and H2 databases. This limitation is similar to that of EclipseLink, primarily due to underlying schema differences. At this time, official documentation is provided exclusively for usage with PostgreSQL.
Please refer to the documentation here:
Expand Down