SigV4 Auth Support for Catalog Federation - Part 2: Connection Config Persistence

[XJDKC]

Milestones

This is Part 2 of the [Splitting] Initial SigV4 Auth Support for Catalog Federation. Upcoming parts will build on this system:

• SigV4 Auth Support for Catalog Federation - Part 1: Entity Transformation System #1899
• SigV4 Auth Support for Catalog Federation - Part 2: Connection Config Persistence #1900
• SigV4 Auth Support for Catalog Federation - Part 3: Service Identity Info Injection
• SigV4 Auth Support for Catalog Federation - Part 4: Connection Credential Manager

Introduction

This PR introduces DPOs (data persistence objects) that allow Polaris to persist SigV4 authentication parameters and service identity references associated with remote catalog connections.

The core idea is to persist a reference to Polaris's own service credentials (e.g. AWS IAM user) rather than the credentials themselves. This enables secure, pluggable credential resolution from external secret stores (e.g. a vault or secret manager) and supports the ability to assume user-specified roles at runtime via SigV4.

Design Overview

Each ConnectionConfigInfoDpo (used for remote catalog federation) now contains a ServiceIdentityInfoDpo, which in turn holds a ServiceSecretReference. This design allows:

• Polaris to store only references to its service identity (e.g. AWS IAM user)
• The actual credentials to be stored securely in a vault or secret manager
• Runtime resolution of credentials based on these references
• Role assumption using SigV4AuthenticationParametersDpo (supplied by the user)

This separation of identity metadata and authentication parameters provides a secure and flexible foundation for credential management.

Key Components

• SigV4AuthenticationParametersDpo: Holds user-supplied role assumption parameters like:
  • roleArn
  • roleSessionName: optional
  • externalId: optional
• ServiceIdentityInfoDpo: Stores metadata about the Polaris-side service identity, including a secret reference.
• AwsIamServiceIdentityInfoDpo:
• ServiceSecretReference: Specialized version of ServiceIdentityInfoDpo for AWS IAM. It includes:
  • iamArn: Polaris's AWS user or role
  • ServiceSecretReference: (points to credentials in a vault)
• ServiceSecretReference: Points to a credential (e.g., in a vault). This is a logical URN or identifier — Polaris never persists raw secrets.

Flowchart