Motivation
FileIO currently accepts object-store credentials as static properties. A long-lived FileIO cannot observe rotated temporary credentials without being rebuilt.
For example, an external process may atomically update a JSON file containing AK/SK/session token and expiration. The same FileIO should reload it when the cached credential expires. This is needed by the Milvus external-table integration, where explicitly configured delegated authentication must also fail closed.
Proposal
Following iceberg-rust's CustomAwsCredentialLoader, allow object-store builders used by FileIO to accept provider-specific reqsign credential providers.
- The provider returns the native credential and its expiration where supported. reqsign caches it and invokes the provider again when it becomes invalid.
- An explicitly configured provider replaces the default credential chain; refresh errors are returned without falling back to another identity.
- Existing property-based configuration remains unchanged. STS, OIDC, ARN, impersonation, and credential-broker implementations stay outside paimon-rust.
- Support S3 first, then use the same pattern for OSS, GCS, and Azure where OpenDAL / reqsign expose equivalent hooks.
Reloading credentials revoked before their declared expiration requires structured invalidation and retry support from OpenDAL / reqsign. Paimon should not implement this by matching cloud error strings.
Motivation
FileIOcurrently accepts object-store credentials as static properties. A long-livedFileIOcannot observe rotated temporary credentials without being rebuilt.For example, an external process may atomically update a JSON file containing AK/SK/session token and expiration. The same
FileIOshould reload it when the cached credential expires. This is needed by the Milvus external-table integration, where explicitly configured delegated authentication must also fail closed.Proposal
Following iceberg-rust's
CustomAwsCredentialLoader, allow object-store builders used byFileIOto accept provider-specific reqsign credential providers.Reloading credentials revoked before their declared expiration requires structured invalidation and retry support from OpenDAL / reqsign. Paimon should not implement this by matching cloud error strings.