Skip to content

fix(dlf): refresh expiring credentials - #714

Merged
JingsongLi merged 2 commits into
apache:mainfrom
XiaoHongbo-Hope:codex/dlf-token-refresh
Aug 15, 2026
Merged

fix(dlf): refresh expiring credentials#714
JingsongLi merged 2 commits into
apache:mainfrom
XiaoHongbo-Hope:codex/dlf-token-refresh

Conversation

@XiaoHongbo-Hope

Copy link
Copy Markdown
Contributor

What changed

  • Parse the standard ECS Expiration field into milliseconds so DLF catalog credentials are refreshed before expiry.
  • Keep a refresh-capable REST token FileIO attached to loaded tables instead of freezing the first data token.
  • Coalesce concurrent refreshes, rebuild the storage operator with rotated credentials, and preserve the catalog local cache.
  • Document ECS RAM-role authentication and REST data-token configuration.

Static AK/SK credentials remain unchanged and are not rotated.

Validation

  • cargo test -p paimon --lib (2254 passed, 1 ignored)
  • cargo test -p paimon token
  • cargo test -p paimon io::file_io:: (31 passed)
  • cargo clippy -p paimon --all-targets -- -D warnings
  • All non-Python workspace targets compile, including DataFusion, C bindings, REST server, and integration tests.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit b83e0f2 into apache:main Aug 15, 2026
13 checks passed
jerry-024 added a commit to jerry-024/paimon-rust that referenced this pull request Aug 17, 2026
* main:
  perf(vindex): size native batches by active indexes (apache#709)
  fix(data-evolution): require row tracking and row IDs (apache#718)
  feat(go): add table write bindings (apache#658)
  fix(scan): preserve Data Evolution file order in row-id groups (apache#717)
  fix(file_index): align file index format with Java V1 (apache#719)
  fix: configure OpenDAL writer chunk size (apache#713)
  fix(python): release GIL during catalog I/O (apache#716)
  feat(write): add fixed-bucket write primitives for postpone tables (apache#659)
  feat: rust examples for creating and querying Paimon tables (apache#648)
  fix(table): reject row ranges for format tables at read construction (apache#700)
  perf(arrow): prune IN predicates with row-group stats (apache#705)
  feat(io): support in-memory local cache (apache#710)
  fix(dlf): refresh expiring credentials (apache#714)
  fix(datafusion): normalize index_type in global index procedures (apache#715)
  fix(auth): fail closed on query-auth tables outside the read boundary (apache#691)

# Conflicts:
#	crates/paimon/src/table/vector_search_builder.rs
#	crates/paimon/src/vindex/reader.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants