Conversation
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Related to apache#2213 ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
) ## Which issue does this PR close? Split off from apache#1851 - Partially fixes apache#1731. ## What changes are included in this PR? This change honors the compression setting for metadata.json file (`write.metadata.compression-codec`). ## Are these changes tested? Add unit test to verify files are gzipped when the flag is enabled. BREAKING CHANGE: Make `write_to` take `MetadataLocation` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io>
…thon (apache#2228) Bumps [quinn-proto](https://github.com/quinn-rs/quinn) from 0.11.13 to 0.11.14. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/quinn-rs/quinn/releases">quinn-proto's releases</a>.</em></p> <blockquote> <h2>quinn-proto 0.11.14</h2> <p><a href="https://github.com/jxs"><code>@jxs</code></a> reported a denial of service issue in quinn-proto 5 days ago:</p> <ul> <li><a href="https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98">https://github.com/quinn-rs/quinn/security/advisories/GHSA-6xvm-j4wr-6v98</a></li> </ul> <p>We coordinated with them to release this version to patch the issue. Unfortunately the maintainers missed these issues during code review and we did not have enough fuzzing coverage -- we regret the oversight and have added an additional fuzzing target.</p> <p>Organizations that want to participate in coordinated disclosure can contact us privately to discuss terms.</p> <h2>What's Changed</h2> <ul> <li>Fix over-permissive proto dependency edge by <a href="https://github.com/Ralith"><code>@Ralith</code></a> in <a href="https://redirect.github.com/quinn-rs/quinn/pull/2385">quinn-rs/quinn#2385</a></li> <li>0.11.x: avoid unwrapping VarInt decoding during parameter parsing by <a href="https://github.com/djc"><code>@djc</code></a> in <a href="https://redirect.github.com/quinn-rs/quinn/pull/2559">quinn-rs/quinn#2559</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/quinn-rs/quinn/commit/2c315aa7f9c2a6c1db87f8f51f40623a427c78fd"><code>2c315aa</code></a> proto: bump version to 0.11.14</li> <li><a href="https://github.com/quinn-rs/quinn/commit/8ad47f431e7deb82c08b09c2e33ef85aa88fd212"><code>8ad47f4</code></a> Use newer rustls-pki-types PEM parser API</li> <li><a href="https://github.com/quinn-rs/quinn/commit/c81c0289abe30d8437ccbf9b6304e2bc9c707cea"><code>c81c028</code></a> ci: fix workflow syntax</li> <li><a href="https://github.com/quinn-rs/quinn/commit/0050172969f7e69e136c433181330da7790d8d73"><code>0050172</code></a> ci: pin wasm-bindgen-cli version</li> <li><a href="https://github.com/quinn-rs/quinn/commit/8a6f82c58d1c565eab78f986e614223e6ed76a85"><code>8a6f82c</code></a> Take semver-compatible dependency updates</li> <li><a href="https://github.com/quinn-rs/quinn/commit/e52db4ad8df0f9720e7b0e32ecc0e48c9a93de0f"><code>e52db4a</code></a> Apply suggestions from clippy 1.91</li> <li><a href="https://github.com/quinn-rs/quinn/commit/6df7275c582ca9b7225e0ccf9f9871a55eb73155"><code>6df7275</code></a> chore: Fix <code>unnecessary_unwrap</code> clippy</li> <li><a href="https://github.com/quinn-rs/quinn/commit/c8eefa07e087b06d8f2b78ff262ce8ac952994f1"><code>c8eefa0</code></a> proto: avoid unwrapping varint decoding during parameters parsing</li> <li><a href="https://github.com/quinn-rs/quinn/commit/9723a977754c8662001b0fef97aab8f3ddf1df92"><code>9723a97</code></a> fuzz: add fuzzing target for parsing transport parameters</li> <li><a href="https://github.com/quinn-rs/quinn/commit/eaf0ef30252cef4acec21f150427e604cd4271c9"><code>eaf0ef3</code></a> Fix over-permissive proto dependency edge (<a href="https://redirect.github.com/quinn-rs/quinn/issues/2385">#2385</a>)</li> <li>Additional commits viewable in <a href="https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.13...quinn-proto-0.11.14">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Which issue does this PR close? - Closes apache#2086 . ## What changes are included in this PR? In this pr we introduced catalog test suite in catalog-loader, which could unify the behavior of catalogs. ## Are these changes tested? Yes. --------- Co-authored-by: Ray Liu <liurenjie2008@gmail.com>
Bumps [datafusion](https://github.com/apache/datafusion) from 52.2.0 to 52.3.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/datafusion/commit/28d012a41a3017b5f682ef6b01468a7ff9a48fb7"><code>28d012a</code></a> [branch-52] Bump to 52.3.0 and changelog (<a href="https://redirect.github.com/apache/datafusion/issues/20790">#20790</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/1bd7082b798d0d55c1e90c7be1d7e3dba057c288"><code>1bd7082</code></a> [branch-52] Fix repartition from dropping data when spilling (<a href="https://redirect.github.com/apache/datafusion/issues/20672">#20672</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20777">#20777</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/9797095e152749721bec07c0944fe664acaa0849"><code>9797095</code></a> [branch-52] perf: sort replace free()->try_grow() pattern with try_resize() t...</li> <li><a href="https://github.com/apache/datafusion/commit/afc1c72a15bdd31e15a7e354e86a505be7882f08"><code>afc1c72</code></a> [branch-52] FFI_TableOptions are using default values only (<a href="https://redirect.github.com/apache/datafusion/issues/20705">#20705</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/d317d00b886bbf11cb489e4c4bdc2280b3ca9e07"><code>d317d00</code></a> [branch-52] fix: <code>HashJoin</code> panic with String dictionary keys (don't flatten ...</li> <li><a href="https://github.com/apache/datafusion/commit/72ea8ec086e59220f6b255ea565e710990ad7967"><code>72ea8ec</code></a> [branch-52] Fix constant value from stats (<a href="https://redirect.github.com/apache/datafusion/issues/20042">#20042</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20709">#20709</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/9a67de58c027e6057aa37327ae4d0192d5c45fc5"><code>9a67de5</code></a> [branch-52] Fix Arrow Spill Underrun (<a href="https://redirect.github.com/apache/datafusion/issues/20159">#20159</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/20684">#20684</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/19a0fcaa276c86beda544c6e01c75f6e0639767e"><code>19a0fca</code></a> [branch-52] SortMergeJoin don't wait for all input before emitting (<a href="https://redirect.github.com/apache/datafusion/issues/20699">#20699</a>)</li> <li>See full diff in <a href="https://github.com/apache/datafusion/compare/52.2.0...52.3.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 3.17.0 to 3.18.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/jonasbb/serde_with/releases">serde_with's releases</a>.</em></p> <blockquote> <h2>serde_with v3.18.0</h2> <h3>Added</h3> <ul> <li>Support <code>OneOrMany</code> with more sequence and set types (<a href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li> </ul> <h3>Changed</h3> <ul> <li>Bump MSRV to 1.88 due to the <code>darling</code> dependency</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/jonasbb/serde_with/commit/d50ec962c6ecad7d8972f95d7ee7cea398b7eb41"><code>d50ec96</code></a> Bump version to 3.18.0 (<a href="https://redirect.github.com/jonasbb/serde_with/issues/931">#931</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/984fe3252ecd47526f452e39736f70f96b503f7c"><code>984fe32</code></a> Bump version to 3.18.0</li> <li><a href="https://github.com/jonasbb/serde_with/commit/4ba41c70c7f12b2e543ae81480a50b4d76245419"><code>4ba41c7</code></a> Bump actions/upload-artifact from 6 to 7 in the github-actions group (<a href="https://redirect.github.com/jonasbb/serde_with/issues/927">#927</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/8fb2468ce24e822fc29cd9aa8ebb3feb3ddf1eb3"><code>8fb2468</code></a> Bump actions/upload-artifact from 6 to 7 in the github-actions group</li> <li><a href="https://github.com/jonasbb/serde_with/commit/aec0a23c15943bc4ca82d329695fabefb2b19174"><code>aec0a23</code></a> Bump MSRV to 1.88 (<a href="https://redirect.github.com/jonasbb/serde_with/issues/930">#930</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/25c15a2c5c53f8fa71af91d699877147568338b8"><code>25c15a2</code></a> Update time dependency to 0.3.47</li> <li><a href="https://github.com/jonasbb/serde_with/commit/93bd3f4bebec516e5608a12f09ad1859cdced9a7"><code>93bd3f4</code></a> Update test output after darling update</li> <li><a href="https://github.com/jonasbb/serde_with/commit/f825dbffb12dd758c80247259a271956f1c484b4"><code>f825dbf</code></a> Upgrade darling to 0.23.0</li> <li><a href="https://github.com/jonasbb/serde_with/commit/65cbd738f090f25d89ec4b350501b4aa5b38bd9e"><code>65cbd73</code></a> Bump MSRV to 1.88</li> <li><a href="https://github.com/jonasbb/serde_with/commit/daff02ea264c3136131bfcff079304714f359bd9"><code>daff02e</code></a> Extend OneOrMany implementation to more collection types (<a href="https://redirect.github.com/jonasbb/serde_with/issues/929">#929</a>)</li> <li>Additional commits viewable in <a href="https://github.com/jonasbb/serde_with/compare/v3.17.0...v3.18.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to 0.12.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's changelog</a>.</em></p> <blockquote> <h1>0.12.1 (2026-03-14)</h1> <h3>Security Fix</h3> <ul> <li>Fix handling of invalid match offsets during decompression <a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a> (thanks <a href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li> </ul> <pre><code>Invalid match offsets (offset == 0) during decompression were not properly handled, which could lead to invalid memory reads on untrusted input. Users on 0.12.x should upgrade to 0.12.1. </code></pre> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a> bump version to 0.12.1</li> <li><a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a> fix handling of invalid match offsets during decompression</li> <li>See full diff in <a href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…pache#2238) Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.12.0 to 0.12.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md">lz4_flex's changelog</a>.</em></p> <blockquote> <h1>0.12.1 (2026-03-14)</h1> <h3>Security Fix</h3> <ul> <li>Fix handling of invalid match offsets during decompression <a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154">#a0b9154</a> (thanks <a href="https://github.com/Marcono1234"><code>@Marcono1234</code></a>)</li> </ul> <pre><code>Invalid match offsets (offset == 0) during decompression were not properly handled, which could lead to invalid memory reads on untrusted input. Users on 0.12.x should upgrade to 0.12.1. </code></pre> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PSeitz/lz4_flex/commit/fa48c987a88df5059a49fe7519c028d6f2b8caf4"><code>fa48c98</code></a> bump version to 0.12.1</li> <li><a href="https://github.com/PSeitz/lz4_flex/commit/a0b9154becbe22da3ce91211d7b6619c289723cf"><code>a0b9154</code></a> fix handling of invalid match offsets during decompression</li> <li>See full diff in <a href="https://github.com/pseitz/lz4_flex/compare/0.12.0...0.12.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com>
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2065 ## What changes are included in this PR? - Add `delete_stream` to `Storage` trait to support batch delete - Expose `delete_stream` in `FileIO` as well <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Added uts Addded integtests for opendal <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2210 ## What changes are included in this PR? - Add OpenDalResolvingStorage <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Added a new test <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - publish has to be done one by one, otherwise we may see failure like this: https://github.com/apache/iceberg-rust/actions/runs/23260056698 ## What changes are included in this PR? - Change publish parallism back to 1 <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
Removed GitHub Actions dependency update configuration. ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Related to apache/iceberg-python#3186 Dont auto update since we now depend on github action being allowlisted by asf-infra first, https://github.com/apache/infrastructure-actions/blob/main/approved_patterns.yml <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Pin `astral-sh/setup-uv` to commit SHAs from Apache's [infrastructure-actions allowlist](https://github.com/apache/infrastructure-actions/blob/07f5f9d2b05fe0ec9886e3ef0a9d79797817f0cb/approved_patterns.yml#L9) Fixes apache/infrastructure-actions#550 <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
…atures (apache#2274) ## Which issue does this PR close? - Fix the audit check by updating `aws-lc-sys` and `rustls-webpki`. - Avoid pulling both the legacy `rustls` / Hyper 0.14 stack and the newer `default-https-client` stack through inherited AWS SDK defaults. ([AWS SDK announcement](awslabs/aws-sdk-rust#1257)) ## What changes are included in this PR? - Bump to `aws-lc-sys>=0.39.0` and `rustls-webpki>=0.103.10` to pass security audit. - Disable inherited AWS SDK default features for `aws-sdk-glue` and `aws-sdk-s3tables` - Explicitly enable `default-https-client` and `rt-tokio` - Bump the minimum `aws-sdk-glue` version to `1.85`, the first version that provides `default-https-client` ## Are these changes tested? --------- Co-authored-by: blackmwk <liurenjie1024@outlook.com>
Bumps [minijinja](https://github.com/mitsuhiko/minijinja) from 2.17.1 to 2.18.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/mitsuhiko/minijinja/blob/main/CHANGELOG.md">minijinja's changelog</a>.</em></p> <blockquote> <h2>2.18.0</h2> <ul> <li>Added keyword argument support (<code>width</code>, <code>first</code>, <code>blank</code>) to the <code>indent</code> filter for Jinja2 compatibility in Rust and Go. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/864">#864</a></li> <li>Added support for dotted integer lookup (for example <code>foo.0</code>) in Rust and Go for Jinja compatibility. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/881">#881</a></li> <li>Added support for dotted filter and test names (including <code>foo . bar . baz</code>) for Jinja compatibility. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/879">#879</a></li> <li>Fixed string escape handling to preserve unknown escapes (such as <code>\s</code>) for Jinja compatibility in Rust and Go. <a href="https://redirect.github.com/mitsuhiko/minijinja/issues/880">#880</a></li> <li>Improved generic performance across template parsing, compilation, and rendering.</li> <li>Fixed <code>minijinja-cabi</code> ownership and pointer-safety issues that could leak <code>mj_value</code> values on error paths.</li> <li>Added high-priority <code>minijinja-cabi</code> APIs for callback-based functions/filters/tests, globals, loaders, path joining, auto-escape configuration, and fuel limits.</li> <li>Switched <code>minijinja-cabi</code> header maintenance to manual source-based syncing and removed cbindgen-based generation tooling.</li> <li>Added lightweight C smoke tests for <code>minijinja-cabi</code> (via <code>make -C minijinja-cabi test</code>) with coverage across all exported C ABI functions, and wired them into top-level testing and CI.</li> <li>Added <code>render_captured</code> and <code>render_captured_to</code> methods on <code>Template</code> which return a <code>Captured</code> type holding the rendered output and the template state.</li> <li>Added <code>into_output</code> method on <code>Captured</code> to consume and return the output string.</li> <li>Deprecated <code>render_and_return_state</code>, <code>eval_to_state</code>, and <code>render_to_write</code> in favor of the new <code>render_captured</code> / <code>render_captured_to</code> / <code>Captured</code> API.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/mitsuhiko/minijinja/commit/92f114d1fd62525b2b4dc1adb77ae1e83c1214a9"><code>92f114d</code></a> release 2.18.0</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/80d30a7526a0119981a1664fab8036b7e64c0d14"><code>80d30a7</code></a> refactor(vendor): prune unused self_cell API surface</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/50ce37a18ad368f22b4c40ff2b3355895ff58556"><code>50ce37a</code></a> fix: typos</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/24891e10c207846fa264c0f8eca930045bbb5fca"><code>24891e1</code></a> feat(filters): add kwargs support to indent filter for Jinja2 parity</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/4cca670f8a346832771d2a567f778b5dc4058156"><code>4cca670</code></a> refactor: deprecate render_to_write in favor of render_captured_to</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/ac88f8e619e0b7d5a4e23819ed5d2ebc046029c6"><code>ac88f8e</code></a> fix: correct typo render_capturedd_to -> render_captured_to</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/710137b2626cfae81b1eb935ea4c9df2435c053d"><code>710137b</code></a> chore: remove dead_code allow and unused MutBorrow from vendored self_cell</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/39d00e61a9f7246b7015dcf655f11159cde1d8cd"><code>39d00e6</code></a> feat: Added new capture methods for state</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/42b0d089333363b8bd667ec99ab67ff7977ef6d4"><code>42b0d08</code></a> feat: vendor self_cell and make loader default</li> <li><a href="https://github.com/mitsuhiko/minijinja/commit/cc12ae0812b8d85dd5963cfa373971fb0b1ff6da"><code>cc12ae0</code></a> fix: make cabi compatible with older rustc</li> <li>Additional commits viewable in <a href="https://github.com/mitsuhiko/minijinja/compare/minijinja-go/v2.17.1...minijinja-go/v2.18.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com>
Bumps [datafusion](https://github.com/apache/datafusion) from 52.3.0 to 52.4.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/datafusion/commit/e5bad58716cf74612ff3b245010411425063c3ec"><code>e5bad58</code></a> [branch-52] Update version to 52.4.0 and update changelog (<a href="https://redirect.github.com/apache/datafusion/issues/21004">#21004</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/e034c6b0b103c674c4576644007b30480565bec3"><code>e034c6b</code></a> [branch-52] Update to use lz4_flex 0.12.1 and quinn-proto 0.11.14 (<a href="https://redirect.github.com/apache/datafusion/issues/21009">#21009</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/664099b60640097a982e63174a96d8828fe1dc0d"><code>664099b</code></a> [branch-52] fix: InList Dictionary filter pushdown type mismatch (<a href="https://redirect.github.com/apache/datafusion/issues/20962">#20962</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/2">#2</a>...</li> <li><a href="https://github.com/apache/datafusion/commit/74aaa65001afd7bc649f471bcf634d52744c46fd"><code>74aaa65</code></a> [branch-52] chore: Ignore RUSTSEC-2024-0014 (<a href="https://redirect.github.com/apache/datafusion/issues/20862">#20862</a>) (<a href="https://redirect.github.com/apache/datafusion/issues/21020">#21020</a>)</li> <li><a href="https://github.com/apache/datafusion/commit/5881edec5d937036891bbec9e7cb01837d9155a5"><code>5881ede</code></a> [branch-52] fix: SanityCheckPlan error with window functions and NVL filter (...</li> <li><a href="https://github.com/apache/datafusion/commit/7e20eb7ddb3acf8174af7adec52859e28333d570"><code>7e20eb7</code></a> [branch-52] perf: Cache num_output_rows in sort merge join to avoid O(n) reco...</li> <li><a href="https://github.com/apache/datafusion/commit/e5547e2772fbaed693e7472f38feab690a7fe3ef"><code>e5547e2</code></a> [branch-52] Fix duplicate group keys after hash aggregation spill (<a href="https://redirect.github.com/apache/datafusion/issues/20724">#20724</a>) (#...</li> <li><a href="https://github.com/apache/datafusion/commit/2947378e9ef9dbdda75b4ff047edcfc1a06ef0d2"><code>2947378</code></a> [branch-52] fix: disable dynamic filter pushdown for non min/max aggregates (...</li> <li><a href="https://github.com/apache/datafusion/commit/41acbf8e4bb4ac15003bd5365661e6b17551f7f0"><code>41acbf8</code></a> [branch-52] fix: Return <code>probe_side.len()</code> for RightMark/Anti count(*) querie...</li> <li><a href="https://github.com/apache/datafusion/commit/a5f6fbb4cd89a47e1036986abe201def15542093"><code>a5f6fbb</code></a> [branch-52] fix: interval analysis error when have two filterexec that inner ...</li> <li>Additional commits viewable in <a href="https://github.com/apache/datafusion/compare/52.3.0...52.4.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com>
…ing wrong version (apache#2277)
…python (apache#2278) Addresses the security advisory GHSA-pwjx-qhcg-rvj4 for rustls-webpki < 0.103.10 in the Python bindings lockfile. This is a rebase of apache#2268 onto main which already includes the root Cargo.lock audit fix from apache#2274 (aws-lc-sys >= 0.39.0). ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? ## Are these changes tested? ci.
…ache#2281) Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.11.0 to 1.11.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/bytes/releases">bytes's releases</a>.</em></p> <blockquote> <h2>Bytes v1.11.1</h2> <h1>1.11.1 (February 3rd, 2026)</h1> <ul> <li>Fix integer overflow in <code>BytesMut::reserve</code></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md">bytes's changelog</a>.</em></p> <blockquote> <h1>1.11.1 (February 3rd, 2026)</h1> <ul> <li>Fix integer overflow in <code>BytesMut::reserve</code></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tokio-rs/bytes/commit/417dccdeff249e0c011327de7d92e0d6fbe7cc43"><code>417dccd</code></a> Release bytes v1.11.1 (<a href="https://redirect.github.com/tokio-rs/bytes/issues/820">#820</a>)</li> <li><a href="https://github.com/tokio-rs/bytes/commit/d0293b0e35838123c51ca5dfdf468ecafee4398f"><code>d0293b0</code></a> Merge commit from fork</li> <li>See full diff in <a href="https://github.com/tokio-rs/bytes/compare/v1.11.0...v1.11.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…che#2282) Bumps [time](https://github.com/time-rs/time) from 0.3.44 to 0.3.47. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/time-rs/time/releases">time's releases</a>.</em></p> <blockquote> <h2>v0.3.47</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> <h2>v0.3.46</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> <h2>v0.3.45</h2> <p>See the <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">changelog</a> for details.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/time-rs/time/blob/main/CHANGELOG.md">time's changelog</a>.</em></p> <blockquote> <h2>0.3.47 [2026-02-05]</h2> <h3>Security</h3> <ul> <li> <p>The possibility of a stack exhaustion denial of service attack when parsing RFC 2822 has been eliminated. Previously, it was possible to craft input that would cause unbounded recursion. Now, the depth of the recursion is tracked, causing an error to be returned if it exceeds a reasonable limit.</p> <p>This attack vector requires parsing user-provided input, with any type, using the RFC 2822 format.</p> </li> </ul> <h3>Compatibility</h3> <ul> <li>Attempting to format a value with a well-known format (i.e. RFC 3339, RFC 2822, or ISO 8601) will error at compile time if the type being formatted does not provide sufficient information. This would previously fail at runtime. Similarly, attempting to format a value with ISO 8601 that is only configured for parsing (i.e. <code>Iso8601::PARSING</code>) will error at compile time.</li> </ul> <h3>Added</h3> <ul> <li>Builder methods for format description modifiers, eliminating the need for verbose initialization when done manually.</li> <li><code>date!(2026-W01-2)</code> is now supported. Previously, a space was required between <code>W</code> and <code>01</code>.</li> <li><code>[end]</code> now has a <code>trailing_input</code> modifier which can either be <code>prohibit</code> (the default) or <code>discard</code>. When it is <code>discard</code>, all remaining input is ignored. Note that if there are components after <code>[end]</code>, they will still attempt to be parsed, likely resulting in an error.</li> </ul> <h3>Changed</h3> <ul> <li>More performance gains when parsing.</li> </ul> <h3>Fixed</h3> <ul> <li>If manually formatting a value, the number of bytes written was one short for some components. This has been fixed such that the number of bytes written is always correct.</li> <li>The possibility of integer overflow when parsing an owned format description has been effectively eliminated. This would previously wrap when overflow checks were disabled. Instead of storing the depth as <code>u8</code>, it is stored as <code>u32</code>. This would require multiple gigabytes of nested input to overflow, at which point we've got other problems and trivial mitigations are available by downstream users.</li> </ul> <h2>0.3.46 [2026-01-23]</h2> <h3>Added</h3> <ul> <li>All possible panics are now documented for the relevant methods.</li> <li>The need to use <code>#[serde(default)]</code> when using custom <code>serde</code> formats is documented. This applies only when deserializing an <code>Option<T></code>.</li> <li><code>Duration::nanoseconds_i128</code> has been made public, mirroring <code>std::time::Duration::from_nanos_u128</code>.</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/time-rs/time/commit/d5144cd2874862d46466c900910cd8577d066019"><code>d5144cd</code></a> v0.3.47 release</li> <li><a href="https://github.com/time-rs/time/commit/f6206b050fd54817d8872834b4d61f605570e89b"><code>f6206b0</code></a> Guard against integer overflow in release mode</li> <li><a href="https://github.com/time-rs/time/commit/1c63dc7985b8fa26bd8c689423cc56b7a03841ee"><code>1c63dc7</code></a> Avoid denial of service when parsing Rfc2822</li> <li><a href="https://github.com/time-rs/time/commit/5940df6e72efb63d246ca1ca59a0f836ad32ad8a"><code>5940df6</code></a> Add builder methods to avoid verbose construction</li> <li><a href="https://github.com/time-rs/time/commit/00881a4da1bc5a6cb6313052e5017dbd7daa40f0"><code>00881a4</code></a> Manually format macros everywhere</li> <li><a href="https://github.com/time-rs/time/commit/bb723b6d826e46c174d75cd08987061984b0ceb7"><code>bb723b6</code></a> Add <code>trailing_input</code> modifier to <code>end</code></li> <li><a href="https://github.com/time-rs/time/commit/31c4f8e0b56e6ae24fe0d6ef0e492b6741dda783"><code>31c4f8e</code></a> Permit <code>W12</code> in <code>date!</code> macro</li> <li><a href="https://github.com/time-rs/time/commit/490a17bf306576850f33a86d3ca95d96db7b1dcd"><code>490a17b</code></a> Mark error paths in well-known formats as cold</li> <li><a href="https://github.com/time-rs/time/commit/6cb1896a600be1538ecfab8f233fe9cfe9fa8951"><code>6cb1896</code></a> Optimize <code>Rfc2822</code> parsing</li> <li><a href="https://github.com/time-rs/time/commit/6d264d59c25e3da0453c3defebf4640b0086a006"><code>6d264d5</code></a> Remove erroneous <code>#[inline(never)]</code> attributes</li> <li>Additional commits viewable in <a href="https://github.com/time-rs/time/compare/v0.3.44...v0.3.47">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-rust/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - N/A. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> - Add DataFusion Comet to the list of users with a description. ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> - N/A.
…ache#2026) Add Core Encryption Primitives for Iceberg Encryption Support. Part of apache#2034 ## Summary This PR introduces the foundational cryptographic primitives needed for implementing encryption in iceberg-rust, providing AES-GCM encryption operations that match the Java implementation's behavior and data format. ## Motivation Iceberg's Java implementation supports table-level encryption to protect sensitive data at rest. To achieve feature parity and ensure interoperability between Java and Rust implementations, we need to build encryption support from the ground up. This PR provides the core cryptographic operations that will serve as the foundation for the complete encryption feature. ## Changes New Module: encryption Added a new encryption module with core AES-GCM cryptographic operations: - encryption/crypto.rs - Core encryption implementation - EncryptionAlgorithm enum supporting AES-128-GCM as this is the only algorithm currently supported in arrow parquet - SecureKey struct with automatic memory zeroization for security - AesGcmEncryptor providing encrypt/decrypt operations with AAD support Key Features 1. Java-Compatible Format: Ciphertext format matches Java's implementation exactly: [12-byte nonce][encrypted data][16-byte GCM authentication tag] 1. This ensures files encrypted by Java can be decrypted by Rust and vice versa. 2. Secure Key Handling: Uses the zeroize crate to automatically clear encryption keys from memory when dropped, preventing key material from lingering in memory. 3. Additional Authenticated Data (AAD): Full support for AAD to ensure integrity of associated metadata that isn't encrypted. 4. Comprehensive Testing: 8 tests covering: - Round-trip encryption/decryption for both AES-128 and AES-256 - AAD validation - Empty plaintext handling - Tamper detection - Format compatibility verification Dependencies Added - aes-gcm = "0.10" - Industry-standard AES-GCM implementation - zeroize = "1.7" - Secure memory cleanup for encryption keys Compatibility This implementation directly corresponds to Java's https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/Ciphers.java: | Java Class | Rust Implementation | |-----------------------------|------------------------------------------| | Ciphers.AesGcmEncryptor | AesGcmEncryptor::encrypt() | | Ciphers.AesGcmDecryptor | AesGcmEncryptor::decrypt() | | EncryptionAlgorithm.AES_GCM | EncryptionAlgorithm::Aes128Gcm| Testing Future Work This PR is the first in a series to implement full encryption support. Upcoming PRs will add: 1. Table properties for encryption configuration 2. Key management interfaces (KeyManagementClient trait) 3. EncryptionManager implementation 4. Native Parquet encryption integration 5. AWS KMS support 6. Integration with Table and FileIO Review Notes - This PR is intentionally minimal and self-contained - No existing code paths are modified - this is purely additive - The module is public but won't be used until future PRs wire it up - Format compatibility with Java has been prioritized to ensure interoperability ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. apache#2035 ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2133 ## What changes are included in this PR? - Add catalog/utils.rs to provide helpers to delete table data using file_io and table_metadata - Add new API `purge_table` to `Catalog` trait and add default implementation - Implement purge_table for S3TableCatalog and RestCatalog <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Added new tests in table_suite <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
- Bump DataFusion to 53.0.0, Arrow/Parquet to 58, sqllogictest to 0.29,
pyo3 to 0.28.
- Adapt to DataFusion 53 API changes in physical plan executors and
python bindings.
- Update SLT expected test output.
## Are these changes tested?
Existing tests.
---------
Co-authored-by: Xander <zander181@googlemail.com>
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.22.0 to 1.23.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/uuid-rs/uuid/releases">uuid's releases</a>.</em></p> <blockquote> <h2>v1.23.0</h2> <h2>What's Changed</h2> <ul> <li>feat: add support for 'hyphenated' format in the serde module by <a href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li> <li>Fix a number of bugs in time-related code by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/872">uuid-rs/uuid#872</a></li> <li>Reword invalid char error message by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/873">uuid-rs/uuid#873</a></li> <li>Impl cleanups by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/874">uuid-rs/uuid#874</a></li> <li>Use LazyLock to synchronize v1/v6 context initialization by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/875">uuid-rs/uuid#875</a></li> <li>Prepare for 1.23.0 release by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/876">uuid-rs/uuid#876</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/FrenchDilettante"><code>@FrenchDilettante</code></a> made their first contribution in <a href="https://redirect.github.com/uuid-rs/uuid/pull/865">uuid-rs/uuid#865</a></li> </ul> <h2>Special thanks</h2> <p><a href="https://github.com/meng-xu-cs"><code>@meng-xu-cs</code></a> raised a series of bugs against the timestamp logic in <code>uuid</code> using automated tooling. The issues themselves were reasonably and responsibly presented and the end result is a better <code>uuid</code> library for everyone. Thanks!</p> <h1>Deprecations</h1> <p>This release includes the following deprecations:</p> <ul> <li><code>Context</code>: Renamed to <code>ContextV1</code></li> <li><code>Timestamp::from_gregorian</code>: Renamed to <code>Timestamp::from_gregorian_time</code></li> </ul> <h1>Change to <code>Version::Max</code></h1> <p><code>Version::Max</code>'s <code>u8</code> representation has changed from <code>0xff</code> to <code>0x0f</code> to match the value returned by <code>Uuid::get_version_num</code>.</p> <h1>Change to <code>Uuid::get_version</code> for the max UUID</h1> <p><code>Uuid::get_version</code> will only return <code>Some(Version::Max)</code> if the UUID is actually the max UUID (all bytes are <code>0xff</code>). Previously it would return <code>Some</code> if only the version field was <code>0x0f</code>. This change matches the behaviour of the nil UUID, which only returns <code>Some(Version::Nil)</code> if the UUID is the nil UUID (all bytes are <code>0x00</code>).</p> <p><strong>Full Changelog</strong>: <a href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/uuid-rs/uuid/commit/00ab922d5351607dfff520f37eb49cb9854fda73"><code>00ab922</code></a> Merge pull request <a href="https://redirect.github.com/uuid-rs/uuid/issues/876">#876</a> from uuid-rs/cargo/v1.23.0</li> <li><a href="https://github.com/uuid-rs/uuid/commit/726ba45fe3491bf6253173d0be6b99ed3b1cbbb9"><code>726ba45</code></a> prepare for 1.23.0 release</li> <li><a href="https://github.com/uuid-rs/uuid/commit/996dadea029e3976f52cba58e5e9b9a08c4f82c4"><code>996dade</code></a> Merge pull request <a href="https://redirect.github.com/uuid-rs/uuid/issues/875">#875</a> from uuid-rs/fix/context-ordering</li> <li><a href="https://github.com/uuid-rs/uuid/commit/e14047993bc5a6180a96119436a983c19d79b084"><code>e140479</code></a> simplify a use stmt</li> <li><a href="https://github.com/uuid-rs/uuid/commit/8ed9142847a22bc7707794bfee6b2016d4470772"><code>8ed9142</code></a> reorganize and document more v7 context internals</li> <li><a href="https://github.com/uuid-rs/uuid/commit/e09a3225a8d99c5eadcbbeb7432195b2ea5ece76"><code>e09a322</code></a> use LazyLock to synchronize v1/v6 context initialization</li> <li><a href="https://github.com/uuid-rs/uuid/commit/0f260cc67135ac20d914e387a47e59960247fdee"><code>0f260cc</code></a> Merge pull request <a href="https://redirect.github.com/uuid-rs/uuid/issues/874">#874</a> from uuid-rs/chore/impl-cleanups</li> <li><a href="https://github.com/uuid-rs/uuid/commit/1419e91097fcffc7afa8f54eb41fdc912200b540"><code>1419e91</code></a> clean up and refactor main lib tests</li> <li><a href="https://github.com/uuid-rs/uuid/commit/ceeaf4b7b59895497c59acdaf286233b1e7cc576"><code>ceeaf4b</code></a> ensure we don't overflow on counters less than 12</li> <li><a href="https://github.com/uuid-rs/uuid/commit/63bc8f52e5042b9c729fa0380b9948b49fe397cc"><code>63bc8f5</code></a> Merge pull request <a href="https://redirect.github.com/uuid-rs/uuid/issues/873">#873</a> from uuid-rs/fix/error-msg</li> <li>Additional commits viewable in <a href="https://github.com/uuid-rs/uuid/compare/v1.22.0...v1.23.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? Relates to apache/iceberg#15742 This PR - Add "ASF allowlist check" - Pin commit for codeql.yml (zizmor recommended) - Add back Github Action auto-update for dependabot (reverts apache#2267) - Add cooldown to dependabot (zizmor recommended) - `Swatinem/rust-cache@v2` -> `swatinem/rust-cache@v2` (fix case sensitivity) [asf infra allowlist uses lowercase](https://github.com/apache/infrastructure-actions/blob/fae466bc0d9821859a623cbc7648c750ff359ec6/approved_patterns.yml#L271) We can add back dependabot for github action because the "ASF allowlist check" will now alert when an action is not allowed (failures will no longer be silent) <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## What changes are included in this PR? - Make `convert_filters_to_predicate` public in the DataFusion integration to allow external usage of the filter conversion logic. ## Are these changes tested? - This is a visibility change (`pub use`) and does not introduce new logic. Co-authored-by: Denis Semenov <d.s.semenov@vkteam.ru>
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> Relates to apache/iceberg#15742 Follow up to apache#2289 ## What changes are included in this PR? Fix github workflow based on zizmor recommendation for security best practice <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Yes ``` ➜ iceberg-rust git:(kevinjqliu/zizmor-fix) uvx --from zizmor zizmor --offline .github/ 🌈 zizmor v1.23.1 INFO audit: zizmor: 🌈 completed .github/actions/get-msrv/action.yml INFO audit: zizmor: 🌈 completed .github/actions/overwrite-package-version/action.yml INFO audit: zizmor: 🌈 completed .github/actions/setup-builder/action.yml INFO audit: zizmor: 🌈 completed .github/dependabot.yml INFO audit: zizmor: 🌈 completed .github/workflows/audit.yml INFO audit: zizmor: 🌈 completed .github/workflows/bindings_python_ci.yml INFO audit: zizmor: 🌈 completed .github/workflows/ci.yml INFO audit: zizmor: 🌈 completed .github/workflows/ci_typos.yml INFO audit: zizmor: 🌈 completed .github/workflows/codeql.yml INFO audit: zizmor: 🌈 completed .github/workflows/publish.yml INFO audit: zizmor: 🌈 completed .github/workflows/release_python.yml INFO audit: zizmor: 🌈 completed .github/workflows/release_python_nightly.yml INFO audit: zizmor: 🌈 completed .github/workflows/stale.yml INFO audit: zizmor: 🌈 completed .github/workflows/website.yml No findings to report. Good job! (1 ignored, 37 suppressed) ```
…tion test (apache#2294) ## Which issue does this PR close? - Closes apache#352. ## What changes are included in this PR? - Replaces hardcoded `-1` with `EMPTY_SNAPSHOT_ID` constant in table metadata deserialization. - Adds `test_empty_snapshot_id_is_normalized_to_none` to verify that the Java-style `-1` sentinel for `current-snapshot-id` is normalized to `None` during deserialization. - Removes the public `UNASSIGNED_SNAPSHOT_ID` constant and moving it to a private constant scoped to the manifest writer module. ## Are these changes tested? Adds a test `test_empty_snapshot_id_is_normalized_to_none` verifying the deserialization normalization.
## Which issue does this PR close? This is an intermediate PR for apache#1731 I'm splitting out changes from apache#1851 to the compression codec to make it easier to review. Once we decide on approach here and merge it I'll update apache#1851 accordingly. ## What changes are included in this PR? - Add optional compression level to gzip and zstd (needed for when avro compression usage). - Add Snappy as a compression codec (also will be used for Avro) - Manually code up some previously auto-generated methods as a result. AI helped with an initial version of this PR. ## Are these changes tested? Additional unit tests
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes apache#2241 ## What changes are included in this PR? - Add `Ancestors` to help scan past snapshots - Moved existing util to the new utils mod <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Yes <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
vustef
approved these changes
Apr 2, 2026
| /// where only the relative path is needed (e.g. bulk deletes where the operator is already | ||
| /// available). | ||
| #[allow(unreachable_code, unused_variables)] | ||
| pub(crate) fn relativize_path<'a>(&self, path: &'a str) -> Result<&'a str> { |
Collaborator
There was a problem hiding this comment.
Do we need our RefreshableStorage here? Or we don't end up calling this function before we resolve to the underlying storage?
Collaborator
Author
There was a problem hiding this comment.
relativize_path is purely path parsing — it just strips the scheme/bucket prefix to return the relative path string, with no operator creation or credential interaction. It's only called in
delete_stream for the "reuse existing deleter" branch, meaning the operator was already created (with credentials resolved) in the Vacant branch above it.
RefreshableOpenDalStorage lives in the iceberg crate and handles credential refresh at the operator-construction level (create_operator). By the time relativize_path is called for a reused
deleter, you've already resolved to the underlying storage — so no, RefreshableStorage doesn't need to be involved here.
The only potential gap is if credentials expire mid-stream while iterating a long path list — the reused opendal::Deleter would fail. But that's a pre-existing limitation of the batching
design, not a relativize_path concern.```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.