Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS: Add S3 Access Grants Documentation #9412

Closed
wants to merge 97 commits into from
Closed

AWS: Add S3 Access Grants Documentation #9412

wants to merge 97 commits into from

Conversation

adnanhemani
Copy link
Contributor

This is a followup to #9385 to add public-facing documentation on how to use the feature.

@github-actions github-actions bot added the docs label Jan 4, 2024
jackye1995
jackye1995 approved these changes Jan 5, 2024
View reviewed changes
Copy link
Contributor

@jackye1995 jackye1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

geruh
geruh reviewed Jan 25, 2024
View reviewed changes
docs/aws.md
--conf spark.sql.catalog.my_catalog.s3.access-grants.enabled=true \
--conf spark.sql.catalog.my_catalog.s3.access-grants.fallback-to-iam=true
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the convention of the other sections, can we append a line here that includes a reference to the AWS public documentation on S3 Access Grants: https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants.html

adnanhemani and others added 26 commits January 29, 2024 14:27
@adnanhemani
Appended further details link
a4719e8
@dependabot @adnanhemani
Build: Bump com.google.errorprone:error_prone_annotations (apache#9369)
b5f5c0c 
Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.23.0 to 2.24.0.
- [Release notes](https://github.com/google/error-prone/releases)
- [Commits](google/error-prone@v2.23.0...v2.24.0)

---
updated-dependencies:
- dependency-name: com.google.errorprone:error_prone_annotations
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@cgpoh @adnanhemani
Flink 1.17: Create JUnit5 version of TestFlinkScan (apache#9185)
a68230e
@ajantha-bhat @adnanhemani
Nessie: Strip trailing slash for warehouse location (apache#9415)
fa646a3
@chinmay-bhat @adnanhemani
Spark 3.5: Migrate tests to JUnit5 (apache#9417)
6a9f745
@nastra @adnanhemani
Spark 3.4: Add support for reading Iceberg views (apache#9422)
a2321bd
@dependabot @adnanhemani
Build: Bump org.assertj:assertj-core from 3.24.2 to 3.25.1 (apache#9427)
3281053 
Bumps [org.assertj:assertj-core](https://github.com/assertj/assertj) from 3.24.2 to 3.25.1.
- [Release notes](https://github.com/assertj/assertj/releases)
- [Commits](assertj/assertj@assertj-build-3.24.2...assertj-build-3.25.1)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@manuzhang @adnanhemani
Parquet: Support reading INT96 column in row group filter (apache#8988)
35d2884
@dependabot @adnanhemani
Build: Bump mkdocs-monorepo-plugin from 1.0.5 to 1.1.0 (apache#9430)
d4e990f 
Bumps [mkdocs-monorepo-plugin](https://github.com/backstage/mkdocs-monorepo-plugin) from 1.0.5 to 1.1.0.
- [Changelog](https://github.com/backstage/mkdocs-monorepo-plugin/blob/master/docs/CHANGELOG.md)
- [Commits](https://github.com/backstage/mkdocs-monorepo-plugin/commits)

---
updated-dependencies:
- dependency-name: mkdocs-monorepo-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @adnanhemani
Build: Bump com.google.errorprone:error_prone_annotations (apache#9429)
04c7738 
Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.24.0 to 2.24.1.
- [Release notes](https://github.com/google/error-prone/releases)
- [Commits](google/error-prone@v2.24.0...v2.24.1)

---
updated-dependencies:
- dependency-name: com.google.errorprone:error_prone_annotations
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@Fokko @adnanhemani
Parquet: Move to ValueReader generation to a visitor (apache#9063)
558a353 
* Parquet: Move to the visitor

* Add one more edge case

* Thanks Amogh!
@nastra @adnanhemani
Core: Add JUnit5 version of TableTestBase (apache#9424)
ee53d1e
@dependabot @adnanhemani
Build: Bump software.amazon.awssdk:bom from 2.21.42 to 2.22.12 (apach…
fc29b78 
…e#9426)

Bumps software.amazon.awssdk:bom from 2.21.42 to 2.22.12.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @adnanhemani
Build: Bump com.fasterxml.jackson.dataformat:jackson-dataformat-xml (a…
5514ee2 
…pache#9395)

Bumps [com.fasterxml.jackson.dataformat:jackson-dataformat-xml](https://github.com/FasterXML/jackson-dataformat-xml) from 2.16.0 to 2.16.1.
- [Commits](FasterXML/jackson-dataformat-xml@jackson-dataformat-xml-2.16.0...jackson-dataformat-xml-2.16.1)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson.dataformat:jackson-dataformat-xml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@bitsondatadev @adnanhemani
Shift site build to use monorepo and gh-pages
42a5e0a
@bitsondatadev @munabedan @adnanhemani
Established structured folders to customize MkDocs Material to an Ice…
a6ff3e6 
…berg look and feel

Co-authored-by: Muna Bedan <munabedan@gmail.com>
@rodmeneses @adnanhemani
Flink: Watermark read options (apache#9346)
a97f943
@ajantha-bhat @adnanhemani
API, Core: Fix errorprone warnings (apache#9419)
8b41e9e 
* API, Core: Fix errorprone warnings

* Address comments

* Remove comment
@bryanck @adnanhemani
Kafka Connect: Initial project setup and event data structures (apach…
b3a53bd 
…e#8701)
@chinmay-bhat @adnanhemani
Spark 3.5: Migrate tests in SQL directory to JUnit5 (apache#9401)
4036039
@ggershinsky @adnanhemani
Spark 3.5: Support encrypted output files (apache#9435)
42603f0
@rodmeneses @adnanhemani
Backporting Flink: Watermark Read Options to 1.17 and 1.16 (apache#9456)
0cef742
@vinitpatni @adnanhemani
Flink: Migrate subclasses of FlinkCatalogTestBase to JUnit5 (apache#9381
546396a 
)
@chinmay-bhat @adnanhemani
Spark 3.5: Migrate remaining tests in source directory to JUnit5 (apa…
58d5c45 
…che#9380)
@ggershinsky @adnanhemani
Core: Minor updates to AES GCM streams (apache#9453)
d4ceeb3
@dependabot @adnanhemani
Build: Bump actions/checkout from 3 to 4 (apache#9474)
f05ecd5 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dependabot bot and others added 10 commits January 30, 2024 12:11
@dependabot @adnanhemani
Build: Bump org.apache.httpcomponents.client5:httpclient5 (apache#9572)
e8933fa 
Bumps [org.apache.httpcomponents.client5:httpclient5](https://github.com/apache/httpcomponents-client) from 5.2.3 to 5.3.1.
- [Changelog](https://github.com/apache/httpcomponents-client/blob/rel/v5.3.1/RELEASE_NOTES.txt)
- [Commits](apache/httpcomponents-client@rel/v5.2.3...rel/v5.3.1)

---
updated-dependencies:
- dependency-name: org.apache.httpcomponents.client5:httpclient5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@nastra @adnanhemani
Spark 3.4: Support creating views via SQL (apache#9580)
9716e7a
@rodmeneses @adnanhemani
Flink: Adds the ability to read from a branch on the Flink Iceberg So…
6af674f 
…urce (apache#9547)
@mas-chen @adnanhemani
Flink: Implement enumerator metrics for pending splits, pending recor…
ff77d86 
…ds, and split discovery (apache#9524)
@amogh-jahagirdar @adnanhemani
Spark 3.4: Fix writing of default values in CoW for rows with NULL co…
24636f3 
…lumns which are unmatched (apache#9556)
@nastra @adnanhemani
Spark: Rewrite identifier when using Subquery expressions in View (ap…
292dd53 
…ache#9587)
@dependabot @adnanhemani
Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-…
08e8c1f 
…plugin (apache#9575)

Bumps [software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin](https://github.com/aws/aws-s3-accessgrants-plugin-java-v2) from 1.0.1 to 2.0.0.
- [Commits](https://github.com/aws/aws-s3-accessgrants-plugin-java-v2/commits)

---
updated-dependencies:
- dependency-name: software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @adnanhemani
Build: Bump software.amazon.awssdk:bom from 2.23.2 to 2.23.12 (apache…
f25abbd 
…#9573)

Bumps software.amazon.awssdk:bom from 2.23.2 to 2.23.12.

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@bitsondatadev @adnanhemani
Docs: Update ASF site to deploy from this repository (apache#9520)
5ee5714
@adnanhemani
changes rebased
09340b5
@adnanhemani
Copy link
Contributor Author

Git history was messed up due to a rebase done on top of shifting the docs file. Will close this PR and open a new one.

@adnanhemani adnanhemani mentioned this pull request Jan 30, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@geruh geruh geruh left review comments

@jackye1995 jackye1995 jackye1995 approved these changes

Assignees
No one assigned
Labels
API arrow AWS build core data DELL docs flink hive INFRA KAFKACONNECT MR NESSIE parquet SNOWFLAKE spark Specification Issues that may introduce spec changes.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet