-
Notifications
You must be signed in to change notification settings - Fork 360
Add doc for not supported operations for credential vending #398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
||
| - **remove_orphan_files:** This operation does not currently support | ||
| credential vending when utilizing Apache Spark as the compute engine. | ||
| - For more details, please refer to [apache/iceberg#7914](https://github.com/apache/iceberg/pull/7914). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be using the new Hugo-style links? I am honestly not clear on that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@eric-maynard is right, I think it's better to use Hugo style link to have a clean documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the info. Will finish validations later today then update the PR with suggestions above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From some quick read, Hugo style link is mainly for internal links within the site and external links seems to be preferred as markdown's style instead. Also, there are many places we are using markdown style links for external reference across current site. If that is more preferred, should we do it in a different PR instead?
Then in terms of table maintenance operations validation with credential vending, following are tested:
Trino:
- optimize: passed
- expire_snapshots: passed
- remove_orphan_files: passed
- drop_extended_stats: passed
Spark:
- expire_snapshots: passed
- remove_orphan_files: failed (not supported as mentioned in this PR)
- rewrite_data_files: passed
- rewrite_manifests: passed
- rewrite_position_delete_files: passed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MonkeyCanCode thanks a lot for validating maintenance procedures. It's a good callout, but it is essentially an Spark issue. Maybe we can reword to something like this:
- **remove_orphan_files:** Apache Spark can't use credential vending for this due to a known issue. See [apache/iceberg#7914](https://github.com/apache/iceberg/pull/7914) for details.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anytime. Updated.
Description
This is for adding doc around which operations are not supported by credential vending.
Fixes # (390)
Type of change
Please delete options that are not relevant.
Checklist:
Please delete options that are not relevant.