-
Notifications
You must be signed in to change notification settings - Fork 497
Add more examples for pyiceberg view #3414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -1527,17 +1527,40 @@ def cleanup_old_snapshots(table_name: str, snapshot_ids: list[int]): | |||||
| cleanup_old_snapshots("analytics.user_events", [12345, 67890, 11111]) | ||||||
| ``` | ||||||
|
|
||||||
| ## Views | ||||||
| ## Create a view | ||||||
|
|
||||||
| PyIceberg supports view operations. | ||||||
|
|
||||||
| ### Check if a view exists | ||||||
| To create a view from a catalog: | ||||||
|
|
||||||
| ```python | ||||||
| import time | ||||||
| import pyarrow as pa | ||||||
| from pyiceberg.catalog import load_catalog | ||||||
| from pyiceberg.view import SQLViewRepresentation, ViewVersion | ||||||
|
|
||||||
| catalog = load_catalog("default") | ||||||
| catalog.view_exists("default.bar") | ||||||
|
|
||||||
| identifier = "default.some_view" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: I would just put the string directly in the create_view call |
||||||
| schema = pa.schema([pa.field("some_col", pa.int32())]) | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: for the example I believe would be best to show an Iceberg schema type instead of arrow but also note that create_view also accepts an arrow schema |
||||||
| view_version = ViewVersion( | ||||||
| version_id=1, | ||||||
| schema_id=1, | ||||||
| timestamp_ms=int(time.time() * 1000), | ||||||
| summary={}, | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just to show what the summary is usually used for |
||||||
| representations=[ | ||||||
| SQLViewRepresentation( | ||||||
| type="sql", | ||||||
| sql="SELECT 1 as some_col", | ||||||
| dialect="spark", | ||||||
| ) | ||||||
| ], | ||||||
| default_namespace=["default"], | ||||||
| ) | ||||||
|
|
||||||
| catalog.create_view( | ||||||
| identifier=identifier, | ||||||
| schema=schema, | ||||||
| view_version=view_version, | ||||||
| ) | ||||||
| ``` | ||||||
|
|
||||||
| ## Register a view | ||||||
|
|
@@ -1551,6 +1574,48 @@ catalog.register_view( | |||||
| ) | ||||||
| ``` | ||||||
|
|
||||||
| ## Load a view | ||||||
|
|
||||||
| Loading the `some_view` view: | ||||||
|
|
||||||
| ```python | ||||||
| view = catalog.load_view("default.some_view") | ||||||
| # Equivalent to: | ||||||
| view = catalog.load_view(("default", "some_view")) | ||||||
| # The tuple syntax can be used if the namespace or view contains a dot. | ||||||
| ``` | ||||||
|
|
||||||
| This returns a `View` that represents an Iceberg view. You can access the SQL representation for a specific dialect: | ||||||
|
|
||||||
| ```python | ||||||
| sql_representation = view.sql_for("spark") | ||||||
| print(sql_representation.sql) | ||||||
| ``` | ||||||
|
|
||||||
| ## Check if a view exists | ||||||
|
|
||||||
| To check whether the `some_view` view exists: | ||||||
|
|
||||||
| ```python | ||||||
| catalog.view_exists("default.some_view") | ||||||
| ``` | ||||||
|
|
||||||
| ## List views | ||||||
|
|
||||||
| To list views in the `default` namespace: | ||||||
|
|
||||||
| ```python | ||||||
| catalog.list_views("default") | ||||||
| ``` | ||||||
|
|
||||||
| ## Drop a view | ||||||
|
|
||||||
| To drop a view: | ||||||
|
|
||||||
| ```python | ||||||
| catalog.drop_view("default.some_view") | ||||||
| ``` | ||||||
|
|
||||||
| ## Table Statistics Management | ||||||
|
|
||||||
| Manage table statistics with operations through the `Table` API: | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would add a section on top of create view to specify that in order to use the view endpoints from the catalog, the view configuration needs to have
"view-endpoints-supported": "true"