Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to load delete and inventory_delete table data #88

Closed
wjxiz1992 opened this issue Aug 5, 2022 · 0 comments · Fixed by #87
Closed

Failed to load delete and inventory_delete table data #88

wjxiz1992 opened this issue Aug 5, 2022 · 0 comments · Fixed by #87
Assignees
Labels
bug Something isn't working

Comments

@wjxiz1992
Copy link
Collaborator

wjxiz1992 commented Aug 5, 2022

After #77, refresh data should be read as independent files, but our code to get delete and inventory_delete table data are still based on Iceberg, we need to change the code to deal with new read manner.

error messages:

Traceback (most recent call last):
  File "/usr/workspace/deploy/temp/spark-rapids-benchmarks/nds/nds_maintenance.py", line 207, in <module>
    query_dict = get_maintenance_queries(args.maintenance_queries_folder,
  File "/usr/workspace/deploy/temp/spark-rapids-benchmarks/nds/nds_maintenance.py", line 98, in get_maintenance_queries
    delete_date_dict = get_delete_date(spark)
  File "/usr/workspace/deploy/temp/spark-rapids-benchmarks/nds/nds_maintenance.py", line 65, in get_delete_date
    delete_dates = spark_session.sql("select * from delete").collect()
  File "/spark/python/lib/pyspark.zip/pyspark/sql/session.py", line 1034, in sql
  File "/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1321, in __call__
  File "/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 196, in deco
pyspark.sql.utils.AnalysisException: Table or view not found: delete; line 1 pos 14;
'Project [*]
+- 'UnresolvedRelation [delete], [], false
@wjxiz1992 wjxiz1992 added the bug Something isn't working label Aug 5, 2022
@wjxiz1992 wjxiz1992 self-assigned this Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant