Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/snowflake compatibility #2181

Merged
merged 2 commits into from
Jun 18, 2024

Conversation

blythed
Copy link
Collaborator

@blythed blythed commented Jun 17, 2024

We should update ibis for the latest features and support. This required a few fixes, which this PR provides.

@blythed blythed force-pushed the fix/snowflake-compatibility branch from c0a3183 to a9b7d73 Compare June 17, 2024 21:16
Copy link
Collaborator

@jieguangzhou jieguangzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After adding this line except (NoSuchTableError, IbisError) then the UT pass

diff --git a/superduperdb/backends/ibis/data_backend.py b/superduperdb/backends/ibis/data_backend.py
index af3085ad2..bd0ec0462 100644
--- a/superduperdb/backends/ibis/data_backend.py
+++ b/superduperdb/backends/ibis/data_backend.py
@@ -5,7 +5,7 @@ from warnings import warn
 import ibis
 import pandas
 from ibis.backends import BaseBackend
-
+from ibis.common.exceptions import IbisError
 from pandas.core.frame import DataFrame
 from sqlalchemy.exc import NoSuchTableError

@@ -153,7 +153,7 @@ class IbisDataBackend(BaseDataBackend):
         try:
             self.conn.table(f'_outputs.{predict_id}')
             return True
-        except NoSuchTableError:
+        except (NoSuchTableError, IbisError):
             return False

@blythed blythed force-pushed the fix/snowflake-compatibility branch from a9b7d73 to 5c6a2b4 Compare June 18, 2024 09:00
@blythed blythed force-pushed the fix/snowflake-compatibility branch from 5c6a2b4 to 4028af1 Compare June 18, 2024 09:02
@blythed blythed marked this pull request as ready for review June 18, 2024 09:02
@blythed
Copy link
Collaborator Author

blythed commented Jun 18, 2024

After adding this line except (NoSuchTableError, IbisError) then the UT pass

diff --git a/superduperdb/backends/ibis/data_backend.py b/superduperdb/backends/ibis/data_backend.py
index af3085ad2..bd0ec0462 100644
--- a/superduperdb/backends/ibis/data_backend.py
+++ b/superduperdb/backends/ibis/data_backend.py
@@ -5,7 +5,7 @@ from warnings import warn
 import ibis
 import pandas
 from ibis.backends import BaseBackend
-
+from ibis.common.exceptions import IbisError
 from pandas.core.frame import DataFrame
 from sqlalchemy.exc import NoSuchTableError

@@ -153,7 +153,7 @@ class IbisDataBackend(BaseDataBackend):
         try:
             self.conn.table(f'_outputs.{predict_id}')
             return True
-        except NoSuchTableError:
+        except (NoSuchTableError, IbisError):
             return False

Ah yes, thanks. This is because ibis now has dropped using SQL-Alchemy I believe.

@blythed blythed force-pushed the fix/snowflake-compatibility branch 3 times, most recently from 512d6d0 to d2a7148 Compare June 18, 2024 09:42

self.job_table = Table(
'job',
'JOB',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Snowflake doesn't work with lower case tables for some reason.

@@ -103,13 +93,6 @@ def _init_tables(self):
*component_table_args,
)

self.meta_table = Table(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used.


metadata = MetaData()
self.query_id_table = Table(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used.

@blythed blythed requested a review from fnikolai June 18, 2024 09:56
@blythed blythed force-pushed the fix/snowflake-compatibility branch 2 times, most recently from cc5e912 to 503e622 Compare June 18, 2024 14:30
@blythed blythed force-pushed the fix/snowflake-compatibility branch from 503e622 to 46c4020 Compare June 18, 2024 14:33
@blythed blythed merged commit acaafde into superduper-io:main Jun 18, 2024
3 checks passed
@blythed blythed deleted the fix/snowflake-compatibility branch July 8, 2024 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants