feat(duckdb): improve attach(), detach(), and attach_sqlite() #11198
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NickCrews
added
the
breaking change
github-actions
bot
added
tests
NickCrews
added
feature
NickCrews
force-pushed
the
duckdb-attach-sqlite
branch
from
9cc5741 to
80bf360
Compare
NickCrews
commented
May 9, 2025
ibis/backends/duckdb/__init__.py
Outdated
| The name to attach the database as. | ||
| If `None`, use the default behavior of DuckDB | ||
| (as of duckdb==1.2.0 this is the basename of the path). | ||
| skip_if_exists |
There was a problem hiding this comment.
very explicitly choosing this language of "skip_if_exists" instead of "exist_ok" because I don't want users to think this has "CREATE OR REPLACE" semantics.
There was a problem hiding this comment.
Could also do on_exists: Literal["ignore", "replace", "error"]="error" and then this would be more inline with how I want to change the create_table() and create_view() APIs to be in the future
There was a problem hiding this comment.
I went with on_exists: Literal["ignore", "error"] = "error" so that it would be consistent with on_missing: Literal["ignore", "error"]="error" of detach()
NickCrews
force-pushed
the
duckdb-attach-sqlite
branch
from
80bf360 to
a1345c2
Compare
NickCrews
changed the title
NickCrews
commented
Jun 6, 2025
ibis/backends/duckdb/__init__.py
Outdated
| cur.execute( | ||
| f"CALL sqlite_attach('{path}', overwrite={overwrite})" | ||
| ).fetchall() | ||
| self._safe_raw_sql(f"SET GLOBAL sqlite_all_varchar={all_varchar}") |
There was a problem hiding this comment.
I don't THINK it's needed to use this as a context manager here (or in fact most places within the duckdb backend), but double check me.
NickCrews
force-pushed
the
duckdb-attach-sqlite
branch
from
a1345c2 to
100149e
Compare
the sqlite_attach() approach is deprecated per https://duckdb.org/docs/stable/extensions/sqlite.html, and the newer syntax opens up read only flag, specifiying a name for the attachment, and not overwriting tables that already exist.
This is breaking in that some params are now keyword only.
NickCrews
force-pushed
the
duckdb-attach-sqlite
branch
from
100149e to
0c1d64d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The sqlite_attach() approach is deprecated per
https://duckdb.org/docs/stable/extensions/sqlite.html,
and the newer syntax opens up read only flag, specifiying a name for the attachment, and not overwriting tables that already exist.
This is breaking:
on_exists: Literal["ignore", "error"] = "error"This is prep for work I want to do to add attach_postgres(), but I want to get the API up to date so that the two APIs can be consistent. In fact, after this PR I don't even need another attach_postgres() method, since now the plain attach() is flexible enough to support this.