Skip to content

DuckDB: on_connection_create callback not passed to pool, never invoked #340

@cofin

Description

@cofin

Summary

The on_connection_create callback in DuckDBConfig.driver_features is accepted and processed, but never actually invoked when connections are created because it's not passed to the DuckDBConnectionPool.

Steps to Reproduce

from sqlspec.adapters.duckdb import DuckDBConfig

def my_callback(connection):
    print("Callback called!")
    connection.execute("LOAD postgres")
    connection.execute("ATTACH 'host=localhost port=5432 ...' AS pg (TYPE POSTGRES)")

config = DuckDBConfig(
    connection_config={"database": "/tmp/test.db"},
    driver_features={
        "on_connection_create": my_callback,
    },
)

# The callback is never called
with config.provide_session() as session:
    # Expected: "Callback called!" printed
    # Actual: Nothing printed, callback never invoked
    pass

Root Cause

In sqlspec/adapters/duckdb/config.py:

  1. Line 313: The callback is correctly popped from driver_features:

    user_connection_hook = cast("Callable[[Any], None] | None", features.pop("on_connection_create", None))
  2. Lines 323-327: It's wrapped and stored in observability config:

    if user_connection_hook is not None:
        lifecycle_override = ObservabilityConfig(
            lifecycle={"on_connection_create": [_DuckDBConnectionHook(user_connection_hook)]}
        )
        local_observability = ObservabilityConfig.merge(local_observability, lifecycle_override)
  3. Lines 365-371: But _create_pool() does NOT pass it to the pool:

    return DuckDBConnectionPool(
        connection_config=connection_config,
        extensions=extensions_dicts,
        extension_flags=extension_flags_dict,
        secrets=secrets_dicts,
        **pool_kwargs,
    )
    # Missing: on_connection_create=<extracted_callback>
  4. In pool.py lines 177-179: The pool DOES support and call the callback, but it's never passed:

    if self._on_connection_create:
        with suppress(Exception):
            self._on_connection_create(connection)

Expected Behavior

The on_connection_create callback should be invoked when DuckDB connections are created, allowing users to perform post-connection setup like:

  • Attaching external databases (PostgreSQL, MySQL, etc.)
  • Setting session-level variables
  • Loading additional extensions with custom configuration

Suggested Fix

In DuckDBConfig._create_pool(), extract the user callback from observability config and pass it to the pool:

def _create_pool(self) -> DuckDBConnectionPool:
    # ... existing code ...
    
    # Extract on_connection_create from observability lifecycle
    on_connection_create = None
    if self.observability_config and self.observability_config.lifecycle:
        hooks = self.observability_config.lifecycle.get("on_connection_create", [])
        if hooks:
            # Unwrap the _DuckDBConnectionHook to get the original callback
            # Or create a wrapper that calls all hooks
            def combined_hook(conn):
                for hook in hooks:
                    hook({"connection": conn})
            on_connection_create = combined_hook
    
    return DuckDBConnectionPool(
        connection_config=connection_config,
        extensions=extensions_dicts,
        extension_flags=extension_flags_dict,
        secrets=secrets_dicts,
        on_connection_create=on_connection_create,  # Add this
        **pool_kwargs,
    )

Environment

  • SQLSpec version: Latest (installed via pip)
  • Python version: 3.12
  • DuckDB version: Latest

Impact

This bug prevents any use of on_connection_create for DuckDB, which is documented as a supported feature in DuckDBDriverFeatures. Use cases affected include:

  • Attaching PostgreSQL/MySQL databases for cross-database queries
  • Setting up custom session configuration
  • Any post-connection initialization

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions