Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,19 @@ SQLSpec is a type-safe SQL query mapper designed for minimal abstraction between
- **Single-Pass Processing**: Parse once → transform once → validate once - SQL object is single source of truth
- **Abstract Methods with Concrete Implementations**: Protocol defines abstract methods, base classes provide concrete sync/async implementations

### Driver Parameter Profile Registry

- All adapter parameter defaults live in `DriverParameterProfile` entries inside `sqlspec/core/parameters.py`.
- Use lowercase adapter keys (e.g., `"asyncpg"`, `"duckdb"`) and populate every required field: default style, supported styles, execution style, native list expansion flags, JSON strategy, and optional extras.
- JSON behaviour is controlled through `json_serializer_strategy`:
- `"helper"`: call `ParameterStyleConfig.with_json_serializers()` (dict/list/tuple auto-encode)
- `"driver"`: defer to driver codecs while surfacing serializer references for later registration
- `"none"`: skip JSON helpers entirely (reserve for adapters that must not touch JSON)
- Extras should encapsulate adapter-specific tweaks (e.g., `type_coercion_overrides`, `json_tuple_strategy`). Document new extras inline and keep them immutable.
- Always build `StatementConfig` via `build_statement_config_from_profile()` and pass adapter-specific overrides through the helper instead of instantiating configs manually in drivers.
- When introducing a new adapter, add its profile, update relevant guides, and extend unit coverage so each JSON strategy path is exercised.
- Record the canonical adapter key, JSON strategy, and extras in the corresponding adapter guide so contributors can verify behaviour without reading the registry source.

### Protocol Abstract Methods Pattern

When adding methods that need to support both sync and async configurations, use this pattern:
Expand Down Expand Up @@ -335,6 +348,13 @@ def test_starlette_autocommit_mode() -> None:
- Disabling pooling - Tests don't reflect production configuration
- Running tests serially - Slows down CI significantly

### CLI Config Loader Isolation Pattern

- When exercising CLI migration commands, generate a unique module namespace for each test (for example `cli_test_config_<uuid>`).
- Place temporary config modules inside `tmp_path` and register them via `sys.modules` within the test, then delete them during teardown to prevent bleed-through.
- Always patch `Path.cwd()` or provide explicit path arguments so helper functions resolve the test-local module rather than cached global fixtures.
- Add regression tests ensuring the helper cleaning logic runs even if CLI commands raise exceptions to avoid polluting later suites.

### Performance Optimizations

- **Mypyc Compilation**: Core modules can be compiled with mypyc for performance
Expand Down Expand Up @@ -2667,6 +2687,7 @@ def _extract_starlette_settings(self, config):
4. **Conditionally Skip DI Setup**:

**Middleware-based (Starlette/FastAPI)**:

```python
def init_app(self, app):
# ... lifespan setup ...
Expand All @@ -2677,6 +2698,7 @@ def init_app(self, app):
```

**Provider-based (Litestar)**:

```python
def on_app_init(self, app_config):
for state in self._plugin_configs:
Expand All @@ -2693,6 +2715,7 @@ def on_app_init(self, app_config):
```

**Hook-based (Flask)**:

```python
def init_app(self, app):
# ... pool setup ...
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/adbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions for the `adbc` adapter.
- **Driver:** Arrow Database Connectivity (ADBC) drivers (e.g., `adbc_driver_postgresql`, `adbc_driver_sqlite`).
- **Parameter Style:** Varies by underlying database (e.g., `numeric` for PostgreSQL, `qmark` for SQLite).

## Parameter Profile

- **Registry Key:** `"adbc"`
- **JSON Strategy:** `helper` (shared serializers wrap dict/list/tuple values)
- **Extras:** `type_coercion_overrides` ensure Arrow arrays map to Python lists; PostgreSQL dialects attach a NULL-handling AST transformer

## Best Practices

- **Arrow-Native:** The primary benefit of ADBC is its direct integration with Apache Arrow. Use it when you need to move large amounts of data efficiently between the database and data science tools like Pandas or Polars.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/aiosqlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions for the `aiosqlite` adapter.
- **Driver:** `aiosqlite`
- **Parameter Style:** `qmark` (e.g., `?`)

## Parameter Profile

- **Registry Key:** `"aiosqlite"`
- **JSON Strategy:** `helper` (shared serializer handles dict/list/tuple inputs)
- **Extras:** None (profile applies bool→int and ISO datetime coercions automatically)

## Best Practices

- **Async Only:** This is an asynchronous driver for SQLite. Use it in `asyncio` applications.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/asyncmy.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide covers `asyncmy`.
- **Driver:** `asyncmy`
- **Parameter Style:** `pyformat` (e.g., `%s`)

## Parameter Profile

- **Registry Key:** `"asyncmy"`
- **JSON Strategy:** `helper` (uses shared JSON serializers for dict/list/tuple)
- **Extras:** None (native list expansion remains disabled)

## Best Practices

- **Character Set:** Always ensure the connection character set is `utf8mb4` to support a full range of Unicode characters, including emojis.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/asyncpg.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions and best practices for working with th
- **Driver:** `asyncpg`
- **Parameter Style:** `numeric` (e.g., `$1, $2`)

## Parameter Profile

- **Registry Key:** `"asyncpg"`
- **JSON Strategy:** `driver` (delegates JSON binding to asyncpg codecs)
- **Extras:** None (codecs registered through config init hook)

## Best Practices

- **High-Performance:** `asyncpg` is often chosen for high-performance applications due to its speed. It's a good choice for applications with a high volume of database traffic.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions for the `bigquery` adapter.
- **Driver:** `google-cloud-bigquery`
- **Parameter Style:** `named` (e.g., `@name`)

## Parameter Profile

- **Registry Key:** `"bigquery"`
- **JSON Strategy:** `helper` with `json_tuple_strategy="tuple"`
- **Extras:** `type_coercion_overrides` keep list values intact while converting tuples to lists during binding

## Best Practices

- **Authentication:** BigQuery requires authentication with Google Cloud. For local development, the easiest way is to use the Google Cloud CLI and run `gcloud auth application-default login`.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/duckdb.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions for the `duckdb` adapter.
- **Driver:** `duckdb`
- **Parameter Style:** `qmark` (e.g., `?`)

## Parameter Profile

- **Registry Key:** `"duckdb"`
- **JSON Strategy:** `helper` (shared serializer covers dict/list/tuple)
- **Extras:** None (profile preserves existing `allow_mixed_parameter_styles=False`)

## Best Practices

- **In-Memory vs. File:** DuckDB can run entirely in-memory (`:memory:`) or with a file-based database. In-memory is great for fast, temporary analytics. File-based is for persistence.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/oracledb.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions and best practices for working with th
- **Driver:** `oracledb`
- **Parameter Style:** `named` (e.g., `:name`)

## Parameter Profile

- **Registry Key:** `"oracledb"`
- **JSON Strategy:** `helper` (shared JSON serializer applied through the profile)
- **Extras:** None (uses defaults with native list expansion disabled)

## Thick vs. Thin Client

The `oracledb` driver supports two modes:
Expand Down
39 changes: 39 additions & 0 deletions docs/guides/adapters/parameter-profile-registry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Driver Parameter Profile Registry

The table below summarizes the canonical `DriverParameterProfile` entries defined in `sqlspec/core/parameters/_registry.py`. Use it as a quick reference when updating adapters or adding new ones.

| Adapter | Registry Key | JSON Strategy | Extras | Default Dialect | Notes |
| --- | --- | --- | --- | --- | --- |
| ADBC | `"adbc"` | `helper` | `type_coercion_overrides` (list/tuple array coercion) | dynamic (per detected dialect) | Shares AST transformer metadata with BigQuery dialect helpers. |
| AioSQLite | `"aiosqlite"` | `helper` | None | `sqlite` | Mirrors SQLite defaults; bools coerced to ints for driver parity. |
| AsyncMy | `"asyncmy"` | `helper` | None | `mysql` | Native list expansion currently disabled until connector parity confirmed. |
| AsyncPG | `"asyncpg"` | `driver` | None | `postgres` | Relies on asyncpg codecs; JSON serializers referenced for later registration. |
| BigQuery | `"bigquery"` | `helper` | `json_tuple_strategy="tuple"`, `type_coercion_overrides` | `bigquery` | Enforces named parameters; tuple JSON payloads preserved as tuples. |
| DuckDB | `"duckdb"` | `helper` | None | `duckdb` | Mixed-style binding disabled; aligns bool/datetime coercion with SQLite. |
| OracleDB | `"oracledb"` | `helper` | None | `oracle` | List expansion disabled; LOB handling delegated to adapter converters. |
| PSQLPy | `"psqlpy"` | `helper` | None | `postgres` | Decimal values currently downcast to float for driver compatibility. |
| Psycopg | `"psycopg"` | `helper` | None | `postgres` | Array coercion delegated to psycopg adapters; JSON handled by shared converters. |
| SQLite | `"sqlite"` | `helper` | None | `sqlite` | Shares bool/datetime handling with DuckDB and CLI defaults. |

## Adding or Updating Profiles

1. Define the profile in `_registry.py` using lowercase key naming.
2. Pick the JSON strategy that matches driver capabilities (`helper`, `driver`, or `none`).
3. Declare extras as an immutable mapping; document each addition in this file and the relevant adapter guide.
4. Add or update regression coverage (see `specs/archive/driver-quality-review/research/testing_deliverables.md`).
5. If behaviour changes, update changelog entries and adapter guides accordingly.

Refer to [AGENTS.md](../../AGENTS.md) for the full checklist when touching the registry.

## Example Usage

```python
from sqlspec.core.parameters import get_driver_profile, build_statement_config_from_profile

profile = get_driver_profile("duckdb")
config = build_statement_config_from_profile(profile)

print(config.parameter_config.default_parameter_style)
```

The snippet above retrieves the DuckDB profile, builds a `StatementConfig`, and prints the default parameter style (`?`). Use the same pattern for new adapters after defining their profiles.
6 changes: 6 additions & 0 deletions docs/guides/adapters/psqlpy.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ This guide provides specific instructions for the `psqlpy` adapter for PostgreSQ
- **Parameter Style:** `numeric` (e.g., `$1, $2`)
- **Type System:** Rust-level type conversion (not Python-level)

## Parameter Profile

- **Registry Key:** `"psqlpy"`
- **JSON Strategy:** `helper` (shared JSON serializer applied before Rust-side codecs)
- **Extras:** Decimal writes coerce through `_decimal_to_float` to match Rust numeric expectations

## Architecture

Psqlpy handles type conversion differently than other PostgreSQL drivers:
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/psycopg.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide provides specific instructions and best practices for working with th
- **Driver:** `psycopg`
- **Parameter Style:** `pyformat` (e.g., `%s`)

## Parameter Profile

- **Registry Key:** `"psycopg"`
- **JSON Strategy:** `helper` (shared JSON serializer wraps dict/list/tuple values before psycopg adapters run)
- **Extras:** None (adapter-specific list/tuple converters remain in driver to preserve array semantics)

## Best Practices

- **General Purpose:** `psycopg` is a robust, general-purpose PostgreSQL adapter. It has excellent type handling and is a good choice for a wide variety of applications.
Expand Down
6 changes: 6 additions & 0 deletions docs/guides/adapters/sqlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This guide covers `sqlite3` (sync) and `aiosqlite` (async).
- **Driver:** `sqlite3` (built-in), `aiosqlite`
- **Parameter Style:** `qmark` (e.g., `?`)

## Parameter Profile

- **Registry Keys:** `"sqlite"` (sync), `"aiosqlite"` (async)
- **JSON Strategy:** `helper` for both drivers (shared serializer handles dict/list/tuple parameters)
- **Extras:** None (profiles apply ISO formatting for datetime/date and convert Decimal to string)

## Best Practices

- **Use Cases:** Ideal for testing, local development, and embedded applications. Not suitable for high-concurrency production workloads.
Expand Down
15 changes: 15 additions & 0 deletions sqlspec/adapters/adbc/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,21 @@ def __init__(
if "arrow_extension_types" not in driver_features:
driver_features["arrow_extension_types"] = True

json_serializer = driver_features.get("json_serializer")
if json_serializer is not None:
parameter_config = statement_config.parameter_config
previous_list_converter = parameter_config.type_coercion_map.get(list)
previous_tuple_converter = parameter_config.type_coercion_map.get(tuple)
updated_parameter_config = parameter_config.with_json_serializers(json_serializer)
updated_map = dict(updated_parameter_config.type_coercion_map)
if previous_list_converter is not None:
updated_map[list] = previous_list_converter
if previous_tuple_converter is not None:
updated_map[tuple] = previous_tuple_converter
statement_config = statement_config.replace(
parameter_config=updated_parameter_config.replace(type_coercion_map=updated_map)
)

super().__init__(
connection_config=self.connection_config,
migration_config=migration_config,
Expand Down
Loading
Loading