v0.2.0rc11 - Release Candidate: Fix Redshift schema discovery on data sharing / Serverless clusters

Summary

Critical bug fix: on some Redshift clusters (data sharing, Lake Formation, Serverless), svv_tables returns 0 rows for users that DO have access. Adds auto-detection at init time with fallback chain: svv_tables -> svv_all_tables -> information_schema. Also fixes cross-DB column isolation and prevents caching empty results.

Bug Fixes

• svv_tables returns empty on some clusters: svv_tables visibility depends on cluster configuration and metadata security settings. When it returns 0 rows, nlp2sql thinks the schema is empty and fails to generate SQL. Added a probe chain that tests each system view at initialization and picks the first one that returns data.
• _get_table_columns missing database filter: When using svv_all_columns (fallback), the column query did not filter by database_name = current_database(). This could return columns from other databases sharing the same table name and schema, leading to incorrect or duplicated column definitions.
• Empty results cached for 24h: If all probes returned 0 tables (e.g., wrong schema name or temporary permission issue), the empty result was cached to disk with a 24h TTL, preventing recovery without manual cache deletion.
• State mutation in bulk query fallback: The exception handler in _get_tables_bulk mutated instance attributes (self._table_view, etc.) before retrying. Unsafe for concurrent coroutines and permanently changed the adapter behavior for all subsequent calls.

Changes

Aspect	Before	After
System views	Hardcoded svv_tables / svv_columns	Auto-detected: svv_tables -> svv_all_tables -> information_schema
Column name handling	table_schema only	Dynamic: table_schema or schema_name based on detected view
Cross-DB isolation	None	database_name = current_database() filter for svv_all_* views
Empty result caching	Cached 0 tables for 24h	Skipped -- empty results not persisted
Bulk query fallback	Mutates self._* and recurses	Separate method _get_tables_bulk_information_schema (no state mutation)
Schema case	Passed as-is	Normalized to lowercase on init

Column Name Mapping

Source	Schema column	Extra filter
svv_tables + svv_columns	table_schema	--
svv_all_tables + svv_all_columns	schema_name	database_name = current_database()
information_schema	table_schema	table_type IN ('BASE TABLE', 'VIEW')

What changed in redshift_adapter.py

• Added _table_view, _column_view, _schema_col, _db_filter, _db_filter_aliased instance attributes for dynamic view configuration
• Added _detect_system_views() -- probes 3 sources at init, picks first with rows > 0, logs which source won
• Added _process_bulk_rows() -- extracted row-to-TableInfo conversion, shared between primary and fallback bulk queries
• Added _get_tables_bulk_information_schema() -- standalone fallback method, no instance state mutation
• Updated _get_tables_bulk(), get_table_info(), search_tables(), get_schema_metadata(), _get_table_columns() to use dynamic view/column names via f-strings (internal values only, no SQL injection risk)
• Added _db_filter to _get_table_columns() WHERE clause for cross-DB isolation
• Added empty result guard in _save_tables_to_cache()
• Normalized schema_name to lowercase in __init__

Tests

Suite	Count	Status
Existing unit tests	195	Passed
New detection/fallback tests	8	Passed
Total	203 passed

New tests:

• test_detect_probe1_wins -- svv_tables succeeds
• test_detect_probe2_wins -- svv_tables empty, svv_all_tables succeeds
• test_detect_probe3_wins -- both SVV empty, information_schema succeeds
• test_detect_all_probes_fail -- all return 0, defaults kept
• test_detect_exception_falls_through -- probe throws, falls to next
• test_schema_name_normalized_to_lowercase -- case normalization
• test_process_bulk_rows_empty -- empty input returns empty list
• test_process_bulk_rows_groups_by_table -- rows grouped correctly

Verification

• nlp2sql inspect against affected cluster -- detects svv_all_tables, finds 95 tables with 2087 columns
• nlp2sql query generates valid SQL and executes against Redshift
• Cache reuse: same schema loads from disk cache, different schema gets fresh query
• Empty schema (public) does not persist to disk cache
• Installed from TestPyPI and verified end-to-end

Files Changed

• 5 files modified
• src/nlp2sql/adapters/redshift_adapter.py (+122, -53)
• src/nlp2sql/__init__.py (version bump)
• tests/test_redshift_integration.py (+100)
• pyproject.toml (version bump)
• uv.lock (regenerated)

Closes

• #37