Skip to content

Discovery surface: report searchable columns, and make multi-database tool sets distinguishable #40

Description

@rohan-hotdata

Extends hotdata_langchain/schema.py (hotdata_describe_tables) to report which columns are searchable, and makes multi-database setups distinguishable to the model.

1. Report which columns are indexed

hotdata_describe_tables currently returns tables, columns and types from information_schema. It cannot say which columns are searchable — and that single missing fact is why hotdata_search_text has to pin its corpus at construction rather than letting the agent choose one.

Newly unblocked: this needs no engine change. Indexes are not visible in SQL —

SELECT * FROM pg_indexes                    -> table 'default.public.pg_indexes' not found
SELECT * FROM information_schema.indexes    -> table 'default.information_schema.indexes' not found

— but the control plane returns them, reachable with what the client already holds:

db  = client.resolve_managed_database(DATABASE)
idx = hotdata.IndexesApi(client.api).list_indexes(db.default_connection_id, "public", "listings")
# -> [('listings_description_bm25', 'bm25', ['description'], IndexStatus.READY)]

So the tool can annotate each column with whether it is searchable and by what kind of index, without waiting on runtimedb.

Design notes:

  • Report the capability, not the mechanism — "searchable by text relevance" rather than "has a BM25 index" — consistent with the naming rule enforced in tests/test_search.py. The raw index type can still appear as detail for humans debugging.
  • Only surface READY indexes as usable; a PENDING index will fail a search.
  • One list_indexes call per table, so the overview (no-argument) call should probably stay index-free and the per-table drill-down carry the annotation, to avoid N API calls for a wide database.

What this unlocks: once an agent can ask what is searchable, hotdata_search_text no longer needs a pinned corpus — the constrained-enum or agent-supplied-table variants become safe, because the agent picks from something it actually read rather than a guess. That is the discovery gap noted when the pinning decision was made.

2. Multi-database description clarity

One client can query many databases — execute_sql(sql, database=...) takes the scope per call, and the same client read from two different managed databases in one session. So registering several tool sets already works:

sales   = hl.make_hotdata_tools(client, database="<id>", search_tool_name="search_sales")
support = hl.make_hotdata_tools(client, database="<id>", search_tool_name="search_support")
agent   = create_agent(model=..., tools=[*sales, *support])

But the tool descriptions never mention which database each set addresses, so with two sets the model cannot tell them apart — and hotdata_execute_sql appears twice with identical text. Descriptions should name their database (and make_hotdata_tools likely needs a name suffix or prefix option so the SQL/describe tools do not collide either).

Cross-database references inside a single query do not work by default (table 'f1_db.public.drivers' not found from another database's scope). DatabasesApi.attach_database_catalog looks like the supported route — unverified, see docs/engine-contract.md.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-native-layerPart of the AI-native query layer effort for LangChainenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions