[DISCUSS] New Agentic lakehouse module & built-in AI agent gateway

[vinothchandar]

(Also mirrored to dev ML)

Hi all,

I'd like to propose adding an optional, self-hostable AI serving layer to
Hudi — making every Hudi table something you can talk to, and over time,
something agents can help operate.

Concretely: a set of Helm charts deploying Trino + a new agent gateway
service that hosts an LLM agent loop with guarded read-only SQL tools over
the lakehouse, exposes the same tools to external agents via MCP, and ships
a small chat UI. Local inference (Ollama) is the out-of-box default; API
providers and vLLM are values-file choices. Nothing about existing Hudi
APIs, configs, or storage changes — it's all additive.

Why in the project rather than as external glue: the tools can be
engine-native (catalog introspection today; incremental queries and table
services operations next), the stack stays fully open and self-hostable, and
answer quality can build on Hudi-specific knowledge.

• Umbrella feature issue with the breakdown: Agentic lakehouse: built-in AI agent serving and operations for Hudi #19256
• Initial implementation (charts + gateway + local dev environment): feat: agentic lakehouse v1 - hudi-lakehouse charts and hudi-agent-gateway #19265

Would love feedback on the direction, the proposed breakdown, and interest
in contributing — especially from folks running Hudi with Trino, or with
opinions on inference deployment and agent tooling. If the direction lands
well here, we'll firm up designs on the individual sub-issues (and RFCs
if engine/format surfaces are touched).

The longer arc is:

• Agents that assist with operating tables — tuning, inspection, maintenance — reducing ops and tuning complexity.
• Context is as important to AI inference as the model itself. Hudi tables support for unstructured data, vector search index - can serve any data at inference time.
• Emitting traces back to Hudi tables from agent-gateway can help eval, fine-tuning.
• Traces also help the agent optimize the layout of tables, build indexes and such.

[vinothchandar]

Sharing a screenshot and the proposed user interaction diagram

[bhasudha]

This is pretty awesome. Thanks @vinothchandar. +1 on the proposal.

[vinishjail97]

This is a really exciting direction. The self-hostable, local-inference-first design is what stands out most to me — being able to deploy open-source LLM models (via Ollama out of the box, and vLLM for higher-throughput serving) means teams can bring agentic capabilities to their lakehouse without shipping data or queries off to a third-party API provider. That's a big deal for anyone with data-residency, privacy, or cost constraints.

+1 on the direction.

[rahil-c]

@vinothchandar

+1

This is really cool and glad to see hudi evolving beyond the storage format and what its been focused on which more as a lakehouse platform offering compute, storage, and now integration with agents within this new AI era. I do have some questions though since I have been thinking about this "gateway" concept, and happy to discuss them in community sync or on the dev list if needed.

1. How does the agent/model find the correct underlying table based on the user question?
2. How does the agent/model load into its window only the relevant part of the underlying table to answer the original user question?

[vinothchandar]

Thanks.

On 1 : We may need to integrate it with more tools for ontology and semantics. TBD.
On 2: The loading of data itself is driven by tool calls. 1M tokens is not much to hold a lakehouse table. In practice, most models only retail parts of the result as they deem relevant for the session. This is orthogonal to our discussion - IIUC its a model concern.

[danny0405]

+1, a nice move!

[cshuo]

+1 on the direction — especially the self-hostable, local-inference-first approach.

One concern is how we ensure reliable and governed table discovery and context selection. Maybe the gateway should avoid relying on the model to freely infer the target table or pull broad data into its context; instead, it should be grounded in catalog/table metadata and execute narrowly scoped, read-only queries. This is also important for access control and auditability as the scope expands from querying to table operations.

[vinothchandar]

good points. text-to-sql accuracy is currently ~70% or so, drops on real data. A bunch of them are open-ended problems. and my points above around making evals and trace data collection - should help users fine-tune the accuracy over time.

In other words, those are the roadmap items, once we release a v1. These are challenges we need to solve. But we are starting to build this alongside the models and ai tools, instead of waiting for maturity.