kerniq-ai: a framework-agnostic AI library for Rust (exploring, feedback wanted)

[kentmarete]

We're exploring kerniq-ai: a standalone library for building AI-powered Rust applications through one consistent API, without having to learn the details of every model provider, vector database or protocol.

It's at the design stage. There's no code yet, and nothing here is a commitment. We'd rather hear what you need before we write the API.

What it would cover

• Model APIs: chat and completion across providers behind one interface.
• Chat client: conversations, streaming and structured output.
• Embeddings and vector stores: storing and searching embeddings for retrieval-augmented generation (RAG).
• Tool calling: expose Rust functions to models as tools.
• MCP: connect to and serve tools over the Model Context Protocol.

Not tied to Kerniq Server

kerniq-ai is planned as a separate project. It should work with any Rust web framework, including Kerniq, Axum, Actix Web, Rocket and Poem, or with no web framework at all. Kerniq will provide a first-class integration.

What would help most

1. Which model providers and vector stores do you need first?
2. Would you use MCP mainly as a client, a server, or both?
3. What do you use for this today, and what would you want done differently?

Reply below, or just 👍 this post if you'd use it. See also the roadmap.

[ashishmishra26]

This looks like a great initiative. Having a unified Rust API for these components would be incredibly useful for the ecosystem.

Since you're gathering feedback on which vector stores and protocols to support for RAG, I'd suggest taking a look at Infino. It's an open-source retrieval engine that runs embedded (in-process, similar to SQLite), which pairs nicely with standalone Rust applications by removing the need to manage a separate search cluster.

A few reasons it might be a strong candidate for kerniq-ai:

• Rust Core: The engine itself is written in Rust, which should make native integration straightforward.
• Embedded RAG: It supports vector kNN, full-text (BM25), and hybrid search over data stored as Parquet (local or object storage).
• MCP Alignment: We also support the Model Context Protocol (via an MCP server), so there's good overlap with your planned MCP tooling.

Would be happy to provide feedback or help out if you decide to explore adding it to the supported vector stores.

Disclosure: I work on Infino.
https://infino.ai