feat: support server side auto embedding

[Mini256]

After this PR, server-side auto embedding will be enabled by default for EmbeddingFunction. Embeddings will be computed automatically on the database side.

Currently, server-side auto embedding has a few limitations:

• Function-level API key configuration is not supported, only support global api key config.
• Image input (multimodal) is not supported for embedding

Usage Example

By default, EmbeddingFunction uses server-side automatic embedding:

from pytidb.embeddings import EmbeddingFunction

text_embed = EmbeddingFunction(
    "openai/text-embedding-3-small",
    # use_server=True (default)
)

However, TiDB Serverless currently does not support multimodal inputs, so for multimodal embedding models, client-side embedding should be used instead.

To indicate that the embedding model supports multimodal input, set multimodal=True:

image_embed = EmbeddingFunction(
    "jina_ai/jina-embedding-v4",
    multimodal=True
)

Example Integration with TableModel:

from app.db import tidb_client
from pytidb.embeddings import EmbeddingFunction
from pytidb.schema import TableModel, Field

# Set API key globally
tidb_client.configure_embedding_provider("openai", os.getenv("OPENAI_API_KEY"))

# Define embedding function
text_embed = EmbeddingFunction("openai/text-embedding-3-small")

# Define table schema with auto embedding config.
class Chunk(TableModel):
    id: int = Field(primary_key=True)
    text: str = Field()
    text_vec: Optional[list[float]] = text_embed.VectorField(source_field="text")

# Create table
tbl = tidb_client.create_table(schema=Chunk, if_exists="overwrite")

# Insert data
tbl.insert(Chunk(id=1, text="foo"))

# Search
results = tbl.search("bar").limit(1).to_pydantic(with_score=True)

[cursor]

Bugbot free trial expires on July 29, 2025
Learn more in the Cursor dashboard.

[cursor]

🚨 Bugbot Trial Expired

Your Bugbot trial has expired. Please purchase a license in the Cursor dashboard to continue using Bugbot.

[cursor]

Bugbot found 3 bugs

To see them, activate your membership in your Cursor dashboard.

[Icemap]

LGTM

[breezewish]

Just curious, for this example, what will happen if I don't keep the text_embed global instance at all?

For example, by using like this:

from app.db import tidb_client
from pytidb.embeddings import EmbeddingFunction
from pytidb.schema import TableModel, Field

# Set API key globally
tidb_client.configure_embedding_provider("openai", os.getenv("OPENAI_API_KEY"))

# Define table schema with auto embedding config.
class Chunk(TableModel):
    id: int = Field(primary_key=True)
    text: str = Field()
    text_vec: Optional[list[float]] = EmbeddingFunction("openai/text-embedding-3-small").VectorField(source_field="text")

# Create table
tbl = tidb_client.create_table(schema=Chunk, if_exists="overwrite")

# Insert data
tbl.insert(Chunk(id=1, text="foo"))

# Search
results = tbl.search("bar").limit(1).to_pydantic(with_score=True)

Will there be any downsides? Seems like in this way the EmbeddingFunction could more bounded with the Schema.

Also, when user wants two tables, do they need to define two embedding functions or they can reuse one?