Embeddings onnxtextembedder embedbatch

OnnxTextEmbedder.EmbedBatch

A vector per text, in one session run.

public float[][] EmbedBatch(IEnumerable<string> texts, EncodingOptions options = null, CancellationToken cancellationToken = default)
public float[][] EmbedBatch(IEnumerable<string> texts, BatchEncoder encoder, CancellationToken cancellationToken = default)
public float[][] EmbedBatch(EncodedBatch batch, CancellationToken cancellationToken = default)

Parameters — texts are the strings to embed. options tunes the encoding — padding, truncation, the maximum length. encoder is a BatchEncoder you have already configured, for when the model's tokenizer is not the default. batch is an EncodedBatch you encoded yourself. cancellationToken abandons the run.

Returns — float[][], one vector of Dimension per input text, in input order.

Exceptions — InvalidOperationException from the overload taking only texts, when the embedder was built without a tokenizer: the other two overloads are the way to supply one. OperationCanceledException when cancellationToken is already cancelled, or is cancelled between sub-batches. ObjectDisposedException after Dispose, for the reason Embed gives.

Example — the shortest path from strings to vectors.

using Lodestar.Embeddings.Onnx;

using var embedder = new OnnxTextEmbedder("model.onnx");

float[][] vectors = embedder.EmbedBatch(
[
    "the cat sat on the mat",
    "a dog lay on the rug",
]);

Remarks — three overloads for one operation, and the choice is about who owns the tokenizer. The first owns it for you and is right when the model ships a standard vocabulary. The second takes an encoder you built, for a model whose tokenizer differs. The third takes an already-encoded batch, for when encoding happened elsewhere — on another thread, or once for several models.

Batching is not merely convenient. A session run has a fixed cost, so embedding a hundred texts one at a time costs a hundred of those; the vectors are identical either way, and the time is not.

Order is preserved, so the nth vector belongs to the nth text however the batch was padded internally.

Applies to — net10.0, netstandard2.0.

See also — OnnxTextEmbedder.Embed, BatchEncoder, EncodedBatch.