Skip to content

Conversation

@afoucret
Copy link
Contributor

@afoucret afoucret commented Jul 11, 2025

Summary

Implements the TEXT_EMBEDDING function for ES|QL to generate dense vector embeddings from text using inference model.

Function Signature: TEXT_EMBEDDING(text: string, inference_id: string) -> dense_vector

Example Usage:

FROM documents 
| WHERE KNN(embedding_field, TEXT_EMBEDDING(content, "my-embedding-deployment"), 10)

Fixes #131022

Implementation Status

Completed in this PR:

  • TEXT_EMBEDDING_FUNCTION capability with snapshot build gating

  • Core Function Infrastructure

    • TextEmbedding function class with proper type validation and serialization
    • InferenceFunction interface for inference-based functions
    • Function registration in EsqlFunctionRegistry
  • Analysis of the inference function (validate existence and type of the inference endpoint)

    • Refactored pre-analysis, so it is able to collect inference ids form both Inference plans and inference function
    • Added validation for inference function in the analysis
  • Documentation generated from the annotations

🚧 TODO (Before Merge):

  • Add a pre-optimizer async phase to the ES|QL query execution
  • Execute the inference in the pre-optimizer
  • Integration tests and end-to-end validation

Notes

The function is enabled only in snapshot builds.
TEXT_EMBEDDING function is tracked into #131022

@github-actions
Copy link
Contributor

github-actions bot commented Jul 11, 2025

🔍 Preview links for changed docs

@afoucret afoucret closed this Jul 11, 2025
@afoucret afoucret deleted the esql-text-embedding-function branch July 11, 2025 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ES|QL: Add TEXT_EMBEDDING function

3 participants