Skip to content

v1.16.12-test

Pre-release
Pre-release

Choose a tag to compare

View all tags
@kkampli-singlestore kkampli-singlestore released this 22 Jul 13:27
· 1 commit to users/kaushik/embed-ctx-length-default since this release
v1.16.12-test
6def72b 
embeddings: restore self-chunking for non-OpenAI models

Re-add _ChunkedOpenAIEmbeddings: for non-Azure models (e.g. Qwen on 'Nova'), send raw
text (check_embedding_ctx_length=False) and split long inputs into character-bounded
chunks, embedding each and length-weighted-averaging into one vector per input, so long
texts never hit the server's context limit. Azure keeps tiktoken (True).

Co-authored-by: Cursor <cursoragent@cursor.com>
Assets 10
Loading