[MEVD] Can't use SearchAsync with embedding generator when the vector property is ReadOnlyMemory<float>

[enyim]

Describe the bug
Exception in InMemoryVectorStore when using SearchAsync

An exception of type 'System.InvalidOperationException' occurred in System.Private.CoreLib.dll but was not handled in user code: ''SearchAsync' requires an embedding generator to be configured. To pass an embedding directly, use 'SearchEmbeddingAsync', otherwise configure an embedding generator with your vector store connector.'
   at Microsoft.SemanticKernel.Connectors.InMemory.InMemoryVectorStoreRecordCollection`2.<SearchAsync>d__23`1.MoveNext()
   at Microsoft.SemanticKernel.Connectors.InMemory.InMemoryVectorStoreRecordCollection`2.<SearchAsync>d__23`1.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(Int16 token)

To Reproduce

Using the sample as basis from https://devblogs.microsoft.com/dotnet/introducing-microsoft-extensions-vector-data/ (slightly updated to fix obsoletions)

Source is attached.

Steps to reproduce the behavior:

1. Using latest packages

<PackageReference Include="Microsoft.Extensions.AI.Ollama" Version="9.4.3-preview.1.25230.7" />
<PackageReference Include="Microsoft.Extensions.VectorData.Abstractions" Version="9.0.0-preview.1.25229.1" />
<PackageReference Include="Microsoft.SemanticKernel.Connectors.InMemory" Version="1.48.0-preview" />

2. Set up the collection using a default EmbeddingGenerator

var generator = new OllamaEmbeddingGenerator(new Uri("http://localhost:11434/"), "all-minilm");
var vectorStore = new InMemoryVectorStore(new InMemoryVectorStoreOptions { EmbeddingGenerator = generator });
var movies = vectorStore.GetCollection<int, Movie>("movies");
await movies.CreateCollectionIfNotExistsAsync();

3. Search throws; also Upsert does not generate embeddings

Expected behavior
Store should us the EmbeddingGenerator passed to the constructor. Looking at the source it seems it's checking if the Vector* attributes on the POCO properties have the generator specified, but does not fallback to default.

Platform

• Language:C# .NET 9

source.zip

[chancezheng]

Use

Microsoft.SemanticKernel.Connectors.Sqlite.SqliteVectorStore v1.48.0 error

Microsoft.SemanticKernel.Connectors.Sqlite.SqliteMemoryStore ok, but it is being obsoleted

Bug

The provided vector type System.Runtime.CompilerServices.AsyncTaskMethodBuilder1+AsyncStateMachineBox1[[System.ReadOnlyMemory1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.SemanticKernel.Embeddings.EmbeddingGenerationExtensions+<GenerateEmbeddingAsync>d__22[[System.String, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.SemanticKernel.Abstractions, Version=1.48.0.0, Culture=neutral, PublicKeyToken=f300afd708cefcd3]] is not supported by the SQLite connector. Supported types are: System.ReadOnlyMemory1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Nullable1[[System.ReadOnlyMemory`1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]

[westey-m]

@enyim, Thanks for raising this.
To generate embeddings on upsert you have to modify your model like this:

    public class Movie
    {
        [VectorStoreRecordKey]
        public int Key { get; set; }

        [VectorStoreRecordData]
        public string Title { get; set; }

        [VectorStoreRecordData]
        public string Description { get; set; }

        [VectorStoreRecordVector(384, DistanceFunction = DistanceFunction.CosineSimilarity)]
        public string Vector => this.Description;
    }

Note that here Vector is just a string that is the same as description. On upsert the configured embedding generator will be used to generate an embedding from the string value of Vector and save the embedding in the database.

        var vectorStore = new InMemoryVectorStore(new InMemoryVectorStoreOptions { EmbeddingGenerator = textEmbeddingGenerator });
        var movies = vectorStore.GetCollection<int, Movie>("movies");
        await movies.CreateCollectionIfNotExistsAsync();

        foreach (var movie in movieData)
        {
            // embedings are generated automatically
            await movies.UpsertAsync(movie);
        }

        var results = movies.SearchAsync("A family friendly movie", 1);
        await foreach (var result in results)
        {
            Console.WriteLine($"Title: {result.Record.Title}");
            Console.WriteLine($"Description: {result.Record.Description}");
        }

When updating the model this way, you should also not see the error message on search anymore.
Some more documentation on this is also available here: https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/embedding-generation?pivots=programming-language-csharp#letting-the-vector-store-generate-embeddings

@roji, I do believe we have a bug here, in that when the vector on the data model does not have a vector type we do not fall back to the configured embedding generator for search.

[westey-m]

Use

Microsoft.SemanticKernel.Connectors.Sqlite.SqliteVectorStore v1.48.0 error

Microsoft.SemanticKernel.Connectors.Sqlite.SqliteMemoryStore ok, but it is being obsoleted

Bug

The provided vector type System.Runtime.CompilerServices.AsyncTaskMethodBuilder1+AsyncStateMachineBox1[[System.ReadOnlyMemory1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Microsoft.SemanticKernel.Embeddings.EmbeddingGenerationExtensions+<GenerateEmbeddingAsync>d__22[[System.String, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Microsoft.SemanticKernel.Abstractions, Version=1.48.0.0, Culture=neutral, PublicKeyToken=f300afd708cefcd3]] is not supported by the SQLite connector. Supported types are: System.ReadOnlyMemory1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Nullable1[[System.ReadOnlyMemory`1[[System.Single, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]

@chancezheng, thanks for reporting this issue. Can you please provide a small code snippet we can use to reproduce the issue you are seeing?

[chancezheng]

@westey-m

the following two ways are failed, but SqliteMemoryStore ok

SqliteMemoryCore ok code:

                try
                {
                    var recordsToSave = new List<MemoryRecord>();
                    for (int i = 0; i < document.ChunkCount; i++)
                    {
                        var chunk = document.Chunks[i];
                        var memoryRecord = new MemoryRecord(
                            metadata: new MemoryRecordMetadata(
                                isReference: false, // 这个 chunk 不是外部引用
                                id: chunk.Id,
                                text: chunk.Text, // 原始文本块
                                description: $"文档块 {i + 1}", // 可选描述
                                externalSourceName: document.FileName, // 可选：来源文件名
                                additionalMetadata: $"Chunk Index: {i}" // 可选：其他元数据
                            ),
                            embedding: new ReadOnlyMemory<float>(chunk.Embedding.ToArray()), // 嵌入向量
                            key: document.Id,
                            timestamp: document.ProcessedTime
                        );

                        recordsToSave.Add(memoryRecord);
                        Console.WriteLine($"准备保存记录 ID: {memoryRecord.Key}, 文本片段: \"{chunk.Text.Substring(0, Math.Min(chunk.Text.Length, 50))}...\"");
                    }

                    Console.WriteLine($"开始批量保存 {recordsToSave.Count} 条记录到集合 {MemoryCollectionName}...");
                    var upsertedKeys = _memoryStore.UpsertBatchAsync(MemoryCollectionName, recordsToSave, cancellationToken);
                    List<string> successfulKeys = await upsertedKeys.ToListAsync();
                    Console.WriteLine("记录保存成功!");
                }
                catch (Exception ex)
                {
                    Console.WriteLine($"保存记录时发生错误: {ex.Message}");
                }

SqliteVectorStore error code:

var vectorStore = SqliteVectorStore($"Data Source={dbPath};Cache=Shared;Foreign Keys=True;", new SqliteVectorStoreOptions { EmbeddingGenerator = embeddingGenerator });
var collection = vectorStore.GetCollection<string, DocumentChunk>(MemoryCollectionName);
await collection.CreateCollectionIfNotExistsAsync(cancellationToken);

/// <summary>
/// 文档块类，表示文档内容的一个片段
/// </summary>
public class DocumentChunk
{
    /// <summary>
    /// 文档块ID
    /// </summary>
    [VectorStoreRecordKey]
    public string Id { get; set; } = "chunk-" + Guid.NewGuid().ToString();

    /// <summary>
    /// 所属文档ID
    /// </summary>
    [VectorStoreRecordData(IsIndexed = true)]
    public string DocumentId { get; set; }

    /// <summary>
    /// 文档名称
    /// </summary>
    [VectorStoreRecordData(IsIndexed = true)]
    public string DocumentName { get; set; }

    /// <summary>
    /// 文档块在文档中的索引
    /// </summary>
    [VectorStoreRecordData(IsIndexed = true)]
    public int Index { get; set; }

    /// <summary>
    /// 文档块文本内容
    /// </summary>
    [VectorStoreRecordData(IsFullTextIndexed = true)]
    public string Text { get; set; }

    /// <summary>
    /// 文档块包含的token数量
    /// </summary>
    [VectorStoreRecordData(IsIndexed = true)]
    public int TokenCount { get; set; }

    /// <summary>
    /// 文档块的向量嵌入
    /// </summary>
    [VectorStoreRecordVector(1536, DistanceFunction = DistanceFunction.CosineSimilarity)]
    public ReadOnlyMemory<float> Embedding { get; set; }

    /// <summary>
    /// 文档块元数据字典，可用于存储与检索相关的各种附加信息
    /// </summary>
    public Dictionary<string, object> Metadata { get; set; } = new Dictionary<string, object>();

    /// <summary>
    /// 文档块类型（段落、标题、列表项等）
    /// </summary>
    public string ChunkType { get; set; } = "paragraph";

    /// <summary>
    /// 文档块所在页码（如果适用）
    /// </summary>
    public int? PageNumber { get; set; }

    /// <summary>
    /// 文档块所在章节
    /// </summary>
    public string Section { get; set; }

    /// <summary>
    /// 创建时间
    /// </summary>
    public DateTime CreatedAt { get; set; } = DateTime.Now;

    /// <summary>
    /// 添加元数据
    /// </summary>
    /// <param name="key">元数据键</param>
    /// <param name="value">元数据值</param>
    public void AddMetadata(string key, object value)
    {
        if (string.IsNullOrEmpty(key))
        {
            throw new ArgumentException("元数据键不能为空", nameof(key));
        }

        Metadata[key] = value;
    }

    /// <summary>
    /// 获取特定元数据
    /// </summary>
    /// <param name="key">元数据键</param>
    /// <returns>元数据值，如不存在则返回null</returns>
    public object GetMetadata(string key)
    {
        if (string.IsNullOrEmpty(key) || !Metadata.ContainsKey(key))
        {
            return null;
        }

        return Metadata[key];
    }

    /// <summary>
    /// 尝试获取特定类型的元数据
    /// </summary>
    /// <typeparam name="T">目标类型</typeparam>
    /// <param name="key">元数据键</param>
    /// <param name="value">输出值</param>
    /// <returns>是否成功获取并转换</returns>
    public bool TryGetMetadata<T>(string key, out T value)
    {
        value = default;

        if (string.IsNullOrEmpty(key) || !Metadata.ContainsKey(key))
        {
            return false;
        }

        try
        {
            if (Metadata[key] is T typedValue)
            {
                value = typedValue;
                return true;
            }
            return false;
        }
        catch
        {
            return false;
        }
    }
}

var collection = new SqliteVectorStoreRecordCollection<string, DocumentProcessorResult>(_connectionString, MemoryCollectionName);
//await collection.UpsertAsync(document, cancellationToken);
await collection.CreateCollectionIfNotExistsAsync(cancellationToken);

        /// <summary>
        /// 文档ID
        /// </summary>
        [VectorStoreRecordKey]
        public string Id { get; set; } = Guid.NewGuid().ToString();

        /// <summary>
        /// 文件名
        /// </summary>
        [VectorStoreRecordData(IsIndexed = true)]
        public string FileName { get; set; } = string.Empty;

        /// <summary>
        /// 文件大小
        /// </summary>
        [VectorStoreRecordData(IsIndexed = true)]
        public long FileSize { get; set; }

        /// <summary>
        /// 文件扩展名
        /// </summary>
        [VectorStoreRecordData(IsIndexed = true)]
        public string FileExtension { get; set; } = string.Empty;

        /// <summary>
        /// 文件类型
        /// </summary>
        [VectorStoreRecordData(IsIndexed = true)]
        public string FileType => FileExtension.TrimStart('.');

        /// <summary>
        /// 文本内容
        /// </summary>
        [VectorStoreRecordData(IsFullTextIndexed = true)]
        public string Content { get; set; } = string.Empty;

        /// <summary>
        /// 处理时间
        /// </summary>
        public DateTime ProcessedTime { get; set; } = DateTime.Now;

        /// <summary>
        /// 文档块数量
        /// </summary>
        [VectorStoreRecordData(IsIndexed = true)]
        public int ChunkCount { get; set; }

        /// <summary>
        /// 文档块列表
        /// </summary>
        //[VectorStoreRecordVector(Dimensions: 4, DistanceFunction = DistanceFunction.CosineSimilarity, IndexKind = IndexKind.Hnsw)]
        public List<DocumentChunk> Chunks { get; set; } = new List<DocumentChunk>();

        //[VectorStoreRecordVector(Dimensions: 1536, DistanceFunction = DistanceFunction.CosineSimilarity, IndexKind = IndexKind.Hnsw)]
        public ReadOnlyMemory<float>? Embedding { get; set; }

        /// <summary>
        /// 文档元数据
        /// </summary>
        public Dictionary<string, object> Metadata { get; set; } = new Dictionary<string, object>();
        
        /// <summary>
        /// 添加元数据
        /// </summary>
        /// <param name="key">元数据键</param>
        /// <param name="value">元数据值</param>
        public void AddMetadata(string key, object value)
        {
            if (string.IsNullOrEmpty(key))
            {
                throw new ArgumentException("元数据键不能为空", nameof(key));
            }
            
            Metadata[key] = value;
        }
        
        /// <summary>
        /// 获取特定元数据
        /// </summary>
        /// <param name="key">元数据键</param>
        /// <returns>元数据值，如不存在则返回null</returns>
        public object GetMetadata(string key)
        {
            if (string.IsNullOrEmpty(key) || !Metadata.ContainsKey(key))
            {
                return null;
            }
            
            return Metadata[key];
        }
        
        /// <summary>
        /// 尝试获取特定类型的元数据
        /// </summary>
        /// <typeparam name="T">目标类型</typeparam>
        /// <param name="key">元数据键</param>
        /// <param name="value">输出值</param>
        /// <returns>是否成功获取并转换</returns>
        public bool TryGetMetadata<T>(string key, out T value)
        {
            value = default;
            
            if (string.IsNullOrEmpty(key) || !Metadata.ContainsKey(key))
            {
                return false;
            }
            
            try
            {
                if (Metadata[key] is T typedValue)
                {
                    value = typedValue;
                    return true;
                }
                return false;
            }
            catch
            {
                return false;
            }
        }
    }

error info:

SQLite Error 1: 'The specified module cannot be found。'.
在 Microsoft.Data.Sqlite.SqliteConnection.LoadExtensionCore(String file, String proc)
在 Microsoft.Data.Sqlite.SqliteConnection.LoadExtension(String file, String proc)
在 Microsoft.SemanticKernel.Connectors.Sqlite.SqliteVectorStoreRecordCollection2.<GetConnectionAsync>d__31.MoveNext() 在 System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable1.ConfiguredValueTaskAwaiter.GetResult()
在 Microsoft.SemanticKernel.Connectors.Sqlite.SqliteVectorStoreRecordCollection`2.d__17.MoveNext()
在 LLMFramework.Core.DocumentProcessing.VectorDatabaseService.d__18.MoveNext() 在 C:\Users\Chance\Desktop\LLMApp\LLMFramework.Core\DocumentProcessing\VectorDatabaseService.cs

[westey-m]

@chancezheng, the 'The specified module cannot be found' error you are seeing is because sqlite doesn't support vector search out of the box, but requires an extension 'sqlite-vec' to be setup on your machine.
Check out the limitations section of our Sqlite connector documentation page:
https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/sqlite-connector?pivots=programming-language-csharp#limitations

Also check out the extension's github repo here: https://github.com/asg017/sqlite-vec

The reason why it is not failing with the old MemoryStore implementation, is because MemoryStore downloads all records from the database for each search and does an in memory vector comparison. This is obviously not scalable and not production quality, and why it is marked as obsolete.