Move Microsoft.Extensions.VectorData.Abstractions over from Semantic Kernel

[roji]

This moves Microsoft.Extensions.VectorData.Abstractions from the Semantic Kernel repo (here) to the dotnet/extensions repo.

Note: this introduces two test projects:

• Microsoft.Extensions.VectorData.Abstractions.Tests - classical uninteresting unit tests for types in Microsoft.Extensions.VectorData.Abstractions
• Microsoft.Extensions.VectorData.ConformanceTests - abstract base classes for tests that MEVD providers should extend; this ensures that a provider is well-implemented and conforms to the expected behavior.
  • This isn't a test project, in the sense that we don't run it (it's just abstract base classes).
  • We need to publish this as a nuget, so that providers can reference it and implement - I might need a bit of help figuring that part out.

Microsoft Reviewers: Open in CodeFlow.

[Copilot]

Pull request overview

This PR ports Microsoft.Extensions.VectorData.Abstractions (and related conformance + unit tests) from the Semantic Kernel repo into dotnet/extensions, including build/packaging wiring for dependencies and test infrastructure.

Changes:

• Adds the Microsoft.Extensions.VectorData.Abstractions library source (vector store + vector search abstractions, provider services, and utilities).
• Introduces a new Microsoft.Extensions.VectorData.ConformanceTests test project and supporting fixtures to validate provider behavior.
• Updates repository package-version props to include required centralized versions (e.g., Microsoft.Extensions.AI.Abstractions, xunit).

Reviewed changes

Copilot reviewed 78 out of 78 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/TypeTests/KeyTypeTests.cs	Adds conformance tests for supported key types and auto-generation behavior.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/TypeTests/EmbeddingTypeTests.cs	Adds conformance tests for supported embedding/vector CLR types and embedding generation scenarios.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/TestSuiteImplementationTests.cs	Ensures provider test suites implement all base test classes (or explicitly ignore them).
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/VectorStoreFixture.cs	Adds base fixture for vector store lifecycle + key generation.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/VectorStoreCollectionFixtureBase.cs	Adds base fixture for collection setup + seeding.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/VectorStoreCollectionFixture.cs	Adds reseeding logic for typed collections.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/TestStore.cs	Adds provider-overrideable store abstraction + wait-for-indexing helper.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/TestRecord.cs	Adds shared record base with [VectorStoreKey].
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Support/DynamicVectorStoreCollectionFixture.cs	Adds reseeding support for dynamic-mapped collections.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/ModelTests/NoVectorModelTests.cs	Adds model tests for providers supporting models without vectors.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/ModelTests/NoDataModelTests.cs	Adds model tests for key+vector-only records.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/ModelTests/MultiVectorModelTests.cs	Adds tests for multi-vector models and required vector-property selection.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/ModelTests/DynamicModelTests.cs	Adds dynamic-mapping CRUD/search tests.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/Microsoft.Extensions.VectorData.ConformanceTests.csproj	Introduces the conformance test project and references required packages/projects.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/IndexKindTests.cs	Adds conformance coverage for index-kind support.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/HybridSearchTests.cs	Adds hybrid search conformance tests and fixtures.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/DistanceFunctionTests.cs	Adds conformance tests for distance function behavior and score threshold support.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/DependencyInjectionTests.cs	Adds DI registration conformance tests for stores/collections and embedding generator resolution.
test/Libraries/Microsoft.Extensions.VectorData.ConformanceTests/CollectionManagementTests.cs	Adds conformance tests around collection management and metadata service exposure.
test/Libraries/Microsoft.Extensions.VectorData.Abstractions.Tests/PropertyModelTests.cs	Adds unit tests for nullability detection behavior in provider property models.
test/Libraries/Microsoft.Extensions.VectorData.Abstractions.Tests/Microsoft.Extensions.VectorData.Abstractions.Tests.csproj	Adds a unit test project for the abstractions library.
test/.editorconfig	Adds a C# preference for static anonymous functions (duplicated entry currently).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStoreMetadata.cs	Adds vector store metadata shape for GetService scenarios.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStoreException.cs	Adds common exception type with store/collection/operation metadata.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStoreCollectionOptions.cs	Adds common options base for collections (definition + embedding generator).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStoreCollectionMetadata.cs	Adds collection metadata shape for GetService scenarios.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStoreCollection.cs	Adds core collection abstraction for CRUD + search operations.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorStorage/VectorStore.cs	Adds core store abstraction for collections + management APIs.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorSearch/VectorSearchResult.cs	Adds search-result record+score container.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorSearch/VectorSearchOptions.cs	Adds vector-search options (filtering, skip, vector selection, score threshold, include vectors).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorSearch/IVectorSearchable.cs	Adds searchable interface for vector similarity search.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorSearch/IKeywordHybridSearchable.cs	Adds hybrid-search interface (dense vector + keyword search).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/VectorSearch/HybridSearchOptions.cs	Adds hybrid-search options (filtering, skip, property selection, include vectors, score threshold).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/Utilities/UnreachableException.cs	Adds UnreachableException polyfill for older TFMs.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordOptions/RecordRetrievalOptions.cs	Adds options for single/batch record retrieval.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordOptions/FilteredRecordRetrievalOptions.cs	Adds options for filtered Get (skip/order/include vectors) and order-by builder types.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreVectorProperty{TInput}.cs	Adds generic vector property definition for custom embedding-generation input types.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreVectorProperty.cs	Adds vector property definition (dimensions, index kind, distance function, generator, embedding type).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreProperty.cs	Adds base definition for record properties with provider annotations.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreKeyProperty.cs	Adds key property definition (type + auto-generation).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreDataProperty.cs	Adds data property definition (indexed/full-text indexed).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/VectorStoreCollectionDefinition.cs	Adds collection schema definition (properties + default generator).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/IndexKind.cs	Adds well-known index-kind constants.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordDefinition/DistanceFunction.cs	Adds well-known distance-function constants.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordAttributes/VectorStoreVectorAttribute.cs	Adds attribute for vector properties (dimensions/index/distance/storage name).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordAttributes/VectorStoreKeyAttribute.cs	Adds attribute for key properties (storage name + auto-generation).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/RecordAttributes/VectorStoreDataAttribute.cs	Adds attribute for data properties (index/full-text index/storage name).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/README.md	Adds package README and links to conceptual docs.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/VectorPropertyModel{TInput}.cs	Adds provider model support for embedding generation with custom input type.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/VectorPropertyModel.cs	Adds provider model support for vector properties, embedding dispatch, and resolution.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/VectorDataStrings.cs	Adds shared provider error-string helpers.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/PropertyModel.cs	Adds base provider property model with POCO/dynamic accessors and nullability detection.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/KeyPropertyModel.cs	Adds provider key-property model.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/IRecordCreator.cs	Adds internal record factory interface for provider implementations.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/Filter/QueryParameterExpression.cs	Adds expression node for parameterized captured values during filter translation.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/Filter/FilterTranslatorBase.cs	Adds base for filter translators, including preprocessing and Contains-pattern support.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/Filter/FilterPreprocessingOptions.cs	Adds options for preprocessing filter expression trees.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/EmbeddingGenerationDispatcher{TEmbedding}.cs	Adds typed dispatcher for embedding generation and embedding type resolution.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/EmbeddingGenerationDispatcher.cs	Adds base dispatcher abstraction/factory for embedding generation.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/DataPropertyModel.cs	Adds provider data-property model.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/CollectionModelBuildingOptions.cs	Adds model-building feature flags (multi-vector, serializer behavior, reserved key name).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/CollectionModel.cs	Adds built model container and helpers for property selection/validation.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/ProviderServices/CollectionJsonModelBuilder.cs	Adds JSON-specific model-building customization hooks.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/Microsoft.Extensions.VectorData.Abstractions.csproj	Introduces the new library project and its TFM/dependency configuration.
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/FilterClauses/FilterClause.cs	Adds obsolete filter clause base type (compat layer).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/FilterClauses/EqualToFilterClause.cs	Adds obsolete equality filter clause type (compat layer).
src/Libraries/Microsoft.Extensions.VectorData.Abstractions/FilterClauses/AnyTagEqualToFilterClause.cs	Adds obsolete tag-contains filter clause type (compat layer).
eng/packages/Tests.props	Adds centralized xunit package version entry.
eng/packages/General-net9.props	Adds centralized Microsoft.Extensions.AI.Abstractions version entry for net9.
eng/packages/General-net10.props	Adds centralized Microsoft.Extensions.AI.Abstractions version entry for net10.
eng/packages/General-LTS.props	Adds centralized Microsoft.Extensions.AI.Abstractions version entry for LTS.
eng/Versions.props	Adds Microsoft.Extensions.AI.Abstractions version properties (net9/net10/LTS).

[adamsitnik]

@roji overall it LGTM (I've seen this code more than once), but I've asked some questions regarding having the source of MEVD in this repo and also referencing MEVD by other projects in this repo at the same time

[roji]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[roji]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[adamsitnik]

LGTM, thank you @roji !

[roji]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).