This pull request introduces semantic search capabilities to the No. Series Copilot, significantly improving its ability to understand user requests and identify relevant number series configurations.

Key Enhancements:

• Semantic Search Implementation: Introduced a new codeunit, No. Series Cop. Semantic Impl. (NoSeriesCopilot/src/Copilot/Search/Semantic/NoSeriesCopSemanticImpl.Codeunit.al), dedicated to handling semantic comparisons using vector embeddings. This includes functions like IsRelevant, CalculateSemanticNearness, and GetVectorFromText.
• Semantic Vocabulary Storage: Added the No. Series Semantic Vocabulary table (NoSeriesCopilot/src/Copilot/Search/Semantic/NoSeriesSemanticVocabulary.Table.al) as a temporary solution to store text payloads and their corresponding vector embeddings (InsertRecord, SaveVectorText, LoadVectorText). This table serves three main purposes:
  1. Stores embeddings for setup context: Caches embeddings for strings composed of setup table names and number series field names (e.g., RemoveTextPart(TableMetadata.Name, '& ') + ' ' + Field.FieldName).
  2. Stores embeddings for user entities: Caches embeddings for key entities (like table or area names, e.g., 'product', 'sales') extracted from the user's request by the LLM.
  3. Performance Optimization: Acts as a cache (GetEmbeddingsFromVocabulary) to avoid redundant calls to the Azure OpenAI embedding endpoint (GetAzureOpenAIEmbeddings) if an embedding for a given text already exists. New embeddings are temporarily stored and then persisted via UpdateSemanticVocabulary.
• Integration with Existing Tools: Updated intent handling codeunits (No. Series Cop. Add Intent, No. Series Cop. Change Intent) and the core tools implementation (No. Series Cop. Tools Impl.) to utilize semantic search. The ToolsImpl.IsRelevant function now orchestrates both classic text matching (No. Series Text Match Impl.) and semantic comparison (No. Series Cop. Semantic Impl.) when filtering setup tables and fields based on extracted user entities (see ListOnlyRelevantNoSeriesFields and ListOnlyRelevantNoSeriesFieldsWithExistingNumberSeries).
• Future Considerations: This implementation uses an AL table for vector storage as a temporary measure. This approach should be revisited when the Business Central platform introduces native support for vector types and operations. The core semantic comparison flow, however, is designed to remain largely the same.

Semantic Search Flow:

The semantic search process enhances the copilot's ability to match user intent with specific number series fields:

graph LR
    A[User asks to create/change No. Series<br>e.g., 'create number series for products'] --> B{LLM understands intent<br>Extracts key entities<br>e.g., 'product'};
    B --> C[System Search: Setup Tables & No. Series Fields<br>Function: ToolsImpl.RetrieveSetupTables];
    C --> D{Loop through Tables/Fields};
    D --> E{Compare Entity with Table+Field<br>Functions: ToolsImpl.IsRelevant,<br>TextMatchImpl.IsRelevant,<br>SemanticImpl.IsRelevant};
    E -- Classic Match? --> F[Match Found];
    E -- Semantic Match? --> F;
    E -- No Match --> D;
    F --> G{Ask LLM to generate No. Series<br>using matched context};

Comparison: Before vs. After Semantic Search

This section demonstrates the improved understanding and relevance achieved with semantic search compared to the previous implementation relying solely on classic text matching.

Example Request: create number series for products

Without Semantic Search (Classic Text Match Only):

With Semantic Search:

Additional Improvements:

• Suggestion Validation Fix: Corrected an issue in NoSeriesCopilotImpl.Codeunit.al where the intent check (if Intent = AddNoSeriesIntent.GetName() then) was not functioning correctly. This prevented the CheckIfExpectedNoSeriesCount validation from running, which compares the number of requested number series against the number actually generated by the LLM. This fix ensures more reliable validation and helps prevent incomplete suggestions.
• Improved Suggestion Filtering: Introduced logic using ExcludeExistingNoSeriesIfNewWhereGenerated to refine the final suggestions presented to the user. Semantic search can identify more existing, already-configured number series relevant to a query. To avoid confusion, if any new number series are generated based on the user's request, only these new suggestions will be displayed. Existing number series will only be shown if no new series were generated.

Summary:

This commit significantly enhances the No. Series Copilot by adding a semantic understanding layer. It lays the groundwork for more intuitive interactions and accurate results, focusing on semantic capabilities and future extensibility while providing a temporary mechanism for vector storage and retrieval. Temporary methods and storage solutions include clear TODOs for future refactoring once platform support is available.

Work Item(s)

Fixes #2107
Fixes AB#571185