Skip to content

Architecture: Semantic Intent Detection Subsystem

Jump to bottom
K. C. Ramakrishna edited this page Aug 16, 2025 · 2 revisions

Architecture: Semantic Intent Detection Subsystem (Simplified)

  • Status: Final
  • Author: KC + Gemini
  • Last Updated: August 9, 2025

1. Overview

This document outlines the architecture for the Semantic Intent Detection subsystem. The primary goal of this system is to allow end-users to interact with the application using natural language. Instead of clicking buttons, a user can type a query (e.g., "I need to bill client X"), and the system will intelligently map this query to the most relevant internal action (e.g., create_invoice).

This revised architecture prioritizes operational simplicity by removing the message broker. It leverages an in-app, asynchronous "fire-and-forget" mechanism for responsiveness, with a scheduled cron job acting as the primary mechanism for ensuring eventual consistency and reliability.

2. Goals and Non-Goals

Goals

  • High Accuracy: The system must accurately map user queries to the correct internal actions.
  • Real-time Performance: User search queries must return results nearly instantly (< 200ms).
  • API Responsiveness: The process of creating embeddings must not block the main API thread.
  • Simplified Infrastructure: The system should minimize the number of external dependencies and services to manage.
  • Guaranteed Eventual Consistency: While in-flight jobs may be lost on an application restart, a recovery mechanism must exist to ensure all labels are eventually processed.

Non-Goals

  • Building Custom ML Models: We will leverage pre-trained, open-source models.
  • Guaranteed Job Durability: Jobs triggered by the API are not persisted in a durable queue and will be lost if the application restarts before they complete.
  • Real-time Job Monitoring: There will be no centralized dashboard to view in-flight asynchronous tasks.

3. High-Level Architecture

The system uses a dedicated Python microservice for ML inference and a Java/Quarkus backend for core logic. Asynchronous tasks are handled within the Quarkus application's thread pool.

graph TB
    subgraph "Your B2B SaaS Platform"
        quarkus_app["Quarkus Backend<br/>Java, Main Business Logic, API, Async Task Execution"]
        fast_api_app["Embedding Service<br/>Python, FastAPI, Sentence Transformers"]
        es[("Elasticsearch<br/>Search & Analytics Engine")]
        mariadb[("MariaDB<br/>Primary Application Database")]
        keycloak["Keycloak<br/>Authentication Service"]
        user_browser["User's Browser<br/>Web Client"]
    end

    user_browser --> quarkus_app
    quarkus_app --> keycloak
    quarkus_app --> mariadb
    quarkus_app --> es
    quarkus_app --> fast_api_app

    %% Labels for relationships
    user_browser -.->|"Makes API calls"| quarkus_app
    quarkus_app -.->|"Validates JWT"| keycloak
    quarkus_app -.->|"Reads/Writes App Data & Job State"| mariadb
    quarkus_app -.->|"Performs kNN Search Queries & Indexes Data"| es
    quarkus_app -.->|"Requests Embeddings (via REST)"| fast_api_app
Loading

Component Responsibilities

  • Quarkus Backend: The main application. It handles all user API requests, business logic, database persistence, and manages the asynchronous execution of background tasks using an internal thread pool. It also runs the scheduled recovery job.
  • Embedding Service: A small, specialized Python/FastAPI microservice. Its sole responsibility is to generate vector embeddings from text.
  • Elasticsearch: The search engine. Stores action labels and their vector embeddings and performs high-speed kNN semantic search.
  • MariaDB: The primary "source of truth" database. It stores all application data, including the embeddings. The state of the embedding column (NULL or not) serves as the de-facto job queue for the recovery process.

4. Core Technologies

Technology Role Justification
Quarkus (Java) Main Application Backend & Task Executor Robust, high-performance framework. Its built-in concurrency tools (@Scheduled, async methods) will be used for background processing.
Python / FastAPI Embedding Microservice Access to the best-in-class, most performant ML libraries (sentence-transformers).
sentence-transformers ML Model Library State-of-the-art library for creating semantic embeddings. We will use all-MiniLM-L6-v2.
Elasticsearch Search Engine Industry standard for search, providing powerful kNN vector search capabilities.
MariaDB Primary Database & Job State The existing database. Its schema will be used to track which labels need processing.

5. Detailed Workflows

5.1 Real-time: Creating a New Label (In-App Asynchronous)

The API remains responsive by offloading the embedding process to a non-blocking, background thread within the application itself.

sequenceDiagram
    title Workflow: Creating a Label with In-App Async

    participant User
    participant Quarkus_API as "Quarkus API Endpoint"
    participant MariaDB
    participant Quarkus_Async as "Quarkus BG Thread"
    participant FastAPI_Service as "Embedding Service"
    participant Elasticsearch
    
    User->>+Quarkus_API: POST /actions (label="New Label Text")
    Quarkus_API->>+MariaDB: INSERT action (text="New Label", embedding=NULL)
    MariaDB-->>-Quarkus_API: Returns newAction (id=123)
    
    Note over Quarkus_API,Quarkus_Async: API triggers async task (fire and forget).<br/>This task is NOT durable and will be<br/>lost if the application restarts now.
    
    Quarkus_API-->>-User: 201 Created (id=123)

    Quarkus_Async->>+FastAPI_Service: POST /create-embedding (text="...")
    FastAPI_Service-->>-Quarkus_Async: Returns embedding vector [0.1, ...]
    Quarkus_Async->>+MariaDB: UPDATE action SET embedding=[...] WHERE id=123
    MariaDB-->>-Quarkus_Async: OK
    Quarkus_Async->>+Elasticsearch: INDEX document (id=123, text="...", embedding=[...])
    Elasticsearch-->>-Quarkus_Async: OK
Loading

5.2 Real-time: Semantic Search (User Query)

This workflow is synchronous, optimized for speed, and is not affected by the removal of RabbitMQ.

sequenceDiagram
    title Workflow: Performing a Semantic Search

    participant User
    participant Quarkus_API as "Quarkus API Endpoint"
    participant FastAPI_Service as "Embedding Service"
    participant Elasticsearch

    User->>+Quarkus_API: GET /search?q="bill my client"
    Quarkus_API->>+FastAPI_Service: POST /create-embedding (text="bill my client")
    FastAPI_Service-->>-Quarkus_API: Returns query vector [0.4, ...]
    Quarkus_API->>+Elasticsearch: kNN Search (query_vector=[0.4,...], min_score=0.85)
    Elasticsearch-->>-Quarkus_API: Returns Top Matching Actions [{id: 1, score: 0.92}, ...]
    Quarkus_API-->>-User: 200 OK (Returns search results)
Loading

5.3 Reliability: The Recovery Cron Job

This scheduled job is the primary guarantee of reliability. It ensures that any jobs lost due to an application restart or transient failure are eventually found and processed.

sequenceDiagram
    title Workflow: Scheduled Recovery Cron Job

    participant Quarkus_Scheduler as "Quarkus Scheduler (@Scheduled)"
    participant MariaDB
    participant Quarkus_Async as "Quarkus BG Thread"

    loop Every Hour
        Quarkus_Scheduler->>Quarkus_Scheduler: Cron job triggers
    end

    Quarkus_Scheduler->>+MariaDB: SELECT id, label FROM actions WHERE embedding IS NULL
    MariaDB-->>-Quarkus_Scheduler: Returns list of unprocessed actions

    loop For Each Unprocessed Action
        Quarkus_Scheduler->>Quarkus_Async: Trigger Async Embedding Process(action)
    end
Loading

6. Data Schema and Storage

The data schemas remain the same as in the previous architecture.

MariaDB

ALTER TABLE actions
ADD COLUMN embedding BLOB;

Elasticsearch

PUT /actions-index
{
  "mappings": {
    "properties": {
      "action_id": {"type": "keyword"},
      "action_text": {"type": "text"},
      "action_embedding": {
        "type": "dense_vector",
        "dims": 384,
        "index": "true",
        "similarity": "cosine"
      }
    }
  }
}

7. Deployment & Operations

  • Containerization: All components (Quarkus App, FastAPI Service, Elasticsearch, MariaDB) will be containerized using Docker.
  • Configuration: External endpoints will be managed via Quarkus's application.properties and environment variables.
  • Monitoring: Key metrics to monitor include:
    • The number of rows in the actions table where embedding is NULL. A consistently high or growing number indicates a problem with the async processing or the Python service.
    • Execution time and error rate of the Recovery Cron Job.
    • Latency and error rate of the Embedding Service.
    • Latency of Elasticsearch knn queries.

8. Appendix

  1. Epic 1
  2. All EPICS

Clone this wiki locally