2. Technical Architecture

This document describes the architecture and operation of ClioDeck's RAG (Retrieval-Augmented Generation) system, an academic writing assistant for historians and humanities researchers.

Table of Contents

• Overview
• Global Architecture
• Indexing Pipeline
• Search System
• Assisted Generation (RAG)
• External Integrations
• Performance and Optimizations

---

Overview

ClioDeck is an Electron application that combines:

• Frontend: React/TypeScript interface with Milkdown Editor
• Backend: Node.js/TypeScript services for indexing and search
• Python Services: Topic modeling and textometric analysis
• Local LLM: Ollama for embeddings and text generation
• Storage: SQLite for data and vector indexes

Design Goals

1. Local-first: All data remains on the user's machine
2. Performance: Optimized for modest machines (8-16 GB RAM, CPU only)
3. Academic: Complete traceability and verifiable citations
4. Interoperability: Integration with Zotero, BibTeX, Markdown, PDF

---

Global Architecture

┌─────────────────────────────────────────────────────────────┐
│                    ClioDeck Electron App                      │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Frontend   │  │   Backend    │  │   Services   │      │
│  │  React/Vite  │◄─┤  TypeScript  │◄─┤    Python    │      │
│  │              │  │              │  │              │      │
│  │ • Editor     │  │ • PDF Index  │  │ • Topic      │      │
│  │ • Chat UI    │  │ • Vector DB  │  │   Modeling   │      │
│  │ • Config     │  │ • Search     │  │ • BERTopic   │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
│                            │                │                │
│                            ▼                ▼                │
│                    ┌──────────────┐  ┌──────────────┐      │
│                    │   Ollama     │  │   SQLite     │      │
│                    │   (Local)    │  │ + HNSW Index │      │
│                    └──────────────┘  └──────────────┘      │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Key Technologies

Component	Technology	Role
UI Framework	React 18 + TypeScript	User interface
Editor	Milkdown	WYSIWYG Markdown editing
Build Tool	Vite	Bundling and hot reload
Desktop	Electron 28	Multi-platform desktop app
Database	better-sqlite3	Synchronous local storage
Vector Index	hnswlib-node	Fast vector search
Sparse Index	natural (BM25)	Keyword search
LLM	Ollama	Local embeddings + generation
PDF Processing	pdfjs-dist	Text extraction
Bibliography	BibTeX parsing	Reference management
Topic Modeling	Python + BERTopic	Thematic analysis

---

Indexing Pipeline

The indexing pipeline transforms PDFs into queryable vector chunks.

Step 1: PDF Extraction

File: backend/core/pdf/PDFIndexer.ts

// 1. Load PDF with pdfjs
const pdf = await pdfjs.getDocument(pdfPath).promise;

// 2. Extract text page by page
const pages = [];
for (let i = 1; i <= pdf.numPages; i++) {
  const page = await pdf.getPage(i);
  const textContent = await page.getTextContent();
  const text = textContent.items.map(item => item.str).join(' ');
  pages.push({ pageNumber: i, content: text });
}

// 3. Extract metadata
const metadata = await pdf.getMetadata();
const title = metadata.info.Title || path.basename(pdfPath);

Output: Array of pages with raw text + document metadata

---

Step 2: Adaptive Chunking

Files:

• backend/core/chunking/AdaptiveChunker.ts (recommended)
• backend/core/chunking/DocumentChunker.ts (fallback)
• backend/core/chunking/SemanticChunker.ts (advanced semantic-aware chunking)
• backend/core/chunking/ChunkDeduplicator.ts (removes near-duplicate chunks)
• backend/core/chunking/ChunkQualityScorer.ts (scores chunk quality for filtering)
• backend/core/chunking/EmbeddingCache.ts (caches embeddings for performance)

Structure Detection

The adaptive chunker automatically detects document structure:

// Header detection
const patterns = {
  markdown: /^#{1,6}\s+(.+)/,           // # Title, ## Subtitle
  numbered: /^\d+(\.\d+)*\s+(.+)/,      // 1. Introduction, 1.1 Context
  uppercase: /^[A-Z\s]{3,}$/,           // INTRODUCTION, METHODOLOGY
  roman: /^[IVX]+\.\s+(.+)/             // I. Introduction, II. Methods
};

// Automatic classification
const sectionTypes = {
  abstract: /abstract|résumé/i,
  introduction: /introduction/i,
  methodology: /method|méthodologie/i,
  results: /results|résultats/i,
  discussion: /discussion/i,
  conclusion: /conclusion/i,
  references: /references|bibliographie/i
};

Chunk Creation

Principles:

1. Respect sentence boundaries: Chunks always end with ., !, ? or ;
2. Document context: Each chunk includes title and section as prefix
3. Skip references: Bibliography section is not indexed
4. Smart overlap: Overlaps occur on complete sentences

Generated chunk example:

[Doc: Active Learning Strategies in Higher Education | Section: Methodology]

The intervention consisted of three phases: pre-assessment, active learning
activities using Bloom's taxonomy, and post-assessment. Students were divided
into control and experimental groups. The control group followed traditional
lecture-based instruction, while the experimental group participated in
collaborative problem-solving activities.

Configuration:

interface ChunkingConfig {
  chunkSize: number;        // Target size in words (300-800)
  overlap: number;          // Overlap in words (50-100)
  maxTokens: number;        // Strict token limit (8192 for nomic-embed-text)
}

const CHUNKING_CONFIGS = {
  cpuOptimized: { chunkSize: 300, overlap: 50, maxTokens: 8192 },
  standard: { chunkSize: 500, overlap: 75, maxTokens: 8192 },
  large: { chunkSize: 800, overlap: 100, maxTokens: 8192 }
};

Output: Array of chunks with metadata

interface DocumentChunk {
  id: string;                 // UUID
  documentId: string;         // Document reference
  content: string;            // Chunk text (with context)
  pageNumber: number;         // Source page
  chunkIndex: number;         // Position in document
  embedding?: number[];       // Vector (768 dimensions)
  metadata?: {
    sectionTitle?: string;    // "Methodology", "Results"
    sectionType?: string;     // "methodology", "results"
    sectionLevel?: number;    // 1, 2, 3...
  };
}

---

Step 3: Embedding Generation

File: backend/core/llm/OllamaClient.ts

// 1. Connect to Ollama
const response = await fetch('http://localhost:11434/api/embeddings', {
  method: 'POST',
  body: JSON.stringify({
    model: 'nomic-embed-text',  // 768 dimensions
    prompt: chunkContent
  })
});

// 2. Retrieve vector
const { embedding } = await response.json();
// embedding: number[] (768 dimensions)

// 3. Normalization (if needed)
const normalized = normalize(embedding);

Model used: nomic-embed-text

• Dimensions: 768
• Max context: 8192 tokens
• Size: ~274 MB
• Performance: ~100 embeddings/second on average CPU

Emergency chunking: If a chunk exceeds 8192 tokens, it's intelligently subdivided:

private chunkText(text: string, maxLength: number): string[] {
  // Find sentence ending in last 200 characters
  const sentenceEndings = /[.!?;](?=\s|$)/g;
  // ... cut at last found period
}

---

Step 4: Multi-Strategy Indexing

File: backend/core/vector-store/EnhancedVectorStore.ts

Three indexes are created simultaneously:

Index 1: SQLite (database)

CREATE TABLE chunks (
  id TEXT PRIMARY KEY,
  document_id TEXT NOT NULL,
  content TEXT NOT NULL,
  page_number INTEGER,
  chunk_index INTEGER,
  embedding BLOB,           -- Serialized vector
  metadata TEXT,            -- JSON
  created_at INTEGER
);

CREATE INDEX idx_document ON chunks(document_id);
CREATE INDEX idx_page ON chunks(page_number);

Role: Persistent storage + fallback for linear search

Index 2: HNSW (fast vector search)

File: backend/core/vector-store/HNSWVectorStore.ts

import { HierarchicalNSW } from 'hnswlib-node';

const index = new HierarchicalNSW('cosine', 768); // 768 dimensions
index.initIndex(maxElements, M=16, efConstruction=100);

// Indexing
for (const chunk of chunks) {
  index.addPoint(chunk.embedding, chunk.id);
}

// Save to disk
index.writeIndexSync(indexPath);

Parameters:

• M=16: Connections per node (speed/precision trade-off)
• efConstruction=100: Construction effort
• efSearch=50: Search effort

Complexity:

• Construction: O(n log n)
• Search: O(log n)

Memory footprint: ~500 MB for 50k chunks (768 dims)

Index 3: BM25 (keyword search)

File: backend/core/search/BM25Index.ts

import natural from 'natural';

// 1. Tokenization
const tokens = tokenizer.tokenize(chunk.content.toLowerCase());

// 2. Build inverted index
for (const token of tokens) {
  invertedIndex[token] = invertedIndex[token] || [];
  invertedIndex[token].push(chunkId);
}

// 3. Calculate IDF
const idf = Math.log((totalDocs - docFreq + 0.5) / (docFreq + 0.5));

// 4. BM25 scoring
const score = idf * ((tf * (k1 + 1)) / (tf + k1 * (1 - b + b * (docLength / avgDocLength))));

Parameters:

• k1=1.5: Term frequency saturation
• b=0.75: Document length normalization

Memory footprint: ~100 MB for 50k chunks

---

Search System

Hybrid Search (Dense + Sparse)

File: backend/core/search/HybridSearch.ts

Hybrid search combines the advantages of two complementary approaches:

Query: "bloom taxonomy methodology"
        │
        ├────────────────────┬────────────────────┐
        │                    │                    │
        ▼                    ▼                    ▼
  ┌──────────┐        ┌──────────┐        ┌──────────┐
  │  Dense   │        │  Sparse  │        │  Fusion  │
  │  (HNSW)  │        │  (BM25)  │        │  (RRF)   │
  └──────────┘        └──────────┘        └──────────┘
        │                    │                    │
        ▼                    ▼                    ▼
   Top-50 results       Top-50 results       Top-10 final
   (semantic)           (keywords)           (best of both)

Step 1: Dense Search (HNSW)

// 1. Generate query embedding
const queryEmbedding = await ollamaClient.generateEmbedding(query);

// 2. HNSW search
const denseResults = await hnswStore.search(queryEmbedding, 50);
// Returns: [{ id, score }, ...]

Advantage: Understands semantic meaning ("active learning" ≈ "apprentissage actif")

Step 2: Sparse Search (BM25)

// 1. Query tokenization
const queryTokens = tokenizer.tokenize(query.toLowerCase());

// 2. BM25 scoring
const sparseResults = bm25Index.search(queryTokens, 50);
// Returns: [{ id, score }, ...]

Advantage: Excellent for rare terms, proper nouns, acronyms ("BERTopic", "Piaget")

Step 3: RRF Fusion (Reciprocal Rank Fusion)

function reciprocalRankFusion(
  denseResults: SearchResult[],
  sparseResults: SearchResult[],
  k: number = 60
): SearchResult[] {
  const scores = new Map<string, number>();

  // Dense search contribution (60%)
  for (let i = 0; i < denseResults.length; i++) {
    const chunkId = denseResults[i].id;
    const rrfScore = 0.6 / (k + i + 1);
    scores.set(chunkId, (scores.get(chunkId) || 0) + rrfScore);
  }

  // Sparse search contribution (40%)
  for (let i = 0; i < sparseResults.length; i++) {
    const chunkId = sparseResults[i].id;
    const rrfScore = 0.4 / (k + i + 1);
    scores.set(chunkId, (scores.get(chunkId) || 0) + rrfScore);
  }

  // Sort by final score
  return Array.from(scores.entries())
    .sort((a, b) => b[1] - a[1])
    .slice(0, 10)
    .map(([id, score]) => ({ id, score }));
}

Weights: 60% dense / 40% sparse (configurable)

RRF Formula: RRF(d) = Σ weight_i / (k + rank_i(d))

Performance

Metric	Linear Search	HNSW Only	Hybrid (HNSW+BM25)
Time	500ms	15ms	30ms
Precision@10	65%	75%	80%
Recall@10	45%	55%	60%

---

Assisted Generation (RAG)

Complete Pipeline

User Query
    │
    ├─► 1. Generate embedding
    │       │
    │       ▼
    ├─► 2. Hybrid search (HNSW + BM25)
    │       │
    │       ▼
    ├─► 3. Retrieve top-K chunks (K=10)
    │       │
    │       ▼
    ├─► 4. Optional: Graph expansion
    │       │
    │       ▼
    ├─► 5. Build context
    │       │
    │       ▼
    └─► 6. Generate with LLM
            │
            ▼
        Response with citations

Steps 1-3: Search (see previous section)

Step 4: Graph Expansion (optional)

File: backend/core/analysis/TopicModelingService.ts

If enabled (useGraphContext: true), similar chunks are also retrieved:

// 1. For each retrieved chunk, find its graph neighbors
const neighbors = graph.neighbors(chunkId);

// 2. Calculate cosine similarity
for (const neighbor of neighbors) {
  const similarity = cosineSimilarity(
    chunk.embedding,
    neighbor.embedding
  );

  if (similarity > graphSimilarityThreshold) {
    additionalChunks.push(neighbor);
  }
}

// 3. Add top-N neighbors
const topNeighbors = additionalChunks
  .sort((a, b) => b.similarity - a.similarity)
  .slice(0, additionalGraphDocs);

Configuration:

• useGraphContext: boolean (default: false)
• graphSimilarityThreshold: number (default: 0.7)
• additionalGraphDocs: number (default: 3)

Step 5: Context Building

function buildRAGContext(chunks: DocumentChunk[]): string {
  let context = "# Context (sources)\n\n";

  for (const chunk of chunks) {
    context += `## [${chunk.documentTitle}, p.${chunk.pageNumber}]\n`;
    context += `${chunk.content}\n\n`;
  }

  return context;
}

Generated context example:

# Context (sources)

## [Active Learning in Higher Education, p.12]
[Doc: Active Learning in Higher Education | Section: Methodology]

The intervention consisted of three phases: pre-assessment, active learning
activities using Bloom's taxonomy, and post-assessment...

## [Bloom's Taxonomy Revised, p.34]
[Doc: Bloom's Taxonomy Revised | Section: Cognitive Processes]

The revised taxonomy introduces six levels of cognitive complexity: Remember,
Understand, Apply, Analyze, Evaluate, and Create...

Step 6: LLM Generation

File: src/main/services/chat-service.ts

const prompt = `You are a research assistant for historians.

${ragContext}

User question:
${userQuery}

Instructions:
- Answer ONLY based on the sources above
- Always cite sources with [Document, p.X]
- If information is not in the sources, clearly say so
- Answer in a clear and academic manner

Answer:`;

const response = await ollama.generate({
  model: 'gemma2:2b',
  prompt: prompt,
  stream: true
});

Streaming: Response is sent token by token for better UX

Stream Cancellation: Users can cancel ongoing generation via the chat:cancel IPC handler.

Context Compression

File: backend/core/rag/ContextCompressor.ts

When the retrieved context exceeds the LLM's context window, ClioDeck applies intelligent compression:

┌─────────────────────────────────────────────────────────────┐
│                  Context Compression Levels                   │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  Level 1: Light (context < 15KB)                              │
│    └── Semantic deduplication only                            │
│    └── Remove near-duplicate sentences                        │
│                                                               │
│  Level 2: Medium (context 15-35KB)                            │
│    └── Deduplication + sentence extraction                    │
│    └── Keep most relevant sentences per chunk                 │
│                                                               │
│  Level 3: Aggressive (context > 35KB)                         │
│    └── Full compression                                       │
│    └── Extract key phrases + summarize                        │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Algorithm:

1. Calculate total context size in tokens
2. Select compression level based on size thresholds
3. Apply relevance scoring to sentences (TF-IDF + query similarity)
4. Preserve keyword-rich sentences
5. Return compressed context with reduction statistics

Configuration:

• enableContextCompression: boolean (default: true)
• Automatic level selection based on context size

RAG Explanation (Explainable AI)

File: src/renderer/src/stores/chatStore.ts

ClioDeck provides transparency about how RAG responses are generated:

interface RAGExplanation {
  totalChunksRetrieved: number;     // How many chunks were found
  chunksUsed: number;               // How many were included in context
  searchStrategy: string;           // "hybrid" | "hnsw" | "bm25"
  compressionApplied: boolean;      // Was context compressed?
  compressionRatio?: number;        // e.g., 0.65 = 35% reduction
  sourcesFromPrimary: number;       // Tropy sources count
  sourcesFromSecondary: number;     // PDF sources count
}

This information is displayed in the chat interface, allowing researchers to understand and evaluate the response context.

---

External Integrations

Zotero

File: src/main/services/zotero-service.ts

// 1. Authentication
const headers = {
  'Zotero-API-Key': apiKey,
  'Zotero-API-Version': '3'
};

// 2. Retrieve collections
const collections = await fetch(
  `https://api.zotero.org/users/${userId}/collections`,
  { headers }
);

// 3. Retrieve items from a collection
const items = await fetch(
  `https://api.zotero.org/users/${userId}/collections/${collectionId}/items`,
  { headers }
);

// 4. BibTeX export
const bibtex = await fetch(
  `https://api.zotero.org/users/${userId}/collections/${collectionId}/items?format=bibtex`,
  { headers }
);

// 5. Download PDFs
for (const item of items) {
  if (item.data.linkMode === 'linked_file') {
    const pdf = await downloadAttachment(item.key);
    await savePDF(pdf, projectPath);
  }
}

BibTeX

Parsing: Uses a custom parser for .bib files

function parseBibTeX(bibContent: string): BibEntry[] {
  const entries = [];
  const entryRegex = /@(\w+)\{([^,]+),\s*([\s\S]*?)\n\}/g;

  let match;
  while ((match = entryRegex.exec(bibContent)) !== null) {
    const [_, type, key, fields] = match;

    const entry = {
      type: type.toLowerCase(),
      key: key.trim(),
      fields: parseFields(fields)
    };

    entries.push(entry);
  }

  return entries;
}

Topic Modeling (Python)

Service: backend/python-services/topic-modeling/main.py

from bertopic import BERTopic
from fastapi import FastAPI

app = FastAPI()

@app.post("/analyze")
async def analyze_corpus(documents: List[str]):
    # 1. Embeddings with sentence-transformers
    model = SentenceTransformer('all-MiniLM-L6-v2')
    embeddings = model.encode(documents)

    # 2. Topic modeling with BERTopic
    topic_model = BERTopic()
    topics, probs = topic_model.fit_transform(documents, embeddings)

    # 3. Extract topics
    topic_info = topic_model.get_topic_info()

    return {
        "topics": topics,
        "topic_info": topic_info.to_dict(),
        "probabilities": probs.tolist()
    }

Communication: TypeScript backend communicates via HTTP (port 8001)

const response = await fetch('http://localhost:8001/analyze', {
  method: 'POST',
  body: JSON.stringify({ documents: chunks })
});

const { topics, topic_info } = await response.json();

---

Performance and Optimizations

Memory Footprint (typical 50k chunks)

Component	RAM	Description
Electron	1.5 GB	App + Chromium
HNSW index	500 MB	50k × 768 dims × 3x overhead
BM25 index	100 MB	Inverted index + vocabulary
SQLite	300 MB	Chunks + metadata
Ollama (nomic)	500 MB	Embedding model in RAM
Total	~3 GB	For an average project

Recommended Configuration

Corpus Size	Min RAM	CPU	Indexing Time
10-50 PDFs	4 GB	Dual-core	5-10 min
50-200 PDFs	8 GB	Quad-core	20-40 min
200-500 PDFs	16 GB	8+ cores	1-2 hours

CPU Optimizations

1. Parallel chunking:

// Process documents in parallel (max 4 simultaneous)
const batches = chunk(documents, 4);
for (const batch of batches) {
  await Promise.all(batch.map(doc => indexDocument(doc)));
}

2. Batch embeddings:

// Generate embeddings in batches of 32
for (let i = 0; i < chunks.length; i += 32) {
  const batch = chunks.slice(i, i + 32);
  const embeddings = await ollama.batchEmbed(batch);
}

3. Background index building:

// Build HNSW index asynchronously
setTimeout(() => {
  hnswStore.rebuild();
}, 5000);

Chunking Configurations

const CHUNKING_CONFIGS = {
  // For modest machines (8 GB RAM)
  cpuOptimized: {
    chunkSize: 300,    // Small chunks = fewer tokens
    overlap: 50,       // Minimal overlap
    maxTokens: 8192
  },

  // Balance performance/precision
  standard: {
    chunkSize: 500,
    overlap: 75,
    maxTokens: 8192
  },

  // For maximum precision (16+ GB RAM)
  large: {
    chunkSize: 800,
    overlap: 100,
    maxTokens: 8192
  }
};

---

Key Architecture Files

backend/
├── core/
│   ├── analysis/
│   │   └── TopicModelingService.ts      # Thematic analysis + graph
│   ├── chunking/
│   │   ├── AdaptiveChunker.ts           # Structure-aware chunking
│   │   └── DocumentChunker.ts           # Simple chunking (fallback)
│   ├── history/
│   │   └── HistoryManager.ts            # Research journal
│   ├── llm/
│   │   └── OllamaClient.ts              # Ollama client (embeddings + chat)
│   ├── pdf/
│   │   └── PDFIndexer.ts                # PDF extraction + indexing
│   ├── search/
│   │   ├── BM25Index.ts                 # Sparse index (keywords)
│   │   └── HybridSearch.ts              # Dense + sparse fusion
│   └── vector-store/
│       ├── VectorStore.ts               # SQLite base store
│       ├── HNSWVectorStore.ts           # Fast HNSW index
│       └── EnhancedVectorStore.ts       # Unified wrapper
│
├── python-services/
│   └── topic-modeling/
│       ├── main.py                      # FastAPI service
│       ├── topic_analyzer.py            # BERTopic logic
│       └── requirements.txt             # Python dependencies
│
├── types/
│   ├── config.ts                        # Configuration types
│   └── pdf-document.ts                  # Document/chunk types
│
src/
├── main/
│   ├── services/
│   │   ├── chat-service.ts              # RAG service
│   │   ├── pdf-service.ts               # PDF management
│   │   ├── project-manager.ts           # Project management
│   │   └── topic-modeling-service.ts    # Python service proxy
│   └── ipc/
│       └── handlers/                    # Electron IPC
│
└── renderer/
    ├── components/
    │   ├── Chat/                        # Chat interface
    │   ├── Config/                      # Settings
    │   ├── Editor/                      # Markdown editor
    │   ├── PDFIndex/                    # PDF management
    │   └── Journal/                     # Research journal
    └── stores/
        └── editorStore.ts               # Zustand state

---

Primary Sources Integration (Tropy)

Architecture

ClioDeck maintains a separate vector store for primary sources from Tropy:

┌─────────────────────────────────────────────────────────────┐
│                    Primary Sources Pipeline                   │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  Tropy Project (.tropy / .tpy)                                │
│         │                                                     │
│         ▼                                                     │
│  ┌──────────────┐                                            │
│  │ TropyReader  │ ──► Read-only access to SQLite database    │
│  └──────────────┘                                            │
│         │                                                     │
│         ▼                                                     │
│  ┌──────────────┐                                            │
│  │  TropySync   │ ──► Sync items, photos, notes, tags        │
│  └──────────────┘                                            │
│         │                                                     │
│         ├──► OCR Pipeline (Tesseract.js) for images          │
│         │                                                     │
│         ▼                                                     │
│  ┌────────────────────────┐                                  │
│  │ PrimarySourcesVectorStore │                               │
│  │   - SQLite: sources.db     │                               │
│  │   - HNSW: sources.hnsw     │                               │
│  │   - BM25: in-memory        │                               │
│  └────────────────────────┘                                  │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Data Flow

1. TropyReader opens the .tpy file in read-only mode
2. TropySync extracts items with metadata, notes, and photos
3. Notes are collected as transcriptions (Tropy notes > OCR > manual)
4. DocumentChunker segments transcriptions (same as PDFs)
5. OllamaClient generates embeddings (nomic-embed-text)
6. PrimarySourcesVectorStore stores chunks with HNSW + BM25 indexes

Unified Search

When querying, both stores are searched and results are merged:

// In chat-service.ts
const pdfResults = await vectorStore.search(embedding, topK);
const tropyResults = await tropyService.search(query, { topK });

// Merge and deduplicate
const allResults = [...pdfResults, ...tropyResults]
  .sort((a, b) => b.similarity - a.similarity)
  .slice(0, topK);

---

Embedded LLM Support

Architecture

ClioDeck supports embedded LLMs via node-llama-cpp for offline generation:

┌─────────────────────────────────────────────────────────────┐
│                  LLM Provider Architecture                    │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  LLMProviderManager                                           │
│         │                                                     │
│         ├──► OllamaClient (primary)                          │
│         │       └─ Embeddings: nomic-embed-text              │
│         │       └─ Generation: gemma2:2b                     │
│         │                                                     │
│         └──► EmbeddedLLMClient (fallback)                    │
│                 └─ Generation: Qwen2.5-0.5B/1.5B (GGUF)      │
│                 └─ NO embeddings (requires Ollama)           │
│                                                               │
│  Provider Selection (mode: auto):                             │
│    1. Check Ollama availability                               │
│    2. If available → use Ollama                               │
│    3. If not → use embedded model (generation only)          │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Embedded Models

Model	Size	Context	Performance
Qwen2.5-0.5B	469 MB	32K tokens	~20 tokens/sec
Qwen2.5-1.5B	1.0 GB	32K tokens	~12 tokens/sec

Prompt Format

Embedded models use ChatML format:

<|im_start|>system
Tu es un assistant académique...
<|im_end|>
<|im_start|>user
Question utilisateur
<|im_end|>
<|im_start|>assistant

---

RAG Configuration Defaults

Updated default values for version 1.0.0-beta.2:

const DEFAULT_RAG_CONFIG = {
  // Search parameters
  topK: 10,                          // Number of chunks to retrieve
  similarityThreshold: 0.12,         // RRF-optimized (not cosine similarity)

  // Chunking
  chunkingConfig: 'cpuOptimized',    // 300 words, 50 overlap
  useAdaptiveChunking: true,

  // Indexes
  useHNSWIndex: true,
  useHybridSearch: true,             // HNSW + BM25

  // Quality
  enableQualityFiltering: true,
  enablePreprocessing: true,
  enableDeduplication: true,

  // Advanced (optional)
  useSemanticChunking: false,
  enableContextCompression: true   // Default: true
};

Similarity Threshold Note

The default similarityThreshold of 0.12 is optimized for RRF (Reciprocal Rank Fusion) scores, which are much smaller than raw cosine similarity scores. Typical RRF scores range from 0.001 to 0.02.

---

Technical References

Papers

• HNSW: Efficient and robust approximate nearest neighbor search (Malkov & Yashunin, 2016)
• BM25: The Probabilistic Relevance Framework: BM25 and Beyond (Robertson & Zaragoza, 2009)
• RRF: Reciprocal rank fusion outperforms condorcet (Cormack et al., 2009)
• RAG: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

Tools and Libraries

• Ollama - Local LLMs
• node-llama-cpp - Embedded LLM execution
• hnswlib - HNSW implementation
• natural - NLP for Node.js
• BERTopic - Topic modeling
• pdfjs-dist - PDF rendering
• better-sqlite3 - Synchronous SQLite
• Tesseract.js - OCR for primary sources
• Tropy - Primary sources management

---

Version: 1.0.0-beta.2 Last updated: 2026-01-27 Status: Beta