Model Portfolio

Model Portfolio

Snapshot: 2026-09-04. This is a selection and evaluation list, not an installation instruction. Every model remains local, opt-in, revision-pinned, and limited to review candidates or derived text and image data. Recheck model cards, licenses, releases, and security advisories before implementation.

Starting Order

Task	Candidate	Decision	Integration
Comment outliers	PyOD ECOD	Implemented; transparent feature baseline remains the comparator	SignalForge
Screenshot OCR	PaddleOCR 3 with PP-OCRv6	Implemented; validate first on short social-media screenshot text	GlyphWatch
Document and layout OCR	PaddlePaddle/PaddleOCR-VL-1.6	Evaluate only when tables, complex layouts, or multi-page sources matter	Dedicated opt-in adapter
Topic and risk triage	MoritzLaurer/bge-m3-zeroshot-v2.0-c	Candidate for configurable multilingual labels without initial training	Local review adapter
Semantic retrieval	BAAI/bge-m3	First retrieval encoder candidate for multilingual case records	Embeddings plus SQLite/vector index
Source reranking	BAAI/bge-reranker-v2-m3	Apply after bge-m3 and only to top candidates	Retrieval stage
Audio from local video	openai/whisper-large-v3-turbo	Optional for explicitly imported local media, not remote retrieval	Media transcript adapter
Source-bound summary	Qwen/Qwen3-8B	Later optional assistance: evidenced facts plus readable model assessment and evidence IDs	Local tool-bounded service

Rationale and Limits

• BAAI/bge-m3 supports dense, sparse, and multi-vector retrieval across many languages and long inputs. Evaluate hybrid retrieval and reranking for case research, never guilt or identity judgments.
• MoritzLaurer/bge-m3-zeroshot-v2.0-c frames triage as entailment. Labels must remain narrow and observable, such as contains a direct link, contains a threat, or contains derogatory language; avoid conclusion labels such as is coordinated or is person X.
• PaddlePaddle/PaddleOCR-VL-1.6 is a multimodal document parsing candidate. PP-OCRv6 remains the smaller screenshot baseline until a fixed benchmark justifies another adapter.
• openai/whisper-large-v3-turbo is an optional local transcription candidate. A transcript must preserve inspectable time references and is not automatically a conclusion about a video's meaning.
• Qwen/Qwen3-8B is a local multilingual assistant candidate. Its output contract separates evidence-backed facts from a readable probabilistic assessment, confidence, supporting references, and uncertainties.

Implementation Patterns

Retrieval Funnel

1. Normalize text and metadata locally.
2. Create bge-m3 embeddings per observation and source; store model revision and input hash.
3. Combine dense results with lexical search.
4. Rerank only top candidates with bge-reranker-v2-m3.
5. Return original URL, timestamp, and evidence ID to the interface.

The retrieval pipeline returns sources, not a free-form answer. Only a human review or a separate source-bound drafting stage may turn those sources into prose.

Triage Funnel

1. Run deterministic markers and ECOD first.
2. Apply zero-shot classification only to rule-relevant or review-relevant content.
3. Store score, label, prompt template, and model revision.
4. Mark nothing below a fixed threshold; return review_required in the uncertainty range.
5. Collect accepted and rejected review decisions as a future gold set.

LLM Funnel

1. A retrieval job returns a closed set of evidence IDs and source excerpts.
2. The LLM receives only those excerpts and config/qwen_response_contract.json.
3. A validator rejects factual claims with missing or unknown evidence IDs; the model assessment remains a distinct readable section.
4. Show evidenced facts, model assessment, uncertainty, and review questions in that order. Additional execution paths must be separate, logged adapters.

Candidate Fine-Tuning Work

Objective	Practical approach	Starting condition
Narrow local review taxonomy	SetFit on a multilingual sentence transformer	Enough double-reviewed labels and an isolated test split
Retrieval for project-specific sources	Contrastive fine-tuning of bge-m3	Query-document pairs with relevance judgments
Draft format or extraction schema	LoRA/PEFT on a local instruct model	Large quality-controlled set with evidence IDs and controlled raw-data handling
OCR adaptation	Error analysis and a language/image benchmark first	Repeated, measurable OCR failure on lawfully held data

SetFit is a candidate for classification with relatively few labeled examples. PEFT/LoRA reduces the number of trainable parameters for later fine-tuning while retaining the base model. See the SetFit documentation and PEFT quick tour.

Evaluation Protocol

• Record model revision, license, size, hardware needs, and offline requirements before installation.
• Version data provenance, purpose, label definitions, reviewers, splits, and exclusion rules before training.
• For classification, report per-label precision, recall, F1, and confusion matrices; allow unknown as a valid output.
• For retrieval, report Recall@k, MRR, and the share of hits with a complete source path.
• For OCR, report character and word error rates plus visual checks of bounding-box and line references.
• Compare every evaluation with the deterministic baseline. More findings without acceptable precision are not progress.