Roadmap

SithAssembly//SithInsta Roadmap

Snapshot: 2026-09-04. This roadmap prioritizes local, evidence-bound casework. Automation and additional execution modes are planned as separate, configurable adapters; they are not part of the current local runtime.

Current State

• The local SQLite server, CommandDeck, case management, graph, timeline, and JSON/PDF export are implemented.
• EvidenceVault, local JSONL runtime logs, and a controlled module registry are implemented.
• Comment outlier scoring, OCR, and relative-depth derivatives are available as opt-in adapters.
• config/agent_registry.json describes local agent roles and topic routing. It does not connect third-party services.

Phase 1: Evidence-Bound Intake

Objective: every case starts with traceable, locally stored data.

• Maintain an import schema for manual records and officially exported files.
• Record source, capture time, hash, operator, and license or usage notes for every import.
• Deduplicate through content and context fingerprints before creating more observations.
• Provide import preview, field validation, and import history in the UI.

Acceptance: a test case can be imported repeatedly and traced completely to the source file without duplicate records.

Phase 1A: Account Connector and Scheduled Collection

Objective: a future account adapter can be explicitly enabled at local server startup, process controlled jobs, and report every run.

• Use config/instagram_accounts.local.json as local account and connector configuration. Keep enabled: false until an adapter is implemented.
• Store persistent collection jobs with account reference, case ID, topic, start time, status, cursor or checkpoint, and idempotency key.
• Use one central queue instead of parallel direct requests. It must operate within the configured provider quota and record requests, response states, backoff, and restart behavior.
• Make autostart an editable connector profile: disabled, manual, scheduled, or local server startup. Every start creates an agent report.
• Map timeouts, errors, and quota responses to backoff, blocked, or failed; do not replace them with account sharding or bypass behavior.
• Write adapter results through the existing import boundary and emit collection.run_reported with batch and evidence references.

Acceptance: an approved test run is scheduled under the selected startup profile, resumes from a checkpoint, and emits a complete report without duplicate records.

Phase 2: Analysis Funnel

Objective: produce candidates in stages without turning a score into a conclusion.

1. Normalize text, time, handle, links, hashtags, and source binding.
2. Extract deterministic signals: duplicates, shared domains, mentions, time windows, and account-change indicators.
3. Apply lightweight models for ECOD outliers, OCR, semantic similarity, and zero-shot topic candidates.
4. Store graph edges only with evidence ID, rule or model version, time, and confidence.
5. Require human review to accept, reject, or qualify candidates.

Acceptance: every candidate shows raw evidence, transformation steps, model or rule version, and the reason for its score.

Phase 3: Analyst Workspace

Objective: make research and review faster than manual spreadsheet work.

• Show runtime and agent status in the interface.
• Filter the review queue by source, confidence, model version, and case.
• Link graph edges and timeline events directly to observations, screenshots, and original timestamps.
• Extend source management with trust level, archive status, and counter-evidence.
• Add comparison views for profile snapshots and documented alias or account-change indicators.

Acceptance: an analyst can navigate from a graph node to original evidence and reconstruct every assignment.

Phase 4: Local Orchestration

Objective: make processing repeatable, cancellable, and auditable.

• Maintain persistent jobs with queued, running, completed, failed, needs_review, and cancelled states.
• Persist the event envelope defined in AGENT_COORDINATION.md; make jobs idempotent through input hash and configuration version.
• Record duration, model revision, error class, input/output references, and resource use for every module.
• Route only topics allowed by config/agent_registry.json; never derive shell or network capability from a registry entry.

Acceptance: an interrupted local run can resume without duplicate edges or duplicate OCR text.

Phase 5: Models and Evaluation

Objective: activate models only after measurable quality control.

• Build a versioned gold set from lawfully stored, anonymized, or explicitly approved examples.
• Define task-specific metrics: OCR CER/WER, retrieval Recall@k, per-class precision and recall, and outlier review yield.
• Split by time and source so duplicates and the same event do not leak across train and test.
• Analyze error rates by language, image quality, and content type.
• Never use a model result as a factual identity decision or conclusion.

Acceptance: a model is activated only if it beats the deterministic baseline without creating unacceptable review load.

Phase 6: Reports and Operations

Objective: keep case records exportable and reproducible.

• Test PDF, JSON, and EvidenceVault output against realistic sanitized cases.
• Document backup and restore for SQLite, evidence, logs, and vault keys.
• Maintain configuration profiles for development, offline analysis, and optional local model serving.
• Automate end-to-end tests for import -> analysis -> review -> export.

Acceptance: a case can be restored on a new local host and validated against its encrypted vault.

Priority Order

1. Evidence intake, the connector boundary, and the persistent job ledger.
2. ECOD and OCR against approved real-world test data.
3. Visible review and agent status in the interface.
4. Semantic retrieval and specialized classification after a gold set exists.
5. Optional LLM assistance last, exclusively as a source-bound draft.

Continue with Model Portfolio, Agent Orchestration, and the versioned AGENT_COORDINATION.md.