Architecture

Architecture

OSINT Forge is a local-first Python orchestration layer around independently installed tools. Core logic stays generic; tool-specific behavior lives in reviewed plugins. The architecture is being extended additively toward a complete intelligence workflow at v1.0.

Major components

• bin/osint — launcher
• forge/osint_forge.py — CLI, catalog, validation, lifecycle, adapter, batch, and case orchestration
• forge/reporting.py — normalized findings, provider intelligence, annotations, redaction, and deterministic multi-format reports
• forge/correlation.py — provider observations, evidence-backed relationships, contradictions, temporal state, and scoped confidence
• forge/providers.py — strict provider contracts, adapter execution, and preserved evidence ingestion
• forge/entities.py — canonical seed-entity projection and provenance-preserving deduplication
• plugins/<tool>/ — manifest, adapters, and lifecycle scripts
• workflows/ — versioned deterministic investigation plans
• config/ — shipped configuration templates
• docs/ — canonical repository documentation and plugin template
• tests/ — deterministic unit and integration coverage
• scripts/ — install, uninstall, development checks, the exact-commit QA harness, and one-command Codex QA launcher

Layered intelligence model

1. Seeds: explicitly authorized inputs such as usernames, emails, phone numbers, names, addresses, domains, IPs, URLs, images, and files.
2. Execution: compatible adapters run with bounded concurrency and safe argument arrays.
3. Evidence: raw tool output, commands, versions, timestamps, status, and provenance are preserved.
4. Normalization: versioned projections represent entities, observations, relationships, and findings without destroying raw evidence.
5. Analysis: deduplication, correlation, review state, and explainable confidence remain distinct from source assertions.
6. Orchestration: eligible derived entities may schedule additional work only through explicit scope, depth, request, time, and loop-prevention controls.
7. Reporting: analyst-reviewable intelligence products link every conclusion back to preserved evidence.

v0.4 establishes reporting and seed entities. v0.5 adds schema-2 entity contracts and deterministic candidate extraction. The planning layer, evidence integrity, and portable bundles ship through v0.7. Stable v0.8 adds explicit provider execution and ingestion plus deterministic, provenance-linked correlation, source-independence analysis, contradictions, temporal assessment, verification states, and scoped confidence. Bounded recursive orchestration remains scheduled for v0.9, followed by stable v1.0 contracts. See Roadmap.

State and data

• Framework state: ~/.local/state/osint-forge/
• User configuration: ~/.config/osint-forge/
• Default batch output: ~/OSINT-Cases/Batch-Runs/
• Default case root: ~/OSINT-Cases/<case-id>/
• Installed framework: /usr/local/share/osint-forge/ with a launcher under /usr/local/bin/

Framework-created state and result artifacts use owner-only permissions. Uninstall preserves user configuration, third-party tools, and case data.

Plugin model

Manifests are versioned contracts. Schema 2 declares metadata, supported input target types, explicit accepted and emitted entity types, lifecycle entry points, root needs, batch capability, safe argument-array adapters, and plugin-owned normalizers. Candidate observations remain separate from relationship and identity conclusions. New fields must be versioned and backward-aware.

Execution and case model

Single-run mode validates one target and launches one adapter without shell interpolation. Batch mode matches compatible installed plugins and executes bounded parallel jobs. Case mode adds durable metadata, authorization scope, normalized targets, append-only activity, resumable jobs, raw evidence, entity projection, and protected reports.

Seed identity, source observation, inferred relationship, analyst finding, and confidence assessment are separate concepts. Deduplication never erases provenance, and confidence never converts an automated lead into a verified fact.

Trust boundaries

OSINT Forge controls its command construction, files, permissions, validation, provenance, planning, and policy limits. Upstream tools control their queries, authentication, dependencies, output quality, and service interactions. Automated output remains a lead until independently reviewed.

Compatibility strategy

The transition through v1.0 is gradual: preserve raw output, add versioned projections, keep source adapters independent, migrate known schemas explicitly, reject unknown future schemas, and freeze stable contracts only after v0.9 hardening and live release testing.

Guided UI and shared services

The complete guided terminal interface is a mandatory v1.0 release gate. Bare osint launches the interface; explicit subcommands continue to launch the advanced CLI.

The UI and CLI must remain thin front ends over the same application-service and engine boundaries:

Feature and plugin registries
            |
Shared application services
       /          \
Guided UI       Advanced CLI
            |
Investigation engine

The UI must not duplicate case, plugin, execution, confidence, evidence, or reporting logic. Target types, plugin capabilities, validators, labels, and compatibility metadata should be registry-driven so ordinary additions appear without bespoke menu pathways. New engine behavior must expose truthful state contracts before the UI claims controls such as pause, cancel, recursive queue progress, or ambiguity resolution.

See Interactive UI and Roadmap.