Gnosis compiles what you have into what you know.
Gnosis is an enterprise knowledge compiler: a local-first Rust tool that discovers digital objects in a repository or S3 bucket, extracts structured knowledge with deterministic providers (Tree-sitter and lightweight document/data parsers), shows its work in a live TUI, and exports an OKF v0.1-style bundle.
This is a proof of concept — not a search engine, vector database, or chat UI.
cargo install gnosiscargo install --git https://github.com/thanos/gnosis
# or from a checkout:
cargo install --path .Prebuilt binaries for Linux, macOS, and Windows are attached to GitHub Releases on each v* tag (with SHA256SUMS).
cargo build --release
./target/release/gnosis scan ./fixtures/mixed-repoHeadless (CI / scripting):
cargo run -- scan ./fixtures/mixed-repo --no-tui --exportS3 (bucket = root, keys = paths; default AWS credentials):
gnosis scan s3://my-bucket --no-tui --export
gnosis scan s3://my-bucket/path/prefix --region us-east-1| Command | Description |
|---|---|
gnosis scan <path> |
Live TUI scan (local directory) |
gnosis scan s3://bucket[/prefix] |
Scan an S3 bucket |
gnosis scan <target> --no-tui |
Headless scan + summary |
gnosis scan <target> --no-tui --export |
Also write knowledge.okf/ |
gnosis about |
Product overview |
| Flag | Description |
|---|---|
--no-tui |
Headless mode (prints summary) |
--quiet |
Suppress per-object event lines in headless mode |
--export |
Write OKF when the scan finishes (headless) |
--output <dir> |
OKF output directory (default: knowledge.okf) |
--max-size <bytes> |
Max bytes read per object (default: 2 MiB) |
--concurrency <n> |
Analysis worker count |
--region <name> |
AWS region for s3:// scans (default credential chain otherwise) |
--job-db <path> |
Persistent job queue database (default: .gnosis/jobs.redb) |
--max-attempts <n> |
Attempts per job including the first (default: 3; 1 disables retries) |
--retry-base-ms <ms> |
Initial retry backoff, doubled after each failure (default: 250) |
--retry-max-ms <ms> |
Upper bound on retry backoff (default: 30000) |
Local scans are recursive under <path> (via ignore::WalkBuilder). Descent skips .gitignore matches, target / node_modules / .git / knowledge.okf / .gnosis, and does not follow symlinks.
S3 scans list object keys under the bucket (or prefix), skip directory markers (…/), and skip keys whose path components match the same basename excludes (target, node_modules, .git, knowledge.okf, .gnosis).
Every discovered artifact is enqueued as a durable analyze_object job (function kind + JSON args + result/error). Async workers claim jobs from the store; the default backend is redb.
A job that fails is retried automatically with exponential backoff: after attempt n it goes back to pending with available_at = now + min(retry_max_ms, retry_base_ms * 2^(n-1)), and no worker can claim it until then. It is only marked failed once --max-attempts is exhausted; the stored error keeps the attempt count.
Each scan assigns a scan:… id; every job is linked to it. Reruns create a fresh rerun:… id.
gnosis jobs scans # list scan ids + counts
gnosis jobs list
gnosis jobs list --status failed
gnosis jobs list --scan-id scan:… # filter by scan (unique prefix ok)
gnosis jobs list --status completed --limit 20
gnosis jobs show job:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
gnosis jobs show <short-id-prefix> # unique prefix also works
gnosis jobs purge 5d # delete jobs older than 5 days
gnosis jobs purge 12h --dry-run # preview only
gnosis jobs purge --scan-id scan:… # delete every job in a scan
gnosis jobs purge 5d --scan-id scan:… # age filter within a scan
gnosis jobs pause --scan-id scan:… # pause pending/running in a scan
gnosis jobs pause abc123de,def456
gnosis jobs unpause --scan-id scan:…
gnosis jobs stop --scan-id scan:… # cancel pending/paused/running
gnosis jobs rerun job:aaa,job:bbb # requeue + re-execute
gnosis jobs rerun --scan-id scan:… # requeue + re-execute whole scan
gnosis jobs rerun abc123de,def456 --no-runsummary · objects [status] · unknown · providers · stats · jobs [status] · job <id> · find <text> · explain <name> · graph <name> · export okf [path] · quit
Shortcuts: s summary · u unknown · e export · J jobs · q quit
- A live (or printed) inventory of what was understood, partial, unknown, or failed
- Extracted entities and relationships (functions, classes, modules, documents, CSV columns, …)
- Optional OKF bundle (
knowledge.okf/by default) — markdown + YAML you can browse, commit, or feed to other tools
- Point at an unfamiliar local Git/repo tree and see structure emerge
- Scan an S3 bucket the same way (keys as file paths) from a laptop or CI
- Demo / PoC of “compile tree → structured knowledge”
- Export a portable knowledge directory for humans or agents
Not for yet: remote Git hosts, web crawl, PDFs/office, vectors/RAG, or chat (see Limitations).
One crate provides both the CLI and the library. Depend on it as:
[dependencies]
gnosis = "0.1"Public modules include pipeline/store/query APIs plus providers, okf, and tui for embedding.
Headless scan + export sketch:
use gnosis::{
default_registry, Exporter, OkfExporter, Pipeline, PipelineEvent, QueryEngine, Result,
ScanConfig,
};
fn main() -> Result<()> {
let config = ScanConfig::with_root("./my-repo");
let pipeline = Pipeline::new(config.clone(), default_registry())?;
let mut handle = pipeline.spawn();
let events = handle.take_events();
while let Ok(ev) = events.recv() {
if matches!(ev, PipelineEvent::ScanCompleted { .. }) {
break;
}
}
handle.wait()?;
let store = handle.store.lock().unwrap();
print!("{}", QueryEngine::new(&store).summary());
OkfExporter::new().export(&store, &config.output_path)?;
Ok(())
}Typical flow: default_registry() → Pipeline::spawn → observe PipelineEvents / query KnowledgeStore → OkfExporter::export.
gnosis (one crate)
├── binary: gnosis
└── library modules:
├── connectors, pipeline, store, query
├── providers (tree-sitter + docs/data + generic)
├── okf (export)
└── tui (Ratatui UI)
Pipeline: Connector → Discovery → Job queue (analyze_object) → ProtoData → Provider selection → Analysis → Knowledge store → Query / OKF export.
Events flow over a bounded channel so the TUI observes progress without coupling to workers.
- Code: C++, Rust, Elixir via Tree-sitter (modules, types, functions, imports/includes, inheritance where syntactic)
- Docs: Markdown, plain text
- Data: JSON, YAML, TOML, CSV (bounded)
- Everything else: honest unknown / partial with metadata and a candidate future provider category
- Connectors today: local filesystem and S3 (no remote Git hosts, web crawl)
- No LLM providers, vectors, or persistent index
- Tree-sitter is syntactic — not compiler-grade semantics
- Git enrichment uses the
gitCLI when available (filesystem scans only; not a pure-Rust Git library) - OKF export is a hand-written markdown+YAML bundle plus
sidecar.json(not theokfcrates.io crate) - No
gnosis.tomlyet — configuration is CLI flags only
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --all -- --check
cargo deny check
cargo run -- scan ./fixtures/mixed-repo --no-tui| Workflow | Trigger | Purpose |
|---|---|---|
| ci.yml | push/PR to main |
fmt, clippy, test (Linux/macOS/Windows), coverage → Coveralls, audit/deny, release build, crates.io package check |
| release.yml | v* tag |
multi-platform binaries, GitHub Release, publish crates to crates.io |
| dependencies.yml | weekly / manual | dependency update PRs |
Secrets / external setup:
- Enable the repo on Coveralls (action uses
GITHUB_TOKEN) - Set
CARGO_REGISTRY_TOKENfor crates.io publish on release tags
Releases: see CONTRIBUTING.md. Push an annotated v* tag matching version in Cargo.toml (publishes the single gnosis crate).
See demo.md.
See CHANGELOG.md for release history.
See CONTRIBUTING.md for development setup, PR expectations, and the maintainer release checklist.
MIT — see LICENSE.