Repository navigation
Getting Started
- Python 3.14, the version used for the verified pinned environment.
- Node.js satisfying the installed Vite requirement:
^20.19.0 || >=22.12.0, plus npm. - The commands below target Windows/PowerShell. Linux CI checks have also been exercised locally; see TASKS.md for their scope.
- Cloud inference avoids running a local model. Hardware performance has not been benchmarked; the project targets a modest workstation.
-
Clone the repository:
git clone https://github.com/KrishnenduOfficial/triagelens.git cd triagelens
-
Create and activate a virtual environment (PowerShell):
python -m venv .venv .\.venv\Scripts\Activate.ps1On a POSIX shell, activation is
source .venv/bin/activate. -
Install dependencies:
python -m pip install -r requirements.txt
-
Configure environment variables:
if (-not (Test-Path -LiteralPath .env)) { Copy-Item .env.example .env }
Edit
.envto configure your API keys and model parameters:GROQ_API_KEY=your_groq_api_key_here GROQ_MODEL=openai/gpt-oss-120b GROQ_FALLBACK_MODELS= DATABASE_PATH=data/triagelens.db VIRUSTOTAL_API_KEY= ABUSEIPDB_API_KEY= URLHAUS_API_KEY=
Groq needs a real key for live triage. Intelligence credentials are optional: enter real values for services you use and leave unused credentials blank, including
URLHAUS_API_KEYin the template. Example placeholder strings are treated as configured credentials; they are not working API keys. (Note:TRIAGELENS_DB_PATHis also recognized as an alias forDATABASE_PATH, andVT_API_KEYas an alias forVIRUSTOTAL_API_KEY).VIRUSTOTAL_MIN_DETECTIONSdefaults to 3 malicious engine detections; set it to a positive integer to adjust the enrichment flag. A result below the threshold, including one or two detections, does not prove an indicator is safe.ENRICHMENT_WAIT_MODEdefaults tofalse; set it totrueto allow rate-limit waits bounded byENRICHMENT_MAX_WAIT_SECONDS.ENRICHMENT_MAX_LOOKUPS_PER_RUN=0is parsed as no cap, but the environment variable is currently unused by the runtime, even when set to a positive value. Use the CLI--max-lookupsoption to enforce a per-run lookup cap.CLI, API, and the evaluation runner share database resolution: explicit CLI override, nonblank
DATABASE_PATH, nonblankTRIAGELENS_DB_PATH, thendata/triagelens.db. Relative paths use the project root containingsrc/, regardless of the process working directory; absolute paths retain their target. Parent directories are created consistently. Blank explicit overrides and filesystem errors fail before SQLite opens. Windows drive-only/root-only paths must be replaced with fully qualified or ordinary project-relative paths.:memory:explicitly selects an ephemeral database local to each connection.--db-pathaffects only that CLI/evaluation invocation. Configure the API with the same target and restart it after changing configuration. Existing databases are not moved or merged. The historical roottriagelens.dbis separate from the defaultdata/triagelens.db; to select it, useDATABASE_PATH=triagelens.dbconsistently for both processes. A genuinely empty selected database still produces zero metrics. See the path verification.The template also documents
GROQ_REQUESTS_PER_MINUTE,GROQ_TOKENS_PER_MINUTE,GROQ_ESTIMATED_TOKENS_PER_REQUEST,GROQ_MAX_WAIT_SECONDS,DATABASE_BACKEND,YARA_SCAN_ROOTS,ALLOWED_ORIGINSandVITE_API_BASE_URL. See .env.example for defaults and optional PostgreSQL settings. Never commit real credentials.Troubleshooting: Missing Groq credentials prevent live triage; provider quotas and timeouts can yield unavailable results. An empty dashboard can indicate an empty database or a different API database path. Check the API health endpoint, the configured path and browser origin, then restart the API after configuration changes. A missing enrichment response is not a clean finding.
Run these from the project root with the virtual environment active. Live CLI runs can contact both Groq and configured intelligence services and consume quota.
-
Prepare a dedicated synthetic alert input folder:
New-Item -ItemType Directory -Force data/raw/demo | Out-Null if (Test-Path -LiteralPath data/raw/demo/telemetry.json) { throw 'Demo input already exists; choose another directory or review it before replacing it.' } Copy-Item -LiteralPath data/synthetic_soc_telemetry.json -Destination data/raw/demo/telemetry.json
data/raw/is Git-ignored. Keep alert inputs separate from the benchmark JSON and databases; do not feed the entire mixeddata/directory into alert intake. -
Run the pipeline:
python -m src.cli --data-dir data/raw/demo --db-path data/triagelens.db
Save the
Evaluation run:UUID and the finalPIPELINE EXECUTION SUMMARYfor the comparison below. A configured Groq key and completed triage outcomes are needed for a meaningful AI metric comparison; missing credentials skip triage. This command makes live requests and may consume quota; one run of this synthetic input produced 50 timelines, potentially requiring 50 triage requests before retries. Check your available quota before starting. To skip Groq while retaining enrichment:python -m src.cli --data-dir data/raw/demo --db-path data/triagelens.db --skip-llm
--skip-llmis not an offline switch: intelligence lookups can still occur. Use the guarded test suite for offline verification. -
Check the database populated by that run:
python -c "import sqlite3; c=sqlite3.connect('file:data/triagelens.db?mode=ro', uri=True); print('timelines:', c.execute('SELECT COUNT(*) FROM timelines').fetchone()[0], 'evaluations:', c.execute('SELECT COUNT(*) FROM evaluations').fetchone()[0]); c.close()"
The read-only open fails if the file is missing. Existing rows can be from earlier runs; the Run selection below isolates this run. A zero evaluation count does not demonstrate populated AI metrics.
-
Start the backend in a separate terminal, from the project root:
.\.venv\Scripts\Activate.ps1 $env:DATABASE_PATH = 'data/triagelens.db' python -m uvicorn src.api.main:app --host 127.0.0.1 --port 8000 --reload
Configure the API's
DATABASE_PATHto the same file as the CLI. Openhttp://127.0.0.1:8000/docsfor the API or/api/metricsfor stored metrics. -
Start the dashboard in another terminal, initially at the project root:
cd dashboard npm ci npm run dev
Open the local URL printed by Vite (normally
http://localhost:5173). The default backend URL ishttp://localhost:8000. For a different backend, put the public URL indashboard/.env.localbefore starting Vite:VITE_API_BASE_URL=http://127.0.0.1:8000
Vite uses the dashboard directory for its environment files; setting this only in the root
.envdoes not configure the frontend. Restart Vite after changes. Groq and intelligence secrets stay in the root.env, never inVITE_*values.
- Open
/healthon the API and verify database connectivity. - Open
http://127.0.0.1:8000/api/metrics?scope=run&run_id=<UUID>, replacing<UUID>with the CLI'sEvaluation run:value. Comparetp,tn,fp,fn,total,unknown,abstainedandunavailablewith the CLI summary; compare applicable rates after converting API fractions to percentages. - On dashboard
/, select Run under Scope, enter the same UUID in Run UUID, and click Apply selection. Verify the displayed Run ID, then compare confusion counts, binary-labeled results, unscored outcomes, undetermined dispositions, unavailable outcomes and applicable rates with the API and CLI. Default History includes earlier evaluations and need not match the latest run. N/A can be valid when a rate's denominator is zero; the CLI may print 0% instead. Check loading/error states; the UI does not certify live provider availability. - Use the timeline list on
/and follow a row to/timeline/:id. Verify its alert details, intelligence and stored verdict. There is no/timelinespage or incident-detail drawer in the current router. - On
/hunt, tryevent_type == "ProcessCreate"orCommandLine contains "powershell". A valid query may return zero matching rows. - Check that failed triage is unavailable rather than a benign prediction and that historical model results remain distinguishable from new Groq results.
- Record whether the same-run numbers match and share screenshots of the CLI summary and dashboard selection/results, with sensitive information removed. Do not claim this check passed until the operator confirms it on populated data.