v1.10 — Security & reliability hardening

What's Changed

Security fixes

• SSRF protection: Intel enrichment now validates target IPs before fetching
• Shell injection prevention: Port values and remote addresses are sanitized before use
• Safe temp files: Installer uses mkstemp instead of predictable /tmp paths
• API key protection: Intel API keys moved from query strings to headers

Reliability fixes

• Race conditions eliminated: SQLite writes serialized with mutex, table creation uses sync.Once
• Process cleanup: journalctl and tail processes properly waited on exit
• Concurrent enrichment: IP enrichment runs in parallel instead of sequentially
• Error handling: Backfill errors logged, parseTime failures surfaced, pip install errors caught

Installer improvements

• Socket-activated sshd: Handles Ubuntu's ssh.socket correctly
• SSH verify gate: Checks admin SSH access before switching ports
• Port validation: Rejects invalid port numbers early
• Duplicate config fix: Handles existing [ssh] section in cowrie.cfg

Other

• Removed obsolete fix-shardlure-py.sh script
• Added config validation test coverage
• Fixed playbook rune counting for multi-byte characters

Full Changelog: v1.9...v1.10

v1.9 — Dragon: hardening, full-window intel & data-consistency

v1.9 — Dragon: hardening, full-window intel & data-consistency

A large correctness, performance, and feature release. Headlined by a sweep that made every dashboard widget draw from authoritative full-data queries (or honestly disclose when it's showing a capped sample), plus seven-provider IP enrichment and a proper uninstaller.

Threat intel

• Seven IP-reputation providers (was three): added Shodan InternetDB (keyless), AlienVault OTX, IPQualityScore, and IPinfo alongside AbuseIPDB / VirusTotal / GreyNoise. Parallel lookups, normalized verdict + score + tags, 24h cache. See the new IP Reputation Enrichment README section for env keys.
• Full-window analytics. MITRE, TTP, IOC export, threat graph, deobfuscation, and wordlists now analyze the entire selected window instead of a recent ~5,000-event sample — a "30d" view is really 30 days. Capped widgets (graph nodes, sessions, payload library) now disclose "N of M" instead of silently truncating.

Dashboard correctness

• Attack Geography / By-country now reflects true hits-by-country across all events (a dominant high-volume attacker can no longer vanish from the chart); both pages share one authoritative aggregation.
• Top Actors / Brute-Force Radar sort by real volume/rate from the DB (not a recent-actor slice).
• Honest counts everywhere: payload "unique", session totals, country counts, pending-uploads — all reconciled with the database.
• Dashboard auth token is now forwarded on every API request and across page navigation (the dashboard no longer 401s its own data when SHARDLURE_DASH_TOKEN is set), with a loud warning when no token is set.

Performance (1-vCPU friendly)

• Incremental cowrie actor rebuild — a live ingest tick is now O(touched actors), not O(all history) (RSS dropped from ~2.7 GB to ~40 MB on the reference box).
• Indexes for the dashboard/artifact hot paths; single-read payload classification; dedup-before-copy on capture; cached country aggregation.
• Writes are serialized by an application mutex (single SQLite writer) without capping the connection pool, so analytics reads run concurrently and never stall live ingest.

Security / hardening

• SSRF guard now blocks unspecified (0.0.0.0/::), CGNAT (100.64/10), and other reserved ranges the stdlib predicates miss; path-containment for cowrie download archiving.
• CSV/IOC export neutralizes spreadsheet formula injection (CWE-1236) on attacker-controlled fields.
• Frontend XSS pass (escaped TTP tags / session fields; safeUrl blocks javascript:/data: schemes).
• admin_ips supports CIDR ranges (e.g. a Tailscale CGNAT range); config written 0600 (it can hold the abuse.ch key).

Installer / ops

• uninstall [--purge] command — reverses the full install in lockout-safe order (SSH restored first), reading the persisted config for accurate firewall/teardown. Documented alongside a step-by-step setup guide in the README.
• Release binaries are now version+commit stamped.

New widgets

• Red Team Pending uploads panel (only not-yet-shared, MalwareBazaar-eligible payloads).
• Overview uptime stat.

Full diff: v1.6...v1.9.

Installer

• Hand-holding setup from the basics: the installer now greets you with what it will do, helps you paste in an SSH public key if the server doesn't have one yet (installing it with correct perms), and pauses at a verify gate to confirm ssh -p <admin-port> works before continuing — so a fresh-VPS user can't get locked out. Ports are validated; uninstall reads the persisted config for accurate teardown.

v1.6 — Dragon

What's new

Dragon Theme

Complete visual overhaul of both dashboard pages. Charred obsidian background, blood-red/molten-gold accent palette, Chakra Petch typography, flat panels (no glass-morphism), sharp 3-4px radii, and a 52px sidebar navigation rail with live indicator.

New Dashboard Widgets (Overview tab)

• Threat Level Gauge — SVG arc gauge (green → gold → red) computed from event volume, actor count, and deploy-intent ratio
• Attack Geography — top 10 countries with flag emoji and heat-colored bars
• Top Credentials — most-tried passwords and usernames at a glance
• Brute-Force Radar — hottest IPs ranked by attempts/hour, gold-to-crimson heat bars
• Live Attack Timeline — scrolling feed of real-time events, color-coded by kind, polls every 3s

MalwareBazaar Dashboard Integration

• Red Team tab widget showing upload history, family classification (Mirai, Komari, RedTail, XMRig, Traffmonetizer, etc.), pending counts, and status badges
• One-click upload from the payload inspector modal — no CLI required
• New API endpoints: GET /api/intel/bazaar, POST /api/intel/bazaar/upload, GET /api/intel/timeline

Persistent Geo Cache

IP geolocation results are now stored in SQLite (ip_enrichment table with source=geo). Lookups survive server restarts with a 7-day TTL. No more "resolving…" on every page load. Negative misses are memoized in-memory for 5 minutes to avoid hammering the DB.

Performance Optimizations

• bazaar.Classify() results cached in sync.Map by SHA256 — eliminates disk I/O per row per request
• ensureEnrichmentTable() DDL hoisted to server init — eliminates 2N round-trips per cold render
• Geo cache negative-miss memoization prevents redundant DB queries for unresolved IPs

Bug Fixes

• capPulse/capFade animations now pulse green (matching the capture badge) instead of red
• Geography widget: fixed 4-child → 3-column grid mismatch causing broken row layout
• Timeline polling: added active-tab guard to prevent 3s DB hammering from inactive tabs
• Upload button: "skipped" status now shows amber instead of misleading green checkmark
• Error responses from bazaar upload endpoint now return proper application/json Content-Type
• Fixed old palette straggler rgba(255,196,87) in verdict-suspicious badge
• Added nil guard in timeline handler for defensive safety
• Pending count query errors are now logged instead of silently swallowed
• Fixed var(--muted) bug in globe page session rows (was undefined, now uses var(--dim))

Upgrade

Binary drop-in replacement — no migration needed. The geo cache table is created automatically on first use.

# Build
make build
# Or cross-compile for ARM64 VPS
GOOS=linux GOARCH=arm64 go build -o shardlure-arm64 ./cmd/shardlure

# Deploy
sudo cp shardlure-arm64 /usr/local/bin/shardlure
sudo systemctl restart shardlure-live

For one-click MalwareBazaar uploads from the dashboard, set SHARDLURE_BAZAAR_KEY in your systemd unit or environment.

v1.3 — MalwareBazaar sharing

v1.3 — share cowrie payloads to MalwareBazaar

Adds shardlure share bazaar for uploading captured payloads to
abuse.ch MalwareBazaar, with automatic
family/architecture classification and sha256-based dedup.

New

• internal/intel/bazaar package: multipart API client with Auth-Key
  support, family classifier (RedTail, Komari, Traffmonetizer, Mirai,
  Gafgyt, SSHScanner, XMRig, c3pool), ELF arch detection
  (x86-64 / i386 / aarch64 / arm / mips / ppc), shell + python
  headerless fallback.
• shardlure share bazaar CLI:
  • --dry-run preview classification without uploading
  • --limit N cap upload count (0 = unlimited)
  • --sha <sha256> upload a single specific sample
  • --since <duration> change the fresh-sample window (default 240h)
  • --status list previous uploads from bazaar_uploads
  • --comment <str> override the per-sample comment
  • --endpoint <url> override the API endpoint
  • --anonymous submit without Auth-Key (not recommended)
• Store: bazaar_uploads table (migration v5) with self-healing
  ensureBazaarUploadsTable; new ArtifactsForShare query filters
  on size, freshness, and dedup.
• Config: intel.bazaar block (api_key, tags, max_bytes,
  freshness_days).
• 14 new tests covering API, classifier, and orchestrator paths.

Fixed

• version subcommand previously printed a hardcoded 0.1.0
  regardless of the released tag. Binaries now report the real
  version and commit, e.g. shardlure v1.3 (commit 5256a5b).

Operational notes

• Requires a registered auth.abuse.ch
  account and Auth-Key in intel.bazaar.api_key. The yaml should be
  root-readable only (chmod 600).
• Manual subcommand only — no automatic background upload.
• file_already_known responses are treated as accepted and recorded
  for dedup; no_api_key and user_blacklisted halt the batch.
• bazaar_uploads is exempt from MaintenancePurge retention,
  so the dedup set remains complete across the 24h cycle.

Pre-release smoke

Smoke-tested on a production cowrie VPS: 26 fresh samples uploaded
(11 inserted, 15 file_already_known) before tagging.

Verifying downloads

sha256sum -c SHA256SUMS

Upgrading from v1.1.1

Drop the new binary in place and restart the shardlure-live unit.
Migration v5 runs on first start and is idempotent.

v1.1.1 — retention purge fix

Patch release fixing a critical bug in v1.1's data retention loop.

What broke in v1.1

MaintenancePurge (the 24h retention sweep introduced in v1.1) silently failed on every cycle:

1. Wrong column name — DELETE FROM ip_enrichment WHERE queried_at < ? referenced a column that does not exist. The actual column is fetched_at. The transaction rolled back, no rows were purged.
2. Lazy-created tables — ip_enrichment, cowrie_tty_index, and artifacts are created on first write. On a brand-new install the first purge (fired at startup) hit no such table: ip_enrichment and never recovered.

Net effect: every v1.1 install kept growing without bound. The whole point of fix #5 in v1.1 was defeated.

Fix

• Pre-create the three lazy tables at the start of MaintenancePurge.
• Use the correct fetched_at column.
• New test (internal/store/purge_test.go) covers fresh-DB, populated-DB row deletion across all four purged tables, and the zero/negative no-op short circuit.

Other changes

• migrate_test.go fresh-open assertion bumped v>=3 → v>=4 (stale since the v4 ladder step shipped in v1.1).
• install.sh SHA256SUMS lookup now uses anchored grep -F + awk $2==b so future suffixed asset names (e.g. -musl) cannot collide.

Upgrading from v1.1

curl -fsSL https://raw.githubusercontent.com/hett-patell/ShardLure/main/scripts/install.sh | sudo bash

Or just download the binary for your arch from the assets below and systemctl restart shardlure-live.

v1.1 — payload dedup, memory caps, diagnostics

Maintenance release. Drop-in upgrade from v1 — no config changes, no schema breaks (migration v3 runs automatically on first start).

Payload library: deduplicated by SHA-256

The payload library previously rendered one row per artifact INSERT, so a single binary delivered via N rotated URLs from M source IPs appeared as N visually identical rows. In a 7-day window the table was 200 mostly-duplicate rows for ~80 unique payloads (the top SHA, an Outlaw SSH key, alone accounted for 462 of 893 captures).

• store: new ListArtifactsAggregatedSince returns one row per unique sha with occurrences, urlCount, ipCount, actorCount, sessionCount, plus the most-recent capture metadata as Last{URL,SrcIP,Actor,Session,TS}. Portable SQL: CTE + correlated subqueries, no window functions.
• /api/intel/payloads: response keeps url/actor/session/srcIp/ts keys (now last-seen) for backward compatibility and adds firstTs, occurrences, urlCount, ipCount, actorCount, sessionCount.
• /api/intel/payloads?limit=>1000 now clamps to 1000 instead of silently falling back to 200.
• /api/intel/sessions accepts ?limit= (default 200, ceiling 2000) instead of being hard-coded at 200 — the session panel was losing ~367 of 567 command-bearing sessions in a 7-day window.
• Dashboard: payload table gains a delivery column showing Nx · Mip; the meta line reads N unique · M captures · Wh window so the dedup story is visible at a glance.

Memory: bound the live collector + geo cache

Across ~2.5 days of uptime, RSS peaked at 1.6 GB. The cause was two unbounded in-memory structures: the live SSH journal collector accumulated per-IP user maps across thousands of attacker IPs × hundreds of distinct usernames each, and the web geo-resolver cache never evicted.

• actor.liveJournalCollector replaces the unbounded collector:
  • LRU cap of 4096 distinct source IPs.
  • Per-IP user-map cap of 256 keys; overflow rolls into an _overflow_ synthetic bucket so aggregate counts stay accurate.
  • Idle-TTL sweep at 12h (opportunistic, ≤4 evictions per insert).
  • DB hydration on re-entry: when an evicted IP returns, store.LoadJournalIPStats(ip) loads persisted counters before the next upsert, so the running total is preserved across in-memory eviction cycles.
• web.geoResolver.cache becomes an LRU capped at 4096 entries (override via SHARDLURE_GEO_CACHE_MAX), with promotion on hit and opportunistic expiry sweep on insert.

Expected ceiling at saturation is now in the low tens of MB rather than unbounded. Verified on the Oracle ARM honeypot: sys=29.8 MB, heapAlloc=2–3 MB, 15 goroutines, flat across sampling.

Diagnostics endpoints

All gated behind the existing dashboard token (SHARDLURE_DASH_TOKEN); same auth as the rest of /api/intel/*.

• /debug/pprof/{,cmdline,profile,symbol,trace} from net/http/pprof.
• /debug/runtime returns a JSON snapshot of runtime.MemStats plus bounded-cache sizes (liveJournalCollectorIPs, geoCacheEntries, etc.).
• HTTP WriteTimeout raised from 20s to 60s so /debug/pprof/profile?seconds=30 completes.

Other

Carried forward from the live-capture branch since v1:

• web: TTY transcripts decoded and surfaced in the intel session modal; shell-session panel populated from live capture; command-column scrub for noisy CRLF.
• store: migration v3 normalizes command to '' instead of NULL so downstream JOINs/aggregations don't drop rows.
• scripts/install.sh: auto-detects architecture, downloads the matching binary from this release, and provisions the systemd unit. --tag overrides the auto-resolved latest.

Install

curl -fsSL https://github.com/hett-patell/ShardLure/releases/download/v1.1/install.sh | sudo bash

Or pin explicitly:

sudo TAG=v1.1 bash -c "$(curl -fsSL https://raw.githubusercontent.com/hett-patell/ShardLure/v1.1/scripts/install.sh)"

Upgrade from v1

Stop the service, replace the binary, restart:

sudo systemctl stop shardlure-live
sudo install -m 0755 shardlure-linux-$(uname -m | sed s/x86_64/amd64/;s/aarch64/arm64/) /usr/local/bin/shardlure
sudo systemctl start shardlure-live

Migration v3 runs automatically. No config changes required.

Full changelog

f8890db actor/web: bound live collector + geo cache, gate pprof behind dash token
4166e1c web/store: aggregate payload library by sha256
658bf67 web: surface decoded TTY transcripts in the intel session modal
6001fbe store: migration v3 sets command='' instead of NULL
1e17e5b live capture: TTY transcripts, shell-session panel, command-column scrub
2ec42ef foundation: add CONTRIBUTING, SECURITY, COC + README badges and ecosystem footer
95056ef add install.sh: auto-detect arch, download binary from release, setup systemd

v1 — first stable release

First stable release of ShardLure.

Included in this release:

• SSH honeypot telemetry analysis (journald + Cowrie)
• Actor clustering by playbook fingerprint / HASSH
• Intent classification (probe, proxy, deploy, mixed)
• Web dashboard with 3D globe arcs
• Terminal TUI dashboard (Bubble Tea)
• Intel dashboard: session timeline, MITRE ATT&CK grid, TTP harvesting, IOC export (STIX 2.1 + CSV), threat intel enrichment (AbuseIPDB / VT / GreyNoise), credential wordlists, infrastructure pivot graph, payload inspection, bash session replay, command deobfuscation
• Live tail mode with streaming ingest
• Quarantined payload capture with SSRF-safe fetcher
• SQLite storage with migration ladder

Binary attached: linux/arm64