Skip to content

Gotham console rebuild - #2

Merged
AndrewCTF merged 39 commits into
masterfrom
gotham-console-rebuild
Jul 4, 2026
Merged

Gotham console rebuild#2
AndrewCTF merged 39 commits into
masterfrom
gotham-console-rebuild

Conversation

@AndrewCTF

Copy link
Copy Markdown
Owner

No description provided.

AndrewCTF added 30 commits June 22, 2026 10:53
…bar, watchbox, annotations, pattern-of-life

Frontend:
- Drone sensor fog-of-war spotlight that follows the selected sim drone
  (PolygonHierarchy hole); FMV frame now renders real overhead satellite
  imagery (/tiles/sat mosaic) and its telemetry reads the imported Cesium
  module instead of an unset window global (was stuck blank).
- Shared map-draw toolbox (point / polyline / circle) with a live draft,
  exposed as a viewer singleton; powers COP placement, annotations, watchbox.
- Interactive MIL-STD-2525 COP editor (place units, draw FLOT/phase lines,
  range rings) backed by an editable copStore; MilSymbolAdapter re-renders it
  live. Best-effort ontology persistence.
- Cmd+K omnibar: fuzzy jump over live entities + actions (open workspace,
  toggle layer); analyst console moved to Cmd+J.
- Focus-mode right-rail suppression (COP/FMV/targeting) + persistent
  classification banner in the top chrome.
- Annotations graphics layer (threat-coloured point/line/circle + labels);
  geofence/watchbox AOIs with a client-side enter/exit/loiter evaluator that
  pushes alerts; right-click "search around" on the globe; entity-scoped
  pattern-of-life replay with dwell clustering.

Backend (routes/modules with tests):
- Ontology graph store + object/search-around/path routes; watchbox + alerts;
  on-demand imagery chip/change + COG tiler + imagery index; satellite
  tasking; targets; maps; pattern-of-life; MCP internal JWT minting.

Checks: pnpm -r typecheck clean, eslint clean, web 121 tests pass,
api 514 tests pass.
- Postgres RLS migration: classification ladder + compartments + shared flag on
  objects/links/target_board; clearance/roles on profiles; append-only action_log
  (trigger + revoked grants); auditor-gated read; collab_docs table.
- classification.py: US IC ladder, marking strings, can_read predicate.
- security.py: Principal with clearance/roles, least-privilege fallback.
- audit.py + /api/audit: immutable who/what/when/where logging, auditor read.
- /api/extract: LLM document -> ontology objects/links with provenance + markings.
- ontology Object/Link carry classification/compartments/shared columns.
…apshots)

- /ws/collab binary fan-out room (Yjs update + awareness relay, server is a dumb
  relay with 1-byte frame tags + heartbeat).
- /api/collab GET/POST snapshot persistence into collab_docs, RLS clearance-gated.
- collab_docs.state stored as base64 text for PostgREST.
…llab

- security/classification.ts + MarkingBadge/ClassificationBanner: US IC ladder
  mirror, marking pills + banner.
- extract/ExtractPanel: paste text -> /api/extract preview/commit with marking.
- collab/useCollabDoc: Yjs doc bound to /ws/collab relay (sync + awareness) with
  RLS-gated snapshot load/save; adds yjs + y-protocols.
- extract + collab snapshot reject compartments the caller does not hold (403);
  RLS create-ceiling policies now also enforce compartments <@ current_compartments.
- /ws/collab join is clearance-gated: resolve the bearer to a Principal and read
  the doc ACL via the collab_doc_acl SECURITY DEFINER RPC, rejecting under-cleared
  joiners before accept (no service-role secret).
- principal_for_token helper; classification.holds(); 6 new tests.
- Extract tab (left rail): document entity-extraction UI.
- Collab tab (right rail): presence roster + live shared-notes (Yjs CRDT).
- store.py: edition slot (set_edition / get_edition / is_edition_stale / edition_age_s); reset() clears it
- routes/news.py: _refresh_edition under _edition_lock (20 min TTL); _empty_edition fallback; GET /api/news/edition serves cache or kicks a 88 s build; refresh_once now builds the edition too (best-effort, loop-safe)
- auth.py: /api/news/edition added to PUBLIC_PATHS (public + keyless by design)
- tests: test_edition_endpoint_empty_state + test_edition_served_from_cache (2 passed)
Adds stub components for VelocityNewsPage and StoryView (Tasks 7/8 will overwrite these with the real implementations).
…tches

Every cluster becomes a categorized story with image/sources/proofs without an
LLM call, so the page is always full; LLM depth (neutral rewrite, bias/
propaganda flags, actions) is added to the top stories in batched calls so
rate-limiting thins depth, not the story count. Adds a keyword category
classifier.
Serif (Georgia) headlines + Inter body + mono data; utility strip, masthead,
sticky category nav, hero with lead + side stories, per-category card grids
with a larger lead card. Signature Trust Row (corroboration dots + bias-flag
count) on every story. Story view gets a measured column, drop cap, and the
truth-layer callouts (what's wrong / actions / facts / proofs).
…globe resolution

Move the high-volume map layers off per-entity Cesium billboards onto shared
BillboardCollection/LabelCollection primitives (PrimitiveEntityLayer). Aircraft,
vessels, and satellites now keep their Cesium entities only as graphics-less
position/name/property shells, so the watchbox evaluator, histogram facets, layer
counts, and selection-by-id still resolve them — only the pixels move to the
batched collections. This removes the per-frame BillboardVisualizer/LabelVisualizer
walk over ~13k+ contacts that pinned the world view to single-digit FPS while the
GPU sat idle.

- VesselClusterPrimitive replaces Cesium EntityCluster for the world-view vessel
  count bubbles (clustering needs entity billboards, which are now gone); icons
  fade to the bubbles above ~600 km exactly as before.
- Globe renders at native device pixels again. It was undersampling on HiDPI
  displays because useBrowserRecommendedResolution was left at its default (true)
  while resolutionScale capped at 1.5, so a 2x display rendered at 0.75x native.
  Resolution is now adaptive: it drops to 1x device pixels while the camera is
  moving and returns to native when still, so panning stays smooth without
  softening the static view. Adds a render-quality setting (Settings -> Display).
- Exclude the 53 MB offline MapLibre basemap (region.pmtiles) from the Cloudflare
  Workers asset bundle — it exceeds the 25 MiB per-asset limit and blocked deploys.

Also snapshots the in-progress gotham-substrate backend and UI work currently in
the tree.
…churn

- adsb: drop position-stale contacts (seen_pos_s past a 15 min cap) at the
  serve boundary so a frozen OpenSky-cache straggler no longer shows up as
  "44m ago"; refuse stale carry-forward in the snapshot merge. Update the
  keyless-feed unit tests to stub the sync pull path the ingester now uses.
- globe: render conflict / fused-incident / outage points as category icons
  (bombing, shelling, clash, drone, naval, jamming, outage, protest…) with a
  label pill instead of a translucent disc plus text. New eventIcons glyph set
  and eventStyle dispatch, unit-tested.
- normal rail: subscribe to a feed status signature and memoize the status
  dots, so a per-second feed health heartbeat no longer re-renders the whole
  layer rail, data-sources tab, and map-health chrome.
- normal.css: add a shared --sp spacing scale for consistent chrome rhythm.
… the timeline

- search: add GET /api/search/objects — faceted search across the shared
  observation store by type + keyword + drawn-AOI bbox + rolling time window,
  returning matched results plus per-type counts for the facet UI. Core filter
  is a pure function (unit-tested: type/keyword/bbox/antimeridian/time/limit).
- NormalSearchPanel: back the object-search workspace with the faceted endpoint
  instead of the 10-result resolver — object-type dropdown now shows live counts
  per type, AOI circle is sent as a bbox and refined client-side, rolling window
  maps to a since filter. Offset the panel clear of the map toolbar.
- AppRouter: raise the predicted-motion badge above the Normal timeline footer
  so it no longer overlaps the lane labels.
- route: add GET /api/route/candidates — generate on-road + off-road options,
  score each against the live GPS-jamming heat cells (score_route_risk, pure +
  unit-tested), and tag them least-risk / shortest / fastest. Returns the threat
  cells so the client can draw rings. EMI resistance = 100 - jamming exposure.
- RouteOptionsPanel: back the workspace with /candidates — cards show Asset EMI
  Resistance and jamming exposure, the least-risk option is tagged, and the
  selected route draws over translucent severity-coloured threat rings.
The footer cursor was pinned at a hardcoded 62%. Position it from the playback
clock within the trailing 24h window (100% = live/now, moves left as the operator
scrubs or steps back) so it reflects real playback state. Transport, Zulu clock,
and speed presets were already bound; the activity lanes remain illustrative.
- draw: add drawPolygon (multi-click closed ring, right-click/finish commits ≥3
  verts) alongside point/polyline/circle, plus a ray-casting pointInRing helper
  (unit-tested, incl. a concave polygon).
- NormalSearchPanel: AOI is now a circle OR a polygon — the bbox is sent to the
  faceted endpoint and the exact shape (circle radius / polygon ring) refines the
  results client-side.
Event billboards had disableDepthTestDistance = Infinity, so an event on the far
side of the globe rendered on top of the earth (the "seen through the globe"
report). Depth-test them instead, matching the label behaviour — the globe now
hides events on the back hemisphere and shows them on the near side.
AndrewCTF added 9 commits July 3, 2026 23:19
- Bundle OurAirports (5,276) + Natural Earth ports (1,081) as keyless
  reference data; /api/places/{airports,ports} serve them as bbox GeoJSON.
- Search resolves IATA/ICAO codes and ranks places/airports/ports above
  fuzzy vessel-name matches, so LAX no longer hides under ship names.
- Zoom-gated airport/port map layers with FR24-style icons: hidden at
  world view, fetched by viewport when zoomed in.
- SIM placement gains a lat/lon and map-centre entry instead of click-only.
- Search results tagged with kind badges (ARPT/PORT/SHIP/AIR/PLACE).
- Bolder tile-backed conflict/incident event glyphs for legibility.
…ound-recon and graph panels

Shared CoordEntry widget: type a 'lat,lon' or a place/port/airport name or
IATA/ICAO code (resolved via /api/search) to place a point, with a
use-map-centre button — clicking the globe stays as a shortcut. Graph gains
a seed-by-callsign/MMSI/name box so an investigation no longer requires a
map-entity click. DrawController now exposes its viewer so viewer-less panels
can fly and read the map centre.
Left sidebar (LeftIconRail flyout) to search the live object store: View/Draw/
Select modes, object-type dropdown, keyword, an intrinsic-date filter (rolling
window or an absolute start/end range), and four independent regions A-D each
defined by a typed coordinate/place or a drawn circle. Results filter to the
union of the active regions and drop into a click-to-select list; regions render
as circles on the globe. Adds start_s/end_s bounds to /api/search/objects.

SIM placement now gives Launch, Target and each jammer its own always-visible
coordinate field (type a lat,lon or place) instead of one shared arm-then-set
selector.
…ntel build

Frontend
- First-run onboarding tour (apps/web/src/onboarding/Onboarding.tsx): a
  localStorage-gated overlay shown once on the live map routes, walking a new
  analyst through the map, the entity inspector, the analysis tools, and where
  keys/layout/local-AI live. Wired into AppRouter as OnboardingGate; re-openable
  via a "Replay welcome tour" action in Settings.
- Gotham console surfaces: app switcher / left icon rail / app surface shell,
  Explorer app, Inbox, Metrics, Reports, object inspector, capture + control
  layers, saved searches, geo-scope and tasking state.

Backend
- SAR vessel detection sweep (intel/sar_sweep.py) with a honest status ladder
  and size-only military hinting.
- Autonomous watch-officer loop (intel/watch_officer.py) writing cited incident
  briefs, exposed at /api/watch_officer.
- Keyless infra/domain OSINT layer (app/osint + routes/osint.py): dns/whois/ip/
  certs, minting domain:/ip: nodes into the ontology graph.
- Local-inference toggle (routes/ai.py) and VHR imagery hooks.

Docs
- README: add a Prerequisites section (Docker path vs local-dev toolchain with
  versions) and a git-clone-first Quick start; note the in-app tour.

Tooling
- scripts/run-api.sh (jemalloc-preloaded API boot), osint-recon tooling.
- Ignore the generated graphify-out/ knowledge-graph artifact.

690 backend tests pass; web typecheck green.
The old screenshots predated the current console — they showed the earlier
layout, not the layer rail, incident overlay, agent console and entity dossier
you actually get now. Recaptured all four (hero spin, world view, dense Europe,
and an aircraft selected with its track) off a live backend.

Also reworked the setup writeup: dropped the tables for plain prose, and
bumped the stale test badge to the real count.
…phase status

Stopped tracking the things that aren't source — the Palantir research PDFs and
docx, the deep-research and sim-snapshot notes, the eusi base64 dumps — plus the
local agent tooling under .claude/ and the planning-skill output under
docs/superpowers/. All of it is now gitignored so it stays local.

Brought the phase list up to date: replay + the history buffer are in, and
Phase 4 now lists what actually shipped (MCP + intel API, SAR dark-vessel
detection, the watch-officer, the OSINT layer, local inference, onboarding).
Rewrote the punctuation to plain sentences, renamed the "What it is, and what
it isn't" section to "Scope and limits", and reworded the VRAM tier note so it
states what was measured without the throwaway framing. Kept the numeric ranges.
Self-host and local builds have no Supabase env, so there's nothing to sign
into. The account chip now returns null when auth is unconfigured, matching the
AuthNotice banner which already short-circuits on the same flag. Hosted builds
(Supabase set) are unchanged and still show sign in / account.
@AndrewCTF
AndrewCTF merged commit 4481358 into master Jul 4, 2026
0 of 2 checks passed
@AndrewCTF
AndrewCTF deleted the gotham-console-rebuild branch July 4, 2026 16:06
AndrewCTF added a commit that referenced this pull request Aug 8, 2026
…ut it

The map toolbar was three runs of unlabelled icons separated by a hairline. The
redesign spec's own §2 says Gotham labels its toolbar groups, and says why: an
icon with a tooltip is exactly the reachable-but-invisible pattern two persona
waves kept reporting. A hairline says these three are different. It does not say
what any of them is for. They are Tools, Draw and View now, in the same 40px
column the buttons already use, so naming them costs the map no width.

What this does NOT do, recorded rather than left to be discovered: that same
spec's structural correction #2 says Gaia's map toolbar is a short horizontal
cluster at the top-left of the map, about 30px tall, not a vertical rail down
one side. Relocating it collides with the category legend that already owns the
top-left and moves the anchor for the measure and area popovers, so it is a
separate change with its own verification rather than a side effect of adding
three labels.

Also checked while here, because the operator's complaint pointed at them: Find
reports 246 rendered characters in the reachability report and is not thin, it
is idle. It is 432 lines with type and window facets, distance ranking and
coordinate parsing, and 246 characters is what its empty state weighs before
anyone has typed. Same for Series, which is wired to MetricsPanel and the
archive card. Neither is unported.
AndrewCTF added a commit that referenced this pull request Aug 8, 2026
…ut it

The map toolbar was three runs of unlabelled icons separated by a hairline. The
redesign spec's own §2 says Gotham labels its toolbar groups, and says why: an
icon with a tooltip is exactly the reachable-but-invisible pattern two persona
waves kept reporting. A hairline says these three are different. It does not say
what any of them is for. They are Tools, Draw and View now, in the same 40px
column the buttons already use, so naming them costs the map no width.

What this does NOT do, recorded rather than left to be discovered: that same
spec's structural correction #2 says Gaia's map toolbar is a short horizontal
cluster at the top-left of the map, about 30px tall, not a vertical rail down
one side. Relocating it collides with the category legend that already owns the
top-left and moves the anchor for the measure and area popovers, so it is a
separate change with its own verification rather than a side effect of adding
three labels.

Also checked while here, because the operator's complaint pointed at them: Find
reports 246 rendered characters in the reachability report and is not thin, it
is idle. It is 432 lines with type and window facets, distance ranking and
coordinate parsing, and 246 characters is what its empty state weighs before
anyone has typed. Same for Series, which is wired to MetricsPanel and the
archive card. Neither is unported.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant