GovPeer

Federated attack surface monitoring for government and K-12 education organizations.

Members scan their own public-facing networks and voluntarily contribute findings to a shared, peer-validated commons. Unlike Shodan or Censys, no unsolicited scanning occurs — every scan is authorized by the network owner.

See GOVPEER_SPEC.md for the full technical specification.

Status

Pilot-grade, feature-complete PoC. All seven specified phases are implemented. scripts/poc_check.sh is a smoke test covering 9 of ~60 test files; full coverage comes from the complete test suite.

• 1421 tests passing — 1247 backend (pytest) + 174 frontend (vitest)
• Sensor + Aggregator APIs, Celery pipelines, three React UIs
• Multi-aggregator federation (peer registration, signed sync, reconciliation) — beyond the v1 spec scope
• Stripe-backed vendor commercial portal with tier-gated APIs and admin claim review

This is ready for a controlled pilot deployment with a small federation of trusted organizations. It is not yet hardened for open production use — see Maturity & known limitations below.

See PROGRESS.md for the per-phase task breakdown.

Maturity & known limitations

GovPeer's crypto and trust foundations are solid:

• Ed25519 signatures verified at every submission boundary — see tests/test_crypto.py and tests/aggregator/test_data_api.py
• Hash-chained trust ledger with append-only guarantees — see tests/aggregator/test_ledger_service.py and tests/aggregator/test_trust_api.py
• Federation sync with replay-protection nonces and signed payloads — see tests/aggregator/test_federation_sync_service.py and tests/aggregator/test_federation_api.py
• All three React UIs render and pass their vitest suites (174 frontend tests)

Known gaps that must be addressed before production use:

Limitation	Detail
Per-endpoint tier gating	Sharing-tier enforcement is applied per route handler, not via global middleware. There is no central policy layer ensuring all new endpoints are automatically gated. Tracked in #92.
Default JWT secrets	GOVPEER_AGGREGATOR_JWT_SECRET and GOVPEER_AGGREGATOR_VENDOR_JWT_SECRET default to CHANGE-ME-IN-PRODUCTION / CHANGE-ME-VENDOR-JWT-SECRET. These must be replaced with strong random secrets before any networked deployment. See govpeer/common/config.py.
No load or chaos testing	The test suite is unit/integration only (in-memory SQLite). There is no load test, stress test, or chaos/fault-injection harness. Behavior under high scan volume or database failover is untested.
Sensor live-scan demo gap	The Masscan/Nmap scan pipeline is wired and the Celery tasks are tested in isolation, but there is no end-to-end demonstration of a real live network scan flowing through to the aggregator UI. Tracked in #94.
No formal security audit	The codebase has not undergone a third-party penetration test or formal security audit. The crypto primitives (PyNaCl/libsodium Ed25519) are well-reviewed externally, but the application layer around them has not been audited.
No operator runbook	There is no documented runbook for production operations: key rotation, certificate renewal, database backup/restore, incident response, or upgrade procedures.

Recommended deployment scope: a closed pilot with a small number of trusted organizations (e.g., a single school district consortium) where operators can monitor closely and harden incrementally.

Capabilities at a glance

Surface	Capabilities
Sensor (local Docker deployment)	Masscan port discovery → Nmap fingerprinting → vendor/product fingerprinting → delta detection → local alert evaluation → signed submission to aggregator
Aggregator	Org registration with DNS-verified domains, Ed25519-signed trust ledger, trust-circle vouching, buddy-pair escalation, community benchmarks, CVE correlation against NVD + CISA KEV, vendor scorecards, Stripe-backed vendor portal, multi-aggregator federation
Sensor UI :8401	Local dashboard, scan results, deltas, alerts, notification config, identity bootstrap
Aggregator UI :8501	Org registration, attestations, trust ledger, circles, community dashboards, CVE feed, vendor directory, alerts, federation peers
Vendor Portal UI :8502	Vendor login, tier-gated benchmarks, product analytics, CVE impact, subscription management

How it maps to CIS Critical Security Controls v8

Strong/partial perimeter coverage of: C1 Asset Inventory · C2 Software Inventory · C7 Continuous Vulnerability Management · C8 Audit Logging (federation events) · C12 Network Infrastructure · C13 Network Monitoring & Defense · C15 Service Provider Management · C17 Incident Response · C18 External Penetration Testing.

Out of scope: endpoint/EDR controls, account/access management, malware defenses, data protection, backup, security awareness.

Architecture

Sensor (local)          Aggregator (central)         UIs
─────────────          ────────────────────         ────
FastAPI :8400          FastAPI :8500                Sensor UI     :8401
Celery worker          Trust ledger                 Aggregator UI :8501
Celery beat            Alert engine                 Vendor Portal :8502
PostgreSQL :5432       CVE correlation
Redis :6379            Stripe billing
Masscan + Nmap         TimescaleDB :5433
                       Federation sync

Prerequisites

• Python 3.12+
• Node.js 20+ (for the UIs)
• Docker & Docker Compose (for databases, or full-stack mode)
• PostgreSQL 16 and Redis 7 (if running databases natively instead of Docker)

Quick Start (Docker Compose)

Full demo with populated dashboards

Builds, starts, and seeds the entire 12-service stack with realistic demo data:

./scripts/demo.sh

This gives you three UIs with data on first load:

Service	URL	What you'll see
Sensor UI	http://localhost:8401	Scan timeline, service distribution, deltas
Aggregator UI	http://localhost:8501	Community benchmarks, trust ledger, org directory
Vendor Portal	http://localhost:8502	Vendor registration, product scorecards
Admin Panel	http://localhost:8501/admin	Approve/reject vendor product claims

API docs: Sensor · Aggregator

Other demo commands:

./scripts/demo.sh --seed   # re-seed only (services already running)
./scripts/demo.sh --down   # stop all services

Minimal start (no seed data)

cd docker
docker compose up --build

Notes:

• Docker quick start sets a default sensor org ID (00000000-0000-0000-0000-000000000001) so API requests can run without manually sending X-Org-Id.

Demo Seed Data

./scripts/demo.sh (or python scripts/seed_data.py) populates both databases so dashboards aren't empty.

Sensor DB

Data	Count	Details
IP ranges	3	10.0.0.0/24, 172.16.0.0/24, 192.168.1.0/24
Scan jobs	5	Completed over past 14 days (8→12 hosts discovered)
Scan results	~46	Varied services (https, ssh, rdp, smtp, dns), TLS info, vendor fingerprints
Scan deltas	15	2 critical (RDP opened, TLS expired), 4 warning, 9 info

Aggregator DB

Data	Count	Details
Organizations	6	"Pitt County Schools" (tier 3) + 2 peers + 3 vouchers — all NC K-12
Attestations	5	Peer orgs vouching for the demo org with signed payloads
Trust ledger	10	Hash-chained events: applied → verified → attested → activated → tier changes
Scan submissions	3	One per main org
Aggregator results	60	20 per org — all 12 seeded vendors represented
Circle	1	"NC K-12 School Districts" with all 3 main orgs
Vendors	12	PowerSchool, Fortinet, Cisco, Palo Alto, pfSense, etc. (with products + fingerprint rules)

Seeding is idempotent — running it again skips existing data.

Federation Demo

Stand up two fully independent aggregator regions and cross-register them as federation peers so you can exercise the aggregator-to-aggregator sync path.

1. Start the federation topology

docker compose -f docker/docker-compose.yml \
               -f docker/docker-compose.federation.yml \
               up --build

This starts all services from the base compose file plus:

Service	Port	Description
aggregator-2	:8510	Second aggregator region
aggregator-db-2	:5434	TimescaleDB for region 2
aggregator-2-celery-worker	—	Celery worker for region 2

Both regions share the existing Redis instance on :6379.

2. Wait for both aggregators to be healthy

curl -s http://localhost:8500/health   # region 1 → {"status":"ok"}
curl -s http://localhost:8510/health   # region 2 → {"status":"ok"}

3. Cross-register the two regions as federation peers

AGGREGATOR1_BASE_URL=http://aggregator:8500 \
AGGREGATOR2_BASE_URL=http://aggregator-2:8510 \
python scripts/seed_federation_peers.py

The script generates an Ed25519 keypair for each region and inserts a FederationPeer row in the opposite database. It is idempotent — running it again leaves existing rows unchanged.

The script logs the private keys that must be set if you want each aggregator to sign outbound sync requests:

Set GOVPEER_AGGREGATOR_FEDERATION_PRIVATE_KEY=<base64-key-for-region-1>
Set GOVPEER_AGGREGATOR2_FEDERATION_PRIVATE_KEY=<base64-key-for-region-2>

4. Verify the peer list via the API

# Peers known to region 1
curl -s http://localhost:8500/api/v1/federation/peers | jq '.[].base_url'

# Peers known to region 2
curl -s http://localhost:8510/api/v1/federation/peers | jq '.[].base_url'

5. Trigger a federation sync (region 1 → region 2)

# Replace <peer-id> with the UUID returned in step 4
curl -s -X POST http://localhost:8500/api/v1/federation/peers/<peer-id>/sync | jq

Tear down

docker compose -f docker/docker-compose.yml \
               -f docker/docker-compose.federation.yml \
               down -v

Running a Live Scan

Once the stack is up (see Quick Start above), scripts/demo_scan.sh walks through the full sensor pipeline end-to-end against a target you own:

./scripts/demo_scan.sh                        # scan localhost (default)
./scripts/demo_scan.sh 10.0.0.0/24           # scan a CIDR you own
./scripts/demo_scan.sh 10.0.0.0/24 top_100   # fastest: top-100 ports only

The script runs six steps automatically: it verifies the sensor and aggregator are reachable, registers the target IP range, starts a scan job with your chosen profile (top_100, top_1000, or dangerous_only), polls until the Masscan discovery + Nmap fingerprinting pipeline completes, prints results and delta detections (new or changed services vs. the prior scan), then checks that the signed submission landed in the aggregator's trust ledger. If Masscan fails due to missing raw-socket privileges, the script prints the exact docker compose command needed to restart the sensor container with --privileged or explains how to set MASSCAN_DISABLE=1 for nmap-only mode.

Authorization reminder: GovPeer never scans assets without operator consent. Only scan networks you own or have explicit written authorization to test.

Proof of Concept Runbook

This is the shortest path to verify a working local PoC.

1. Prepare environment

cp .env.example .env
uv sync --extra dev

2. Start backing services

cd docker
docker compose up -d sensor-db aggregator-db redis
cd ..

3. Run PoC verification

./scripts/poc_check.sh

What this validates:

• Sensor API route wiring and key API behaviors.
• Aggregator auth/trust/org/vendor/CVE route wiring and key behaviors.
• Sensor UI, Aggregator UI, and Vendor Portal UI rendering/flow tests.

4. Manual API smoke (optional)

Start APIs in separate terminals:

uvicorn govpeer.aggregator.main:app --host 0.0.0.0 --port 8500 --reload
uvicorn govpeer.sensor.main:app --host 0.0.0.0 --port 8400 --reload

Check health:

curl -s http://localhost:8500/api/v1/vendor/auth/vendors | jq '.[0]'
curl -s http://localhost:8400/api/v1/health

Sensor identity bootstrap:

curl -s -X POST http://localhost:8400/api/v1/identity/init \
  -H "Content-Type: application/json" \
  -d '{"aggregator_url":"http://localhost:8500"}' | jq

Local Development Setup

1. Start databases with Docker

cd docker
docker compose up -d sensor-db aggregator-db redis

This gives you:

Service	Port	Connection
Sensor DB (PostgreSQL 16)	5432	postgresql+asyncpg://govpeer:devpassword@localhost:5432/govpeer_sensor
Aggregator DB (TimescaleDB)	5433	postgresql+asyncpg://govpeer:devpassword@localhost:5433/govpeer_aggregator
Redis	6379	redis://localhost:6379/0

2. Install Python dependencies

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

3. Configure environment

cp .env.example .env

The defaults work for local dev against the Docker databases — no edits needed to get started.

If you run against an existing long-lived aggregator database, apply federation schema updates:

psql postgresql://govpeer:devpassword@localhost:5433/govpeer_aggregator \
  -f scripts/migrations/20260215_add_federation_tables.sql

4. Run the Sensor API

uvicorn govpeer.sensor.main:app --host 0.0.0.0 --port 8400 --reload

API docs at http://localhost:8400/api/v1/docs (note: sensor uses /api/v1 root path)

5. Run the Aggregator API

uvicorn govpeer.aggregator.main:app --host 0.0.0.0 --port 8500 --reload

API docs at http://localhost:8500/docs

6. Run Celery workers (optional — needed for scanning and scheduled tasks)

# Sensor worker
celery -A govpeer.sensor.tasks worker --loglevel=info

# Sensor beat scheduler (separate terminal)
celery -A govpeer.sensor.tasks beat --loglevel=info

# Aggregator worker
celery -A govpeer.aggregator.tasks worker --loglevel=info

# Aggregator beat scheduler (separate terminal)
celery -A govpeer.aggregator.tasks beat --loglevel=info

7. Run the UIs

Each UI is a React + Vite + Tailwind app. From the project root:

# Sensor UI (proxies API to :8400)
cd govpeer/ui/sensor-ui
npm install
npm run dev          # → http://localhost:8401

# Aggregator UI (proxies API to :8500)
cd govpeer/ui/aggregator-ui
npm install
npm run dev          # → http://localhost:8501

# Vendor Portal UI (proxies API to :8500)
cd govpeer/ui/vendor-portal-ui
npm install
npm run dev          # → http://localhost:8502

Running Tests

Backend (pytest)

# All tests
pytest

# With coverage
pytest --cov=govpeer --cov-report=term-missing

# Specific module
pytest tests/sensor/
pytest tests/aggregator/
pytest tests/test_crypto.py

# Skip slow/integration tests
pytest -m "not slow and not integration"

Tests use aiosqlite (in-memory SQLite) so they don't require a running database.

Frontend (vitest)

# Sensor UI tests
cd govpeer/ui/sensor-ui && npm test

# Aggregator UI tests
cd govpeer/ui/aggregator-ui && npm test

# Vendor Portal tests
cd govpeer/ui/vendor-portal-ui && npm test

Linting & Type Checking

# Lint (ruff)
ruff check .
ruff format --check .

# Type check (mypy)
mypy govpeer/

Environment Variables

All config uses pydantic-settings with the GOVPEER_SENSOR_ and GOVPEER_AGGREGATOR_ prefixes.

Sensor

Variable	Default	Description
GOVPEER_SENSOR_DATABASE_URL	postgresql+asyncpg://govpeer:devpassword@localhost:5432/govpeer_sensor	Sensor database
GOVPEER_SENSOR_REDIS_URL	redis://localhost:6379/0	Redis for Celery
GOVPEER_SENSOR_AGGREGATOR_URL	—	Aggregator URL for federation
GOVPEER_SENSOR_ORG_ID	—	Default org UUID used when X-Org-Id header is omitted
GOVPEER_SENSOR_MASSCAN_RATE	1000	Packets per second for Masscan
GOVPEER_SENSOR_SMTP_HOST	—	SMTP server for email alerts

Aggregator

Variable	Default	Description
GOVPEER_AGGREGATOR_DATABASE_URL	postgresql+asyncpg://govpeer:devpassword@localhost:5433/govpeer_aggregator	Aggregator database
GOVPEER_AGGREGATOR_REDIS_URL	redis://localhost:6379/0	Redis for Celery
GOVPEER_AGGREGATOR_JWT_SECRET	CHANGE-ME-IN-PRODUCTION	Member JWT signing key
GOVPEER_AGGREGATOR_VENDOR_JWT_SECRET	CHANGE-ME-VENDOR-JWT-SECRET	Vendor portal JWT signing key
GOVPEER_AGGREGATOR_ALLOW_INSECURE_DEV_AUTH	false	Allow X-Org-Id auth fallback for local UI development (dev-only)
GOVPEER_AGGREGATOR_DEV_DNS_VERIFICATION_BYPASS	false	Treat DNS verification as successful in local dev
GOVPEER_AGGREGATOR_FEDERATION_BASE_URL	—	Public base URL for this aggregator when signing outbound federation sync requests
GOVPEER_AGGREGATOR_FEDERATION_PRIVATE_KEY	—	Ed25519 private key (base64) used to sign outbound federation sync payloads
GOVPEER_AGGREGATOR_FEDERATION_SYNC_TIMEOUT_SECONDS	10.0	HTTP timeout for outbound federation sync/health calls
GOVPEER_AGGREGATOR_FEDERATION_NONCE_RETENTION_DAYS	14	Retention window for replay-protection sync nonces
GOVPEER_AGGREGATOR_STRIPE_SECRET_KEY	—	Stripe secret key for vendor billing
GOVPEER_AGGREGATOR_STRIPE_WEBHOOK_SECRET	—	Stripe webhook verification

See .env.example for the full list.

Project Structure

govpeer/
├── common/              Shared: crypto, config, database
├── sensor/
│   ├── main.py          FastAPI app
│   ├── api/             REST endpoints (scans, alerts, config, dashboard, federation, identity)
│   ├── scanner/         Masscan, Nmap, fingerprinting, delta detection
│   ├── services/        Alert evaluation, notifications, aggregator push
│   └── tasks.py         Celery scan pipeline
├── aggregator/
│   ├── main.py          FastAPI app
│   ├── api/             REST endpoints (auth, trust, circles, community, CVEs, vendors, portal)
│   ├── models/          SQLAlchemy models
│   └── services/        Trust, ledger, alerts, CVE, vendor scoring, Stripe
└── ui/
    ├── sensor-ui/       React dashboard for local scanning
    ├── aggregator-ui/   React dashboard for community intelligence
    └── vendor-portal-ui/ React portal for vendor subscriptions

Database Migrations

GovPeer uses Alembic for schema migrations. There are two separate migration environments — one for the sensor and one for the aggregator.

Generating a migration

# Sensor
alembic -c alembic-sensor.ini revision --autogenerate -m "describe change"

# Aggregator
alembic -c alembic-aggregator.ini revision --autogenerate -m "describe change"

Review the generated file in govpeer/{sensor,aggregator}/alembic/versions/ before committing.

Applying migrations

Migrations are run out-of-band by the operator before starting or restarting the service — they are not applied automatically on startup.

# Sensor
alembic -c alembic-sensor.ini upgrade head

# Aggregator
alembic -c alembic-aggregator.ini upgrade head

For rollbacks:

alembic -c alembic-sensor.ini downgrade -1
alembic -c alembic-aggregator.ini downgrade -1

Both env.py files read DATABASE_URL from the same environment variables used by the services (GOVPEER_SENSOR_DATABASE_URL / GOVPEER_AGGREGATOR_DATABASE_URL), so no extra configuration is needed.

TimescaleDB Hypertables

The aggregator database runs on TimescaleDB. Several tables are designed as time-series hypertables — partitioned automatically by a TIMESTAMPTZ column for efficient range queries and retention policies.

Intended hypertables

Service	Table	Time column
Aggregator	aggregator_scan_results	scanned_at
Sensor	scan_results	scanned_at
Sensor	scan_deltas	detected_at
Sensor	nuclei_vuln_findings	detected_at

Note: The sensor uses plain PostgreSQL 16. The create_hypertable call for sensor tables should only be applied when the sensor database is also running TimescaleDB.

Why autogenerate doesn't handle this

Alembic's --autogenerate compares SQLAlchemy models to the live schema. create_hypertable is a TimescaleDB catalog operation, not a DDL statement, so autogenerate never emits it. Every operator must add the call manually after running alembic revision --autogenerate.

The op_create_hypertable helper

govpeer.common.alembic_helpers provides two helpers that wrap op.execute:

from govpeer.common.alembic_helpers import op_create_hypertable, op_drop_hypertable

Helper	Purpose
op_create_hypertable(table, time_column, *, chunk_time_interval, if_not_exists)	Convert an existing table to a hypertable.
op_drop_hypertable(table)	Document the downgrade intent (table must still be dropped by op.drop_table).

Both helpers call op.execute so they participate in the same Alembic transaction as the surrounding DDL.

op_create_hypertable is idempotent by default (if_not_exists=True), meaning it is safe to re-run a migration on a table that is already a hypertable.

Adding a hypertable call to an autogenerated migration

After running alembic revision --autogenerate -m "add aggregator_scan_results", open the generated file and add the helper call at the end of upgrade() and a matching call at the start of downgrade():

# In upgrade():
op_create_hypertable("aggregator_scan_results", "scanned_at")

# In downgrade():
op_drop_hypertable("aggregator_scan_results")
op.drop_table("aggregator_scan_results")

Worked example — aggregator_scan_results

The following is a complete minimal migration file for converting aggregator_scan_results into a hypertable. It is idempotent: running it twice on the same database is safe because if_not_exists=True is the default.

"""Convert aggregator_scan_results to TimescaleDB hypertable.

Revision ID: 0001_hypertable_aggregator_scan_results
Revises: <previous_revision>
Create Date: 2026-05-16 00:00:00.000000

"""
from __future__ import annotations

from typing import Sequence, Union

import sqlalchemy as sa
from alembic import op

from govpeer.common.alembic_helpers import op_create_hypertable, op_drop_hypertable

# revision identifiers, used by Alembic.
revision: str = "0001_hypertable_aggregator_scan_results"
down_revision: Union[str, None] = "<previous_revision>"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
    # The table was created by a prior migration; we only need to register it
    # as a TimescaleDB hypertable partitioned by scanned_at.
    # if_not_exists=True (the default) makes this safe to re-run.
    op_create_hypertable(
        "aggregator_scan_results",
        "scanned_at",
        chunk_time_interval="7 days",
    )


def downgrade() -> None:
    # TimescaleDB has no "detach hypertable" function; the table must be
    # dropped and recreated as a plain table.  op_drop_hypertable documents
    # the intent and verifies the catalog entry exists before we proceed.
    op_drop_hypertable("aggregator_scan_results")
    op.drop_table("aggregator_scan_results")

Maintainers

See MAINTAINERS.md for the list of maintainers, their roles, and how to become one.

License

Apache 2.0