StaticDB

StaticDB plans public datasets into deterministic, immutable GitHub shards served through jsDelivr, with manifests and client-side query planning.

This repository contains the v1 TypeScript CLI, compiler core, SDK, and static demo shell.

Quick Start

npm install
npm test
npm run build
node dist/src/cli.js plan examples/openalex.profile.json --out plan.json
node dist/src/cli.js ingest-openalex \
  --input fixtures/openalex-snapshot/data/works/updated_date=2026-05-28/part_0000.gz \
  --out examples/openalex.real.profile.json \
  --owner StevenBuglione
node dist/src/cli.js build examples/openalex.real.profile.json --out demo
node dist/src/cli.js validate demo/manifest.json
node dist/src/cli.js inspect demo/manifest.json --json
node dist/src/cli.js compile-openalex \
  --snapshot-manifest fixtures/openalex-snapshot/data/works/manifest \
  --owner StevenBuglione \
  --dry-run-publish \
  --out publish-plan.json

Serve demo/ with any static server to try the static search shell.

Current Scope

Implemented:

• deterministic compile planning
• shard planning below configured byte limits
• repo bin-packing below configured repo byte limits
• immutable jsDelivr URL generation
• manifest build/validate/inspect CLI commands
• physical Parquet shard writing and read-back tests
• materialized token and facet indexes from real OpenAlex records
• SDK search execution, facet index selection, and primary-key shard routing
• dry-run publish planner with repo batches, CDN URLs, and side-effect-free steps
• static GitHub Pages-ready search demo shell

Next implementation layer:

• full OpenAlex snapshot streaming from S3
• richer BM25/index compression and browser Parquet row extraction
• resumable GitHub publisher with rate-limit backoff
• FreeDomain/GitHub Pages deployment automation