Upspeak

Upspeak is a personal knowledge management system and information client designed to collect, organise, and synthesise data from web sources and your own inputs.

Introduction

Upspeak will let you build personal knowledge repositories into queryable information archives. These repositories can include local data (notes you write) as well as gather data from any system accessible over HTTP, allowing you to annotate the data and send replies back within their contexts.

Upspeak will integrate with tools you already use to consume or create information in multiple shared contexts, such as Matrix, Discourse, and the Fediverse.

High Level Concepts

1. Repository: The core component where all data will be collected, organised, and managed. It will interface with both local and remote Archives.
2. Archives: Persistent, replicable, synced stores for Repositories. Archives can be local (SQLite metadata and node data as files on the user's device) or remote (Postgres metadata and node data in object storage). This separation will allow flexible data management across devices and deployment modes.
3. Nodes and Edges: Fundamental elements within the Repository. Nodes will represent data points, while Edges will define relationships between these points, forming a structured knowledge graph.
4. Threads and Annotations: Threads will group nodes into ordered collections. Annotations will let users add comments, highlights, and notes targeting specific nodes, enriching the knowledge graph with contextual information.
5. Filters and Rules: Reusable condition sets will control what data flows between sources, sinks, and repositories. Rules will automate actions based on events, enabling an autonomous data processing pipeline.
6. Connectors: Upspeak will integrate with tools like Matrix, Discourse, RSS, the Fediverse, email, and webhooks, enabling seamless data flow and interaction in shared contexts.

Architecture

Upspeak is an API-first knowledge infrastructure. It provides a structured knowledge graph for collecting, organising, and querying information from diverse sources. Clients (web, mobile, CLI, AI agents) connect over the HTTP API.

Upspeak uses a hybrid synchronous core + NATS JetStream architecture:

• Writes go synchronously to the archive (confirmed to the client)
• An embedded NATS server with JetStream carries the downstream consequences — inter-module events, background processing, scheduled jobs, and multi-device sync

Upspeak is local-first: offline writes succeed immediately and sync when reconnected. The system runs autonomously in the background (fetching sources, applying rules, processing data) and presents results when the user connects.

The decisions behind this architecture — the API-first stance, hexagonal infrastructure isolation, the hybrid write core, the global NATS JetStream event stream, and the embedded broker — are recorded as Architectural Decision Records in docs/adr/.

Vision: Social Knowledge Graph

Upspeak is designed as personal-first, federated knowledge infrastructure. While the current implementation focuses on single-user, multi-device scenarios, the architecture is being built toward:

• Federation: Upspeak instances will be able to sync knowledge with each other, enabling peer-to-peer knowledge sharing between users
• Social knowledge sharing: Users will be able to publish curated threads and collections with visibility controls, and other users will be able to follow and pull shared content into their own repositories
• Repo chaining: Repositories can subscribe to other repositories as sources, with filters controlling what flows between them — enabling collaborative knowledge pipelines across users and instances

The HTTP API is defined by the module handlers under repo/, filter/, jobs/, connector/, scheduler/, rules/, search/, and realtime/. See docs/next-steps.md for current status and the remaining work to complete the API.

Implementation Progress

• Phase 1: Foundation — Core domain models, UUID v7, NATS isolation, archive interface, repository CRUD, API envelope
• Phase 2: Knowledge Graph — Nodes, edges, threads, annotations, flat URL routing, optimistic concurrency
• Phase 3: Filters + Jobs — Filter CRUD, condition evaluation engine, job tracking, NATS job runner
• Phase 4: Connectors + Schedules — Sources, sinks, collect/publish actions, rate limiting, cron scheduling, job execution with history
• Phase 5: Rules + Search — Rule engine, FTS5 search, graph traversal
• Phase 6a: Real-time — WebSocket events, Hub fan-out, server-side filtering
• Phase 6b: Sync — Multi-device sync, conflict resolution (see docs/next-steps.md)

Develop

# Build the binary
./build.sh build

# Development mode (requires upspeak.yaml)
cp upspeak.sample.yaml upspeak.yaml
./build.sh dev

# Run tests (search needs the sqlite_fts5 build tag; build.sh sets it automatically)
go test -tags sqlite_fts5 ./...

# Clean build artefacts
./build.sh cleanup

AI-assisted development

This project includes a project-local Claude skill (.claude/skills/upspeak-dev/) that provides architecture context for AI-assisted development. Install the NATS skill globally for NATS JetStream reference when working on the event bus.

License

Upspeak is licensed under the Apache License, Version 2.0 (Apache-2.0). See the LICENSE file for the full license text.