Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

🏗️ Foundry

An enterprise-grade, multimodal knowledge platform.

Upload massive volumes of documents → process them asynchronously through durable workflows → store semantic + tabular knowledge → chat with your enterprise knowledge via agentic Retrieval-Augmented Generation.


What is Foundry?

Organizations sit on large, mixed piles of documents — reports, decks, spreadsheets, contracts, recorded calls — and need trustworthy answers and insights without reading everything or waiting on an analyst. Foundry turns that document sprawl into a conversational, cited, analytically capable knowledge base.

Users create workspaces → projects → documents. Every upload is ingested asynchronously through Temporal workflows (OCR, transcription, parsing, chunking, embedding, indexing), then made queryable through a streaming, agentic chat that both retrieves grounded passages and runs real analytics over spreadsheet data — always with inline citations back to the exact source (page / slide / timestamp).

It is built to resemble a real production SaaS — multi-tenancy, RBAC/ABAC, durable orchestration, observability, and evaluation — not a demo.

✨ Key capabilities

  • Multi-tenant accounts — Organization → Workspace → Project → Document, isolated by Postgres Row-Level Security.
  • Invite-based auth — self-built JWT + rotating refresh tokens, email verification, password reset, ABAC authorization.
  • Multimodal ingestion — PDF, DOCX, PPTX, XLSX, Markdown, images, audio, video (≤ 500 MB, direct-to-S3 multipart).
  • Durable pipelines — Temporal workflow-per-file-type with shared activities, dedup + versioning, DLQ, partial resume, cancel/pause signals, and live progress.
  • Dual knowledge store — pgvector (semantic) + DuckDB/Parquet (tabular analytics), all backed by a single PostgreSQL.
  • Agentic RAG chat — streaming, tool-using (hybrid retrieval + text-to-SQL router), grounded inline citations, summarized memory.
  • Evaluation & observability — LangSmith tracing + RAGAS-style CI evals; OpenTelemetry → Prometheus/Tempo/Loki/Grafana; audit logs.
  • Fully Dockerizeddocker compose up brings up the entire platform locally with seed data.

🧱 Tech stack

Versions are pinned to the latest stable / LTS lines as of July 2026 (no beta/RC). Full matrix in [docs/02-tech-stack-and-versions.md](./docs/02-tech-stack-and-versions.md).

Layer Technologies
Backend Python 3.14 · FastAPI 0.139 · Temporal · SQLAlchemy 2.0 · Alembic · Pydantic v2 · Docker
Data PostgreSQL 18 · pgvector 0.8.2 (extension, same DB) · Redis 8 · DuckDB (1.4 LTS) + Parquet
AI OpenAI Responses API + text-embedding-3-small (1536) · self-hosted GLM OCR · self-hosted Whisper · cross-encoder reranker · LangChain (where it adds value) · LangSmith
Frontend Next.js 16 (App Router, RSC) · React 19.2 · TypeScript 6 · Tailwind CSS 4 · TanStack Query 5 · Zustand 5 · nuqs · Node 24 LTS
Storage AWS S3 (MinIO locally)
Observability OpenTelemetry · Prometheus · Tempo · Loki · Grafana · GlitchTip
Deploy Docker Compose (local) · GitHub Actions (CI/CD) · Vercel (frontend) · Render/AWS (backend)

State management note: server state via TanStack Query, client/UI state via Zustand, URL state via nuqs, initial reads via React Server Components. Redux is intentionally not used.

🏛️ Architecture at a glance

flowchart LR
  FE[Next.js 16] --> API[FastAPI /api/v1<br/>SSE + WS]
  API --> PG[(PostgreSQL 18<br/>+ pgvector)]
  API --> RDS[(Redis 8)]
  API --> S3[(S3 / MinIO)]
  API --> TMP[Temporal]
  TMP --> WRK[Workers: OCR · Whisper · chunk · embed]
  WRK --> PG & S3
  API --> OAI[OpenAI] & DDB[DuckDB] & RRK[Reranker]
  DDB --> S3
Loading

Two knowledge paths, unified by the chat agent at query time:

  1. Unstructured → chunks + embeddings in pgvector (hybrid vector + BM25 → rerank → RLS/metadata filters).
  2. Structured (XLSX) → Parquet, queried by DuckDB agentic text-to-SQL (validated, read-only).

📚 Documentation

The complete engineering specification lives in [/docs](./docs) — 22 linked documents written to be consumed by engineers and an autonomous coding agent. Start with the docs index & decision ledger.

# Document
00 Overview
01 Product Requirements (PRD)
02 Tech Stack & Pinned Versions
03 Functional Requirements
04 Non-Functional Requirements
05 User Stories & Acceptance Criteria
06 UX Flows
07 Database Schema & ER Diagram
08 Backend Architecture
09 Frontend Architecture
10 API Specification & Contracts
11 Temporal Workflow Design
12 RAG & Analytics Pipeline
13 Security Architecture
14 Deployment & Infrastructure
15 Observability
16 Testing Strategy
17 Diagrams
18 Risks & Trade-offs
19 Future Enhancements
20 Roadmap & Milestones

🚀 Getting started

Implementation is spec-first; the repo scaffold is generated from /docs. Once code exists, the canonical local workflow is:

# 1. Configure environment
cp infra/.env.sample .env      # fill in OPENAI_API_KEY, secrets, etc.

# 2. Bring up the entire platform (API, workers, Postgres+pgvector, Redis,
#    Temporal, MinIO, model workers, observability)
docker compose up --build

# 3. Run migrations + seed demo data
docker compose run --rm migrate
docker compose run --rm seed

# App:        http://localhost:3000
# API docs:   http://localhost:8000/api/v1/docs
# Temporal UI: http://localhost:8080
# Grafana:    http://localhost:3001

See [docs/14-deployment-and-infrastructure.md](./docs/14-deployment-and-infrastructure.md) for the full service list and environment configuration.

🗺️ Roadmap

Delivered in ordered, independently-demoable milestones (M0 → M7): foundations → identity/tenancy → documents/upload → ingestion pipeline → retrieval/knowledge stores → agentic chat → admin/observability/eval → hardening & delivery. Details in [docs/20-roadmap-and-milestones.md](./docs/20-roadmap-and-milestones.md).

📌 Scope notes

  • In scope: everything above — the full vision, built in milestones.
  • Deferred (schema-ready): SSO/SAML, billing, real-time co-editing, native mobile, video visual understanding. See Future Enhancements.

Foundry — a flagship demonstration of full-stack AI systems architecture.

About

Enterprise knowledge foundry built with Python, FastAPI, Temporal and modern AI infrastructure. Transform raw documents into searchable intelligence.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors