Agatha

Reliable binary data transmission via YouTube's video transcoding pipeline using BCH error correction.

Encodes arbitrary binary files into 4K grayscale video frames (1 bit per pixel), uploads them to YouTube, and recovers the original files with 100% fidelity using BCH error correction. YouTube's VP9 transcoding degrades the signal, but BCH(t=16, m=15) corrects all introduced errors.

Proven at ~140 MB with SHA256-verified recovery at both 30 fps and 60 fps.

Current release: v1.1.0 · Live: agatha.hegga.cl · Paper: PDF (EN) · PDF (ES)

---

How it works

1. Input files are packaged into a canonical TAR and split into 10 MB chunks
2. Each chunk is BCH-encoded (t=16, 2048 B blocks, 30 B overhead each)
3. ECC-protected bits are packed into a 3840x2160 grayscale bitmap (1 bit/pixel)
4. Bitmaps are encoded into FFV1 lossless video, then re-encoded to H.264 CRF 18 for upload
5. Chunk videos are concatenated into a bundle with 1-second white separator frames (audio stripped to avoid AAC edit list DTS corruption)
6. After YouTube transcodes to VP9 2160p, frames are extracted, thresholded at 128, and BCH-decoded
7. BCH corrects all VP9-introduced errors — full recovery is SHA256-verified

White P-frames between chunks use near-zero VP9 bits, leaving the full codec budget for data frames (30-60:1 white:data ratio amplification).

---

Two operating families

As described in the paper:

Family	Mode	Throughput	Stealth	Best BER
Morton Full	Fast 4K VP9 channel	High (~140 MB proven)	Low (visible binary frames)	SHA256 PASS (BER=0 after BCH)
Color Carrier	Modulates luminance of natural images	Lower (~13.6 MB proven)	High (visually natural video)	2.53x10^-7 (thresh m=60)

C16 stealth mode: 1 data frame per 30 frames of real video at 30 fps. BER=0 — visually indistinguishable from legitimate content.

---

Proven results (v1.1.0)

FPS	Chunks	Result	Corrected blocks (max)	Archive
30	14/14	SHA256 PASS	—	~140 MB real files
60	14/14	SHA256 PASS	516	~140 MB real files

VP9 format codes: 313 (2160p <=30 fps) · 315 (2160p 60 fps). Production recovery auto-selects the matching format from the run FPS.

17 validated codec recipes with YouTube proof points in domain/recipes.py.

---

Monorepo layout

packages/core       Domain, application services, ECC, codec, ffmpeg, yt-dlp (hexagonal)
apps/cli            Typer CLI adapter (agatha)
apps/api            FastAPI control-plane adapter (agatha-api)
apps/web            Astro 5 SSR + React 19 operator UI (agatha.hegga.cl)
tests/              75 automated tests (pytest)
migrations/         Alembic schema migrations (SQLite / PostgreSQL)
carriers/           14 reference carrier images for steganographic modes
experiments/        Archived research scripts (Jan-Mar 2026 YouTube campaigns)
papers/             Academic paper (EN + ES) with figures
docker/             Dockerfile + docker-compose for containerized deployment

---

Quick start

Prerequisites

• Python 3.11+
• ffmpeg + ffprobe
• yt-dlp (for YouTube operations)

Install

python3 -m venv .venv
source .venv/bin/activate
pip install -e packages/core -e apps/cli -e apps/api

Initialize database

agatha db init --db-url sqlite:///data/agatha.db

Run API

agatha-api --db-url sqlite:///data/agatha.db

Using Docker

docker compose -f docker/docker-compose.yml up -d
# API on :8000, Web on :4321, PostgreSQL internal

---

Environment variables

AGATHA_DB_URL=sqlite:///data/agatha.db      # or postgresql://...
AGATHA_FFMPEG_BIN=ffmpeg
AGATHA_FFPROBE_BIN=ffprobe
AGATHA_YT_DLP_BIN=yt-dlp
AGATHA_YOUTUBE_TOKEN_CACHE=$HOME/.config/agatha/youtube_oauth_token.json
AGATHA_YOUTUBE_CLIENT_SECRETS=               # required for auto-upload

---

Commands

Pipeline lane (testing / evidence)

Full artifact persistence for reproducible evidence:

agatha pipeline encode   --name smoke --input-file payload.tar --workspace /tmp/ws
agatha pipeline bundle   --run-id <id>
agatha pipeline simulate --run-id <id> --profile light
agatha pipeline recover  --run-id <id> --bundle-video /tmp/ws/bundle/bundle.mp4
agatha pipeline repair   --run-id <id> --recovery-report /tmp/ws/recovery/report.json

Production lane (minimum-persistence)

Keeps only canonical TAR metadata, compact manifest, and remote references:

agatha production receive   --name my-archive --workspace /srv/agatha/runs/my-archive \
                            --input-path /srv/uploads/my-archive --max-fps 30
agatha production transform --run-id <id> --no-cleanup
agatha production deliver   --run-id <id> --output-dir /srv/delivery

YouTube utilities

agatha youtube upload     --run-id <id> --client-secrets /path/to/youtube.json
agatha youtube wait-2160  --run-id <id>
agatha youtube download   --run-id <id> --format-id 313   # 313=30fps, 315=60fps

---

Web control plane

Astro 5 SSR + React 19 + Tailwind + shadcn/ui. Full EN/ES bilingual interface.

cd apps/web
npm install
AGATHA_API_URL=http://127.0.0.1:8000 npm run dev

Features:

• Upload files or TAR directly from the browser
• Select validated recipe from the central catalog
• Auto-upload to YouTube (OAuth) or manual handoff
• Monitor YouTube until 2160p is available
• On-demand recovery and repair
• Real-time progress via SSE

Production deployment via launchd (macOS) + Cloudflare Tunnel:

bash deploy.sh   # build + install services + restart

Services: cl.hegga.agatha.api (:8000) + cl.hegga.agatha.web (:4321) + cl.hegga.agatha.tunnel (agatha.hegga.cl)

---

Running tests

pytest                                    # all 75 tests
pytest --ignore=tests/test_real_30mb.py   # fast tests only (~2 min)
pytest tests/test_real_30mb.py            # 30MB pipeline tests (~15 min)
ruff check .                              # lint (0 errors)
mypy packages/core/src                    # type check

---

CI/CD

GitHub Actions pipeline on every push/PR:

Job	Trigger	Description
lint	push, PR	ruff check
test	push, PR	66 fast tests (pytest)
test-heavy	main push	9 tests with 30MB payload (~15 min)
web	push, PR	Astro type-check + build

---

Architecture

Hexagonal architecture inside packages/core:

Layer	Path	Responsibility
Domain	domain/models.py, domain/recipes.py	Enums, PipelineConfig, reports, 17 codec recipes
Application	application/service.py	PipelineApplicationService — all orchestration
Services	services/	ECC, codec, chunking, tarball, scrambling (no I/O)
Infrastructure	infrastructure/	SQLAlchemy DB, ffmpeg, yt-dlp/YouTube, settings
Ports	ports/	RunRepository interface

Adapters: CLI (Typer), API (FastAPI), Web (Astro SSR).

Database: 5 tables (pipeline_runs, pipeline_chunks, artifacts, youtube_videos, production_runs). Schema managed by Alembic. Supports SQLite and PostgreSQL.

---

Key configuration

Parameter	Default	Notes
chunk_size_bytes	10 MB	Input split size
bitmap_width	3840	Pixels per row
frame_height	2160	4K resolution
max_fps	60	Clamped upper bound
min_seconds	8.0	Minimum chunk video duration
ecc_t	16	BCH error correction capacity
ecc_block_size_bytes	2048	BCH block size

---

Paper

The academic paper describes the system design, experimental methodology, and all validated operating points:

• English: agatha.pdf
• Spanish: agatha_es.pdf

Key findings from the paper:

• Morton Full succeeds at 10, 30, and 300 MB in a robust max-6 fps profile
• Color Threshold and Color Polarity achieve SHA256-verified recovery at 10, 30, and 100 MB
• Best absolute BER: 2.53x10^-7 (color threshold, m=60)
• Deterministic failure at 4 fps (YouTube converts to 6 fps VP9, BER=0.10)
• C16 stealth: 1 data frame per 30 real frames, BER=0, visually indistinguishable

---

License

CC BY-NC-SA 4.0 — Attribution-NonCommercial-ShareAlike.

Free to use, study, and build on for non-commercial purposes. Any derivative work must credit the original author and be released under the same terms. Commercial use is not permitted.

The non-commercial restriction reflects the nature of the project: a covert data channel through a third-party platform requires responsible disclosure and use boundaries.

(c) 2025-2026 Efrain Garay