Skip to content

Dalvae/taptologs

Repository files navigation

TaptoLogs

A high-performance combat log parser and viewer for World of Warcraft WotLK 3.3.5. Successor to uwu-logs, inspired by World of Logs, LegacyPlayers, and Warcraft Logs.

Parse your WoW combat logs in milliseconds, explore metrics instantly, and share reports—all without raw logs ever leaving your browser.

Key Highlights

Edge-first Architecture Parsing happens client-side in your browser via Rust WASM, eliminating server compute costs and ensuring your logs never touch remote servers. Only structured data is uploaded.

Rust WASM Parser CPU-intensive combat log parsing compiled to WebAssembly for near-native performance. Type-safe Rust enums prevent parsing edge cases.

Parquet + DuckDB-WASM Combat data stored as Apache Parquet (10-20x compression of raw logs) and queried in-browser using DuckDB-WASM. SQL queries run in your browser with no API calls.

Zero Server Compute for Parsing Raw logs are never sent to the server. Browsers do all the parsing work, making the system infinitely scalable.

Privacy First Your WoW combat logs remain on your machine until you explicitly upload the parsed, structured Parquet file to Cloudflare R2.

Stack

Layer Technology Location
Parser Rust → WASM Browser (Web Worker)
Data Format Apache Parquet Generated client-side
Query Engine DuckDB-WASM Browser
Object Storage Cloudflare R2 Edge
Database Cloudflare D1 (SQLite) Edge
API SvelteKit API Routes Cloudflare Pages
Frontend SvelteKit Cloudflare Pages
Auth Better Auth D1-backed
Cache Cloudflare KV Edge (leaderboards, hot data)

Architecture

Browser                          Cloudflare Edge
+---------------------------+    +---------------------------+
|                           |    |                           |
|  1. User drops log file   |    |  SvelteKit (Pages)       |
|         |                 |    |    Frontend + API Routes |
|         v                 |    |    |                      |
|  2. Web Worker            |    |    +-> D1 (metadata)      |
|     Rust WASM Parser      |    |    +-> KV (cache)         |
|         |                 |    |    +-> R2 (presigned URLs)|
|         v                 |    |                           |
|  3. Parquet file in memory|    +---------------------------+
|         |                 |
|         v                 |    +---------------------------+
|  4. Upload to R2 ---------|---->  R2 Bucket               |
|     (presigned URL)       |    |    /parquets/{id}.parquet|
|                           |    |                           |
|  5. POST metadata --------|---->  D1                      |
|     to API route          |    |    logs table            |
|                           |    |    rankings table        |
|                           |    +---------------------------+
|  6. View report           |
|     DuckDB-WASM           |    +---------------------------+
|         |                 |    |                           |
|         +--- range req ---|---->  R2 (read parquet)       |
|         |                 |    |                           |
|         v                 |    +---------------------------+
|  7. SQL queries in browser|
|     render charts/tables  |
+---------------------------+

Monorepo Structure

Three packages managed with Bun workspaces:

packages/parser (Rust) Compiles to WebAssembly via wasm-pack. Parses raw WoW combat log text into structured encounters with damage, healing, deaths, and auras. Outputs Apache Parquet for efficient storage and querying.

packages/web (SvelteKit) Frontend and API routes deployed to Cloudflare Pages. Svelte 5 with runes syntax ($state, $props, $derived). Handles file upload, viewer UI, and API endpoints for metadata storage and rankings.

packages/shared (@taptologs/shared) Shared TypeScript types and constants: boss definitions, class/spec metadata, spell IDs, and class colors. Imported by the web package.

Getting Started

Prerequisites

  • Bun (package manager and runtime)
  • Rust toolchain (for parser development)
  • wasm-pack (for building Rust to WASM)

Development

# Start SvelteKit dev server (main workflow)
bun run dev:web

# Build Rust parser to WASM
bun run build:parser

Build & Check

# Build all packages
bun run build

# Check SvelteKit app (Svelte + TypeScript)
cd packages/web && bun run check

Rust Development

# Run parser unit tests
cd packages/parser && cargo test

# Run tests with stdout output
cd packages/parser && cargo test -- --nocapture

# Manually build WASM
cd packages/parser && wasm-pack build --target web

Shared Types

# Check shared types (TypeScript strict mode)
cd packages/shared && bun run lint

How It Works

Upload Flow

  1. User drags WoWCombatLog.txt into the browser
  2. Web Worker loads the file and transfers it to Rust WASM parser (zero-copy)
  3. Parser detects boss encounters by GUID, segments fights/pulls, and extracts damage, healing, deaths, auras
  4. Parser outputs structured data as Apache Parquet (compressed ~10-20x vs raw log)
  5. Browser requests presigned URL from SvelteKit API route
  6. Parquet uploaded directly to Cloudflare R2
  7. Browser POSTs metadata to API route:
    • Log ID, raid date, server, bosses found
    • Player list with GUIDs, names, classes, specs
    • Per-encounter summary (duration, raid composition)
  8. SvelteKit API stores metadata in D1 SQLite database and updates rankings

View Flow

  1. User navigates to report URL
  2. SvelteKit server-side renders the page, loading metadata from D1 (fast, lightweight)
  3. Browser initializes DuckDB-WASM
  4. DuckDB queries Parquet file on R2 using HTTP range requests
    • Columnar format means only fetching needed columns
    • Predicate pushdown filters data at the storage level
  5. Results rendered with interactive charts and tables
  6. All navigation and filtering use local DuckDB queries—no API calls, instant updates

Core Features

Boss Encounter Detection Recognizes all 64 WotLK bosses (Naxxramas through Ruby Sanctum) with difficulty detection (10N, 10H, 25N, 25H).

Damage & Healing Metrics Automatic DPS/HPS aggregation with pet consolidation. Break down damage by spell, target, and time window.

Player Tracking Identify players by GUID, track class/spec automatically, and build historical profiles across logs.

Ranking System Per-boss, per-spec leaderboards cached in Cloudflare KV for instant access.

Data Integrity Three-layer security: upload tokens (time-limited HMAC), WASM-signed rankings (HMAC-SHA256 baked into the binary prevents leaderboard manipulation), and timestamp integrity checks (detects log tampering).

Privacy Raw logs are parsed locally and never transmitted to the server. Only the compressed Parquet file and metadata leave your browser.

Technical Decisions

Why Parquet + DuckDB-WASM?

  • 10-20x compression: 100MB raw log shrinks to 5-10MB Parquet
  • Columnar queries: DPS view reads only 3 columns, not all 15+
  • HTTP range requests: Fetch only needed data from R2, zero API overhead
  • SQL in browser: Familiar query language, no custom parsing engine
  • Instant navigation: All queries run locally after initial load

Why Rust WASM?

  • CPU-bound: Combat log parsing is intensive string processing
  • Near-native speed: WASM runs close to native performance
  • Zero-copy parsing: Rust's memory model prevents allocations
  • Type safety: Rust enums prevent entire classes of parsing bugs
  • Parquet generation: arrow-rs is the reference implementation

Why SvelteKit API Routes?

  • Single deployment: Everything on Cloudflare Pages with atomic deploys
  • No CORS: Same origin for frontend and API
  • Shared auth: One token verification for all routes
  • Type safety: Shared types between frontend and backend
  • Lower latency: API and UI on the same edge location

Performance

Expected bundle sizes (compressed):

Asset Size
First load (SvelteKit shell) ~200 KB
Upload page (+Rust WASM parser) ~1.7 MB
Report page (+DuckDB-WASM) ~3.5 MB

Large asset downloads lazy-load only when needed. After first load, DuckDB and parser cache in browser for instant subsequent visits.

Security & Environment

Data Integrity Layers

  1. Upload Tokens — Time-limited HMAC-SHA256 tokens (UPLOAD_SECRET) bind uploads to a specific logId + timestamp. Expire after 1 hour.
  2. Rankings Signature — The Rust WASM parser signs computed DPS data with a secret embedded at build time (WASM_SIGNING_SECRET). The server verifies the signature before accepting rankings. An attacker cannot forge valid signatures without the secret baked into the WASM binary.
  3. Timestamp Integrity — The parser computes monotonicity, event density, and gap metrics. Suspicious logs (backwards timestamps, future dates, low density) are flagged in logs.integrity_flags.

Environment Variables

Copy packages/web/.env.example and fill in values:

# Cloudflare R2 (presigned URLs)
R2_ACCESS_KEY_ID=
R2_SECRET_ACCESS_KEY=
R2_BUCKET_NAME=
R2_PUBLIC_URL=

# Security secrets (generate each with: openssl rand -hex 32)
UPLOAD_SECRET=          # HMAC for upload tokens
WASM_SIGNING_SECRET=    # HMAC for rankings signatures

Important: WASM_SIGNING_SECRET must be set both as a Cloudflare Pages env var (runtime) and passed at WASM build time:

WASM_SIGNING_SECRET=<your-secret> wasm-pack build --target web

In local dev, the parser falls back to a hardcoded dev secret automatically.

License

See LICENSE file for details.

Contributing

Contributions welcome. Please review the CLAUDE.md and ARCHITECTURE.md documents for project conventions and architecture details.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors