v0.4.0: Step-matched Comparison & Usability

Highlights

Step-matched comparison - Essential for curriculum learning where comparing final metrics is misleading:

runwise compare run1@50000 run2@50000      # Compare at same training step
runwise compare run1@10000 run2@20000      # Compare at different steps

Custom keys for summaries - Show only the metrics you care about:

runwise latest -k loss,val_loss,accuracy
runwise run abc123 -k train/loss,val/loss

Comparison filtering - Handle 100+ metric comparisons:

runwise compare A B -t 5        # Only show >5% changes
runwise compare A B -g          # Group by prefix (train/, val/)
runwise compare A B -f val      # Filter to validation metrics

New Features

CLI

• Step-matched comparison with @step syntax
• Custom keys for summaries (-k/--keys)
• Comparison filtering (-g, -t, -f)
• Dynamic column widths (no more truncated metric names)
• Step mismatch warnings

W&B API

• --history -k: Get metric trajectories from cloud
• --best metric --max: Find best run by any metric
• Project auto-detection from wandb/ directory
• Comparison filtering for API comparisons

MCP Server

• New health_check tool: Quick "how's training going?" check
• Fixed JSON-RPC response format
• Improved tool descriptions (list_keys first pattern)
• Step-matched comparison support

Installation

pip install runwise --upgrade

Full Changelog

See CHANGELOG.md

v0.3.1 - Stability Analysis & Better Error Messages

What's New

Stability Analysis

• New runwise stability command for analyzing training stability using rolling standard deviation
• Configurable window size (-w/--window, default: 100 steps)
• CSV output option (--csv) for further analysis
• Support for local JSONL files (runwise local <file> --stability)

Improved Error Messages

• Missing history data: Clear explanation when wandb-history.jsonl is missing, with guidance on wandb.finish() and wandb sync
• W&B API auth: Detects authentication errors and guides users to run wandb login
• Project not found: Helpful troubleshooting for 404 errors

Documentation

• Added W&B Best Practices section to README (context managers, proper cleanup)
• Updated examples with stability command output
• Added error message examples to expected_outputs.md

Installation

pip install --upgrade runwise

Usage

# Analyze training stability
runwise stability -k loss,val_loss

# Custom window size
runwise stability -k loss -w 50

# CSV output for analysis
runwise stability -k loss --csv

# Local JSONL files
runwise local metrics.jsonl --stability -k loss

v0.3.0: Markdown Export, W&B API Support, Examples

What's New

Markdown Export (--format md)

Export summaries as GitHub-flavored markdown for issues, PRs, Notion, or documentation:

runwise latest --format md
runwise list --format md
runwise compare run_a run_b --format md

W&B API Support

Access runs from W&B cloud without local sync (requires pip install wandb):

runwise api --project my-project               # List runs
runwise api -p my-project -r abc123            # Summarize specific run
runwise api -p my-project --state running      # Filter by state

Examples

New /examples directory with:

• training_log.jsonl - Sample JSONL training log
• runwise_config_example.json - Example configuration
• expected_outputs.md - Reference outputs for all commands
• README.md - Examples documentation

README Badges

Added PyPI version, CI status, License, and Python version badges.

Installation

pip install runwise==0.3.0

Full Changelog

v0.2.3...v0.3.0

v0.2.3: Sparklines, Anomaly Detection, TensorBoard

What's New

Sparklines

Token-efficient trend visualization using Unicode block characters:

loss: ▇▆▅▄▃▂▁▁ ↓ (1.5→0.2)

Conveys trend in ~10 tokens vs 50+ for raw numbers.

Anomaly Detection

Automatically flags training issues:

• Loss spikes - Robust z-score using Median Absolute Deviation (MAD)
• Overfitting - Val/train loss ratio divergence
• Plateaus - No improvement over extended period
• Gradient issues - Vanishing or exploding gradients
• NaN/Inf - Critical failure detection

Zero tokens for healthy runs - only outputs when problems exist.

TensorBoard Support

Optional parsing of tfevents files (requires pip install tensorboard):

runwise tb              # List TB runs
runwise tb -r train_1   # Summarize specific run

New CLI Flags

• --no-spark - Disable sparklines (faster)
• --no-anomalies - Skip anomaly detection

New MCP Tools

• detect_anomalies - Run anomaly detection on a run
• get_sparkline - Get sparkline visualization for metrics

Install

pip install runwise==0.2.3

Full Changelog: v0.2.2...v0.2.3

v0.2.2

What's New

Features

• Run context support: Parse name, notes, tags, group from W&B metadata
• New runwise notes command to view run descriptions
• runwise stats and runwise history auto-detect keys when -k omitted
• runwise live now shows run ID, name, and state
• runwise compare -f <prefix> filters metrics by prefix
• runwise compare -d shows config differences between runs

Infrastructure

• Added 54 unit tests
• Added py.typed marker for PEP 561 type hint support
• Added GitHub Actions CI/CD workflows

Bug Fixes

• Fixed ruff linting issues