TrainLens 0.20.0

TrainLens 0.20.0

TrainLens 0.20.0 improves the reliability, privacy, and scalability of local-first
machine-learning experiment analysis and agent-assisted research workflows.

Highlights

• Dataset profiling: correct handling of empty columns, finite numeric
  statistics, optional pandas nulls, and bounded first-N-row profiles through
  explain_dataset(max_rows=...). Truncated profiles are explicitly labeled
  as non-representative prefixes.
• Safer local projects: kernel-managed file locks avoid deleting another
  process's live lock and release ownership automatically after process exit.
• Predictable metric semantics: reject cross-metric alias theft, validate
  step metadata, and expose explicit step-aware recent slopes.
• Bounded monitoring: cap retained callback alerts while retaining the
  total alert-event count.
• Safer LLM workflows: supported-object allowlisting in outbound notebook
  values, typed provider errors, and optional retries for transient failures.
• Machine-readable CI: trainlens check --format json --output result.json
  produces structured gate and regression results with unchanged exit codes.
• Compatibility checks: optional native Keras/Transformers/Lightning
  smoke tests and manually dispatched Windows/macOS/Linux project tests.

Compatibility

TrainLens remains notebook-first, local by default, and supports Python 3.11–3.14.
Training frameworks are still optional. By default, the provider does not retry
requests and callbacks retain complete alert histories unless limits are supplied.
The existing portable run schema remains unchanged.

A few inputs are deliberately validated more strictly: invalid explicit metric
steps, cross-metric alias collisions, and unsupported arbitrary notebook objects
in outbound sanitized context. The project lock file is now retained as a stable
inode even after all writers have exited.

Upgrade

python -m pip install --upgrade trainlens

TrainLens 0.19.2

TrainLens 0.19.2

This stability release makes portable-run CI checks safer, aligns metric selectors across
the CLI and Python APIs, and improves checkpoint isolation and long-run storage.

Fixes

• CI metric gates now use split-qualified, finite metrics. Missing baseline metrics
  fail checks unless explicitly acknowledged with --allow-removed.
• Checkpoint selection rejects all known held-out test split aliases and preserves
  original observation indices when skipping non-finite observations.
• Project queries avoid redundant index parsing. Project.repair() diagnoses
  dangling entries and orphan files after interrupted operations; cleanup is opt-in.
• Run-file removal writes the index before deletion, making interruptions recoverable.
• MonitorConfig(max_observations=N) optionally bounds history and deduplication
  memory for long runs. The default behavior remains unchanged.
• Agent verification exposes has_valid_evidence_references and explicitly
  distinguishes citation checking from proving scientific or causal validity.

Compatibility

No supported APIs have been removed. The only deliberate stricter default is
that trainlens check --against fails when a baseline metric disappears.
Use --allow-removed if this is an intentional change.

Requires Python 3.11–3.14.

TrainLens 0.19.1

TrainLens 0.19.1 — Agent safety and monitoring reliability

A backwards-compatible maintenance release for the Python-first TrainLens package.

Agent Mode safety and context guarantees

• Sanitize portable-run parameters, notes, model names and run identifiers before exposing them as citable evidence to external coding agents.
• Use percent-encoded parameter evidence IDs to distinguish names such as learning rate and learning_rate.
• Keep the generated agent Markdown within ContextPolicy.max_chars; with very small budgets, compact evidence sections only contain actually visible evidence IDs. Output schema and instructions remain available through structured AgentContext fields.
• Ensure dataset-context truncation never overflows the configured character budget.

Monitoring performance

• Avoid creating a full immutable copy of the training-observation history on each monitor step when only built-in detectors are configured.
• Custom detectors continue receiving the same complete immutable history snapshot.

Compatibility and tests

• Public Python analysis, notebook, plotting, PDF, CLI and Agent Mode APIs remain available.
• Add regression tests covering privacy, evidence ID uniqueness, character-budget edge cases and monitor history copying.
• The source remains pure Python: native Rust acceleration is not part of this release.

Upgrade with pip install --upgrade trainlens==0.19.1.

TrainLens 0.19.0

TrainLens 0.19.0 — One install, complete core features

One-command installation

pip install trainlens

No separate plotting or PDF extra is necessary. The default TrainLens
installation includes:

• deterministic notebook/CLI analysis, Agent Mode and metric-history diagnostics
• Matplotlib training-curve plots
• ReportLab PDF report export (plus Markdown, HTML and JSON)
• the built-in OpenAI-compatible client for optionally generated LLM reports

The old trainlens[plots] and trainlens[pdf] selectors remain valid,
no-op compatibility aliases for existing projects.

Scope and caveats

Third-party ML training frameworks (PyTorch, TensorFlow, Transformers,
Lightning), a running LLM endpoint, a chosen model, and provider credentials
are not automatically installed. TrainLens uses existing training state
and remains fully usable offline without a provider.

Reliability and documentation

• Set ReportLab's minimum to a version that does not require Cairo for
  ordinary PDF rendering.
• Add fresh-install plot and PDF tests to package CI and the release wheel smoke
  test; preserve lazy plot rendering and headless CI compatibility.
• Update the README, Colab quickstart, and English/Spanish guides to describe
  the real installation, optional model integration and Python 3.11–3.14 support.
• Guard against stale instructions requiring plot/PDF extras.

Upgrade: pip install --upgrade trainlens==0.19.0.

TrainLens 0.18.1

TrainLens 0.18.1 — Robust training history and clearer plots

This patch fixes a history-sketch edge case and improves the visual diagnosis of
long model-training runs without changing the portable history format.

Bug fixes

• Fix large-offset metric trajectories whose real local changes could previously
  be hidden by a tolerance derived from the absolute metric baseline.
• Use explicit observation labels when training-step metadata is incomplete;
  forced step mode preserves unknown positions as missing instead of fabricating
  step coordinates.

Better notebook visuals

• Bound long plotted histories with deterministic min/max buckets, keeping isolated
  spikes/troughs and original endpoints; stored metric series are not modified.
• Avoid thousands of distracting point markers on dense training curves.
• Mark the best validation observation; show the final held-out test result with a
  distinctive symbol so it is not confused with a training trajectory.
• New optional plot_training_curves(..., x_axis="auto", max_plot_points=1200, show_best=True) controls; Matplotlib remains an optional install.

Documentation and tests

• Extend the Colab quickstart with a reproducible 1,000-step demonstration and a
  bounded, provider-free notebook-context preview.
• Update the README, Python API guide, and English/Spanish notebook documentation.
• Regression tests cover large-offset histories, missing steps, long-run rendering,
  validation-vs-test behavior and notebook source compilation.

Upgrade with pip install --upgrade trainlens==0.18.1.

TrainLens 0.18.0

TrainLens 0.18.0 — Signal-aware, token-efficient history

TrainLens 0.18.0 lets AI agents understand long training histories using small,
deterministic curve sketches instead of flooding a context window with all steps.

Highlights

• Signal-aware sampling replaces uniform snapshots: preserve endpoints, both global
  extrema when max_metric_points >= 4, and high-deviation local changes.
• Adaptive token savings: nearly linear or flat histories can use only two endpoint samples rather than always consuming 12 slots; more complex curves use more slots.
• 100, 200, and 1,000 epochs remain small in Agent Mode and direct LLM context.
  Full metric trajectories and explicit step metadata stay intact in portable runs.
• Progressive inspection with inspect_history_window() lets agents drill down
  into a narrow interval of the full local series on demand.
• Public, deterministic APIs: HistoryDigest, summarize_metric_history(),
  and inspect_history_window().
• Evidence and privacy: original observation positions, unknown steps, and
  train/validation/test separation are preserved; no new provider dependency.
• Documentation and tests cover diagnostic spikes, different history lengths,
  bounded samples, and exact window selection.

Usage

from trainlens import load_run, summarize_metric_history, inspect_history_window

run = load_run("candidate.json")
metric = next(m for m in run.metrics if m.name == "loss" and m.split == "validation")
summary = summarize_metric_history(metric, max_points=12)
focus = inspect_history_window(metric, start=100, end=180, max_points=8)

The number of observations and context characters is bounded, not an exact
provider-specific token count. The summary is intentionally lossy; local portable
history remains lossless and is always available for deeper inspection.

TrainLens 0.17.0

TrainLens 0.17.0

TrainLens 0.17.0 focuses on lossless experiment evidence and safer downstream decisions.

Portable runs now retain complete metric trajectories and steps, and every consumer keeps training, validation, and held-out test metrics distinct. Ambiguous metric selectors fail explicitly rather than silently collapsing splits.

This release also tightens the experimental layer: repeated-seed parameter effects avoid pseudo-replication, multi-objective leaderboards only rank runs with complete objective data, portable JSON rejects non-finite/boolean metric values, Agent Context is deeply immutable, and Project writes are serialized for parallel training jobs.

Evidence identifiers for summaries, signals, and dataset findings are now semantic rather than positional, making agent plans less sensitive to simple finding reordering.

TrainLens 0.16.1

TrainLens 0.16.1 — clearer LLM and agent workflows

TrainLens 0.16.1 simplifies the README and makes the three supported workflows explicit from the start.

Highlights

• put the end-to-end Colab quickstart notebook at the top of the README
• distinguish local deterministic analysis, Agent Mode, and direct LLM reports
• clarify that Agent Mode can use any LLM/provider available to the surrounding agent
• explain that Claude, Codex, Cursor, or another host agent normally owns that LLM connection, avoiding duplicate provider configuration inside TrainLens
• make clear that Agent Mode is optional: TrainLens can still call an OpenAI-compatible provider directly for generated reports, diagnoses, and improvement ideas
• shorten the README substantially and focus it on the main usage paths

No Agent Mode API behavior changes in this patch; this release clarifies how the existing capabilities fit together.

TrainLens 0.16.0

TrainLens 0.16.0 — Agent Mode

TrainLens 0.16.0 adds a provider-free Agent Mode for coding agents and AutoResearch-style ML
skills.

The design keeps responsibilities explicit: TrainLens produces bounded deterministic evidence;
the already-running agent (Claude, Codex, Cursor, or another runtime) performs the reasoning.
TrainLens does not call a second LLM in Agent Mode, so no TrainLens-side model provider, SDK, or
API key is required.

Highlights

• add public AgentContext with Markdown and JSON serialization
• add build_agent_context() for live Python/Jupyter state
• add build_agent_context_from_run() for portable TrainingRun files
• include stable deterministic evidence IDs, agent instructions, and a machine-readable output
  schema in every agent context
• add verify_agent_plan() to validate structured external-agent recommendations against real
  TrainLens evidence IDs
• add trainlens agent-context for terminal-first Skill workflows
• add trainlens verify-agent-plan for machine-checkable plan verification
• include recorded portable-run parameters and notes as citeable evidence
• document a recommended evidence → hypothesis → controlled experiment → evidence loop
• add a reusable examples/agent-skill/SKILL.md template
• keep the existing optional direct LLM provider integration fully available and backwards
  compatible

Agent Mode is deliberately not AutoML. It gives agents a reliable evidence boundary so they can
decide when to run a normal experiment, use HPO, or stop, rather than defaulting to blind search.

TrainLens 0.15.1

Image and multimodal dataset support

TrainLens 0.15.1 extends explain_dataset() with dependency-light support for image datasets and image-centered multimodal datasets.

Image evidence without decoding or uploading media

TrainLens can recognize common visual representations already present in Python or Jupyter, including PIL-like objects, NumPy/PyTorch-like image shapes, Hugging Face-style {"path", "bytes"} image values, common image-path extensions, and rows containing multiple images.

The resulting aggregate evidence can include:

• image counts and images per row;
• available width/height ranges and means;
• aspect-ratio summaries;
• channel counts and image modes;
• observed image formats;
• variable image dimensions;
• material mean image-size differences across splits.

No Pillow, NumPy, PyTorch, or Hugging Face dependency is required by TrainLens for this analysis.

Multimodal dataset summaries

Image features can now be combined with text or tabular features and are represented through MultimodalDatasetExplanation, DatasetModalitySummary, and DatasetImageSummary.

from trainlens import explain_dataset

context = explain_dataset(
    {
        "image": images,
        "caption": captions,
        "label": labels,
    },
    target="label",
    name="train",
)

print(context.is_multimodal)
print(context.modality_summaries[0].modalities)

This covers common image+text, image+metadata, VLM/VQA-style, document-image, and multi-image-row workflows.

Privacy-preserving context for result explanations

Visual dataset context remains aggregate-only. TrainLens does not render or implicitly send pixels, image bytes, private file paths, or filenames. Image and modality observations can flow into request previews and verified improvement plans as deterministic evidence, while the underlying media remains outside the outbound context.

Documentation and validation

The README and dataset-aware analysis guide now document image and multimodal workflows and include a dependency-free runnable example. CI covers Python 3.11–3.14 plus declared minimum dependencies, including privacy tests for outbound multimodal context.

Implementation, privacy coverage, and documentation were reviewed separately in pull requests #139, #140, and #141 before this release preparation.