feat: LLM-friendly output formats (TAP + JSON)#382
Merged
Conversation
Approved design for TAP + JSON output formats: polymorphic Formatter family, output config key + -o/--output CLI flag, output contracts, testing and docs plan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bite-sized TDD plan: base Formatter refactor, TapFormatter, JsonFormatter, FormatterFactory, output config + CLI flag, docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure refactor under green tests: base Formatter defines the event protocol with no-op defaults and protected helpers; ConsoleFormatter holds the existing human output unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Also drive the run timer in the formatter test helper so duration is realistic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CLI ordering validation only whitelisted -d/-e as value-taking flags, so a value following -o/--output was rejected as a stray path param. Include the output identifiers in the whitelist and add a regression test through getPathsAndConfigurationParams. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CLAUDE.md documents commands, architecture, conventions, and extension points for AI-assisted development in this repo - doc/plans/ added to .gitignore; AI-generated implementation plans are ephemeral and should not be part of the permanent history - Remove the two plan files from the index (files remain on disk) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extracts the text-collapsing logic from TapFormatter into a reusable normalizeToSingleLine() function in lib/utils/formatting.js, per ADR-0002. Uses split/join instead of regex to avoid the ReDoS hotspot (S5852) flagged by SonarCloud. Adds 5 tests covering the new utility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ngarbezza
added a commit
that referenced
this pull request
Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #380 — the first quick win of the AI roadmap (see
doc/ai-ideas.md). Adds machine-readable output so an LLM agent (or any tool) can consume Testy's results with few tokens. Driver: simplicity as cost.What's included
Formatterfamily: a baseFormatter(event protocol + protected helpers, no-op defaults) with three subclasses:ConsoleFormatter— the existing human-readable colored output (default, unchanged).TapFormatter— TAP version 13, streaming one line per test.JsonFormatter— a single structured JSON object printed at the end.FormatterFactoryselects the formatter;ConsoleUIwires it fromconfiguration.output().outputkey in.testyrc.json(defaultconsole) and-o/--output <console|tap|json>CLI flag, with validation.--helpupdated.README.md+README_es.md(config option, CLI flag, "machine-readable output" section).Machine output is language-neutral (statuses/keys are not translated); only failure messages are localized to the configured language. AI lives entirely outside the core — zero new runtime dependencies.
Design / plan
doc/plans/2026-06-06-llm-friendly-output-design.mddoc/plans/2026-06-06-llm-friendly-output.mdExamples
TAP:
JSON (single line):
{ "tool": "@pmoo/testy", "version": "...", "summary": { total, passed, failed, errored, pending, skipped, durationMs }, "suites": [ { name, file, tests: [ { name, status, failure? } ] } ] }Testing
-o tap,-o json, and default.Known limitations / follow-ups (non-blocking)
message:collapses internal newlines to spaces (error stack traces become one line).outputvalue coming from.testyrc.json(not the CLI) silently falls back toconsole, matching howlanguageis handled today.displayError(top-level runtime errors during a run) is a no-op in TAP/JSON mode — machine consumers rely on the non-zero exit code; emitting aBail out!/{"error":...}could be a follow-up.# timeassertions.Closes #380.
🤖 Generated with Claude Code