Skip to content

Report elapsed time in JUnit XML#146

Merged
fabian-barney merged 6 commits into
mainfrom
codex/128-junit-report-metadata
May 15, 2026
Merged

Report elapsed time in JUnit XML#146
fabian-barney merged 6 commits into
mainfrom
codex/128-junit-report-metadata

Conversation

@fabian-barney
Copy link
Copy Markdown
Owner

Summary

  • classify the JUnit timing finding as partially valid and report elapsed analysis time in JUnit XML
  • keep errors=0 unchanged because threshold breaches are modeled as JUnit failures and execution errors do not emit a report
  • set suite/testsuite time from elapsed seconds and distribute testcase time as a best-effort share

Closes #128

Verification

  • mvn -pl core test
  • mvn verify
  • git diff --check

Copilot AI review requested due to automatic review settings May 15, 2026 18:42
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds elapsed-duration reporting to JUnit XML output so CI consumers can display non-zero suite and testcase times.

Changes:

  • Adds elapsedSeconds to CrapReport with a helper for System.nanoTime() deltas.
  • Uses elapsed time for JUnit testsuites, testsuite, and per-testcase time attributes.
  • Updates formatter and integration tests for the new dynamic time values.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
core/src/main/java/media/barney/crap/core/CrapReport.java Carries elapsed seconds on reports.
core/src/main/java/media/barney/crap/core/Main.java Records elapsed time for pre-resolved module runs.
core/src/main/java/media/barney/crap/core/ReportFormatter.java Formats elapsed time into JUnit XML attributes.
core/src/test/java/media/barney/crap/core/ReportFormatterTest.java Tests elapsed JUnit formatting.
core/src/test/java/media/barney/crap/core/MainTest.java Updates JUnit sidecar assertions for dynamic time values.
Comments suppressed due to low confidence (5)

core/src/test/java/media/barney/crap/core/MainTest.java:258

  • Like the earlier integration assertion, this only proves that a time attribute was emitted; it still passes for the default 0.0 value produced when elapsed time is not propagated through CliApplication. Add a non-flaky check that this CLI path uses a real elapsed duration.
        assertTrue(junit.contains("<testsuites tests=\"3\" failures=\"1\" errors=\"0\" skipped=\"1\" time=\""));

core/src/test/java/media/barney/crap/core/MainTest.java:289

  • This weakened assertion accepts any time value, including the default 0.0, so it does not cover the new behavior for this CLI path. The test should verify that elapsed time is actually populated in the JUnit sidecar rather than just checking for the attribute prefix.
        assertTrue(junit.contains("<testsuites tests=\"3\" failures=\"1\" errors=\"0\" skipped=\"1\" time=\""));

core/src/test/java/media/barney/crap/core/MainTest.java:323

  • This assertion does not catch elapsed-time propagation failures because time="0.0" also satisfies it. Since agent mode still emits a JUnit sidecar through the CLI path, the test should check a deterministic elapsed value or otherwise prove that the sidecar is not using the default report time.
        assertTrue(junit.contains("<testsuites tests=\"3\" failures=\"1\" errors=\"0\" skipped=\"1\" time=\""));

core/src/test/java/media/barney/crap/core/MainTest.java:392

  • This sidecar assertion is too broad for the new timing behavior: it passes even when the full JUnit report still contains the default time="0.0". Add coverage that verifies elapsed time is propagated while keeping the sidecar complete under --failures-only.
        assertTrue(junit.contains("<testsuites tests=\"3\" failures=\"1\" errors=\"0\" skipped=\"1\" time=\""));

core/src/test/java/media/barney/crap/core/MainTest.java:520

  • This integration check also accepts the old/default time="0.0" output, so it does not verify that the legacy pre-resolved-module overload attaches elapsed time before publishing the JUnit XML. Consider adding a deterministic timing hook or parsing the XML and asserting the expected elapsed value for this path.
        assertTrue(Files.readString(junitReport).contains("<testsuites tests=\"3\" failures=\"1\" errors=\"0\" skipped=\"1\" time=\""));

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core/src/main/java/media/barney/crap/core/Main.java
Comment thread core/src/test/java/media/barney/crap/core/MainTest.java Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comment thread core/src/main/java/media/barney/crap/core/ReportFormatter.java
Comment thread core/src/main/java/media/barney/crap/core/ReportFormatter.java
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

…t-metadata

# Conflicts:
#	core/src/test/java/media/barney/crap/core/MainTest.java
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

@fabian-barney fabian-barney merged commit d171f57 into main May 15, 2026
17 checks passed
@fabian-barney fabian-barney deleted the codex/128-junit-report-metadata branch May 15, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Info] code-quality: ReportFormatter JUnit XML always emits time='0' and errors=0

2 participants