Skip to content

ConsoleLauncher should output diff of expected and actual String values #3139

@carolosf

Description

@carolosf

As a developer if I am running JUnit tests using the Console Launcher on the command line if two large strings are found to be different and I get "expected:" "but was:" if I want to see what was different I need to copy the strings and manually diff them.

It would be easier if there was --details flag or another flag (I think another flag is a better option) that would show me a diff between the two strings for actionable results. In most cases using an IDE is suitable but in workflows such as Bazel and CI it would be useful if a diff would be displayed.

Apologies if this is already supported didn't find anything obvious in the docs.

Activity

marcphilipp

marcphilipp commented on Feb 17, 2023

@marcphilipp
Member

Potential library to implement this: https://github.com/java-diff-utils/java-diff-utils

@carolosf We've briefly discussed this in our team call today and think this would be feasible since AssertionFailedError from opentest4j carries actual and expected in separate fields.

We'll have to take a closer look first to decide whether --details diff would be the right way of enabling this. Maybe it could even be enabled by default.

carolosf

carolosf commented on Feb 17, 2023

@carolosf
Author

@marcphilipp

If it becomes the default behaviour it might break IDEs and plug-ins - which is why I suggested a totally separate flag.

I didn't spend much time on it but I noticed previously that the Bazel plugin for IntelliJ does a regex for "expected but was" and splits the results.

Thanks for discussing though - if implemented it would significantly reduce my debugging time!

marcphilipp

marcphilipp commented on Apr 22, 2023

@marcphilipp
Member

@carolosf IDEs and build tools usually do not run tests via ConsoleLauncher but use the Launcher API directly so this shouldn't be a problem.

changed the title [-]ConsoleLauncher output --details diff flag[/-] [+]ConsoleLauncher should output diff of expected and actual String values[/+] on May 5, 2023
marcphilipp

marcphilipp commented on May 5, 2023

@marcphilipp
Member

Team decision: Change ConsoleLauncher output for test failures to include diff if actual and expected of AssertionFailedError are of type CharSequence.

added this to the 5.11 M1 milestone on May 5, 2023
marcphilipp

marcphilipp commented on May 5, 2023

@marcphilipp
Member

@carolosf Would you be interested to work on a PR for this?

45 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Participants

    @sbrannen@marcphilipp@bechte@carolosf@itkhanz

    Issue actions

      ConsoleLauncher should output diff of expected and actual String values · Issue #3139 · junit-team/junit-framework