Bug with recursive comparison of objects with String containing %

[abryantsev]

Summary

During migration from the usage of the unitils recursive comparison to the AssertJ (3.19.0) we trapped into the issue with flaky tests using .usingRecursiveComparison() feature. Unfortunately the summary of the broken test doesn't give too much explanation about the differences because of the not overridden toString() method and just looks like (very close example):

java.lang.AssertionError: 
      Expecting:
        [com.models.response.A@311aa36e]
      to be equal to:
        [com.models.response.A@1432b204]
      when recursively comparing field by field, but found the following difference:
  
      Top level actual and expected objects differ:
      - actual value   : [com.models.response.A@311aa36e]
      - expected value : [com.models.response.A@1432b204]
      The following actual elements were not matched in the expected SingletonSet:
        [com.models.response.A@311aa36e]
  
      The recursive comparison was performed with this configuration:
      - no overridden equals methods were used in the comparison (except for java types)
      - collection order was ignored in all fields in the comparison
      - these types were compared with the following comparators:
        - java.lang.Double -> DoubleComparator[precision=1.0E-15]
        - java.lang.Float -> FloatComparator[precision=1.0E-6]
        - java.math.BigDecimal -> org.assertj.core.util.BigDecimalComparator
      - actual and expected objects and their fields were compared field by field recursively even if they were not of the same type, this allows for example to compare a Person to a PersonDto (call strictTypeChecking(true) to change that behavior).

Trying to get more insights into the root of the differences I have overridden toString() method in all models in the hierarchy of the comparing objects and trapped into even more problems catching a bug with String formatting having % character.

The following snippet shows a part of the stack trace (JDK 15):

java.util.UnknownFormatConversionException: Conversion = 'F'
	at java.base/java.util.Formatter$FormatSpecifier.conversion(Formatter.java:2839)
	at java.base/java.util.Formatter$FormatSpecifier.<init>(Formatter.java:2865)
	at java.base/java.util.Formatter.parse(Formatter.java:2713)
	at java.base/java.util.Formatter.format(Formatter.java:2655)
	at java.base/java.util.Formatter.format(Formatter.java:2609)
	at java.base/java.lang.String.format(String.java:3292)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator$ComparisonState.addDifference(RecursiveComparisonDifferenceCalculator.java:86)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator.compareUnorderedIterables(RecursiveComparisonDifferenceCalculator.java:439)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator.determineDifferences(RecursiveComparisonDifferenceCalculator.java:237)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator.compareUnorderedIterables(RecursiveComparisonDifferenceCalculator.java:422)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator.determineDifferences(RecursiveComparisonDifferenceCalculator.java:237)
	at org.assertj.core.api.recursive.comparison.RecursiveComparisonDifferenceCalculator.determineDifferences(RecursiveComparisonDifferenceCalculator.java:185)
	at org.assertj.core.api.RecursiveComparisonAssert.determineDifferencesWith(RecursiveComparisonAssert.java:1201)
	at org.assertj.core.api.RecursiveComparisonAssert.isEqualTo(RecursiveComparisonAssert.java:160)

Example

To reproduce the bug the following code could be used. Without overridden toString() method the comparison fails but the summary is printed.

    private static class Tuple {

        private final String a;
        private final String b;

        public Tuple(String a, String b) {
            this.a = a;
            this.b = b;
        }

        @Override
        public String toString() {

            return "Tuple [a=" + a + ", b=" + b + "]";
        }
    }

    private static class TestObj {

        private final Iterable<Tuple> tuples;

        public TestObj(Iterable<Tuple> tuples) {
            this.tuples = tuples;
        }

        @Override
        public String toString() {

            return "TestObj [strings=" + tuples + "]";
        }
    }

    @Test
    void testComparison() {

        final TestObj first = new TestObj(Set.of(
            new Tuple("VtQh0ZAo%2FKCnQcirWL", "foo %"),
            new Tuple("%F", "VtQh0ZAo%2FKCnQcirWL")));
        final TestObj second = new TestObj(List.of(
            new Tuple("%F", "VtQh0ZAo%2FKCnQcirWL"),
            new Tuple("VtQh0ZAo%2FKCnQcirWL", "bar %")));

        assertThat(first)
            .usingRecursiveComparison()
            .ignoringCollectionOrder()
            .isEqualTo(second);
    }

Is it is a known bug or something new? Could you please advise a workaround if it is available? I couldn't find any looking into the source code. It looks like executing this code

    if (!unmatchedActualElements.isEmpty()) {
      String unmatched = format("The following actual elements were not matched in the expected %s:%n  %s",
                                expected.getClass().getSimpleName(), unmatchedActualElements);
      comparisonState.addDifference(dualValue, unmatched);
    }

it implicitly calls formatting of the string that represents summary of the unmatched elements that in turn can have special characters thus bug is imminent.

differences.add(new ComparisonDifference(dualValue, format(description, args)))

Thank you,
Andrii

[joel-costigliola]

Thanks for reporting this, this is likely a bug.