Recursive comparison api improvements

[joel-costigliola]

Recursive comparison api

Here's an example to give users an idea of what it looks like:

assertThat(person).usingRecursiveComparison()
                  .ignoringNullFields()
                  .ignoringFields("name", "age")                                                 
                  .forcingRecursiveComparisonFor(Child.class, Sister.class)
                  .isEqualTo(expectedPerson);

This will be in Beta as it might change according to people's feedback.

Requirements

Recursive comparison Fluent API configuration

Implements an easy to use/discover API as in:

assertThat(person).usingRecursiveComparison()
                  .ignoringNullFields()
                  .ignoringFields("name", "age")                                                 
                  .isEqualTo(expectedPerson);

• expose ignoring actual null fields
• expose ignoring fields
• expose ignoring overridden equals
• expose setting a comparison strategy per fields/types
• expose enabling strict type checking
• documentation

Ignoring fields in comparison

Users can specify the comparison to ignore all null fields in the object under test.

Users can specify to ignore specific fields in the object under test:

• giving their full path, ex: person.sister.name
• by regex, ex: .*name means name, person.name and person.sister.name are ignored.

Note if you ignore specific field, all sub fields are ignored too, for example if person is ignored then person.name, person.address, person.address.number are ignored too.

Nice to have: report all the fields ignored in the comparison

status

• ignore all null fields
• ignore fields specified with a full path
• fail if an ignored fields specified with a full path does not match any field
• ignore fields specified with a regex
• report the fields configured to be ignored in the comparison
• report all the fields actually ignored in the comparison (to show which fields the regex matched)
• documentation

Allow to use overridden equals method or not

By default the recursive comparison will use equals if it had been overridden but users should be able to specify to ignore overridden equals methods and compare objects field by field.

This can be done:

• for all types matching given regexes
• a specific given list of types with forcingRecursiveComparisonFor(Child.class, Sister.class)
• a specific given list of fields with forcingRecursiveComparisonForFields("child", "sister") Nice to have

Example:

assertThat(person).usingRecursiveComparison()
                  .forcingRecursiveComparisonFor(Child.class, Sister.class)
                  .forcingRecursiveComparisonForFields("name", "city")
                  .isEqualTo(expectedPerson);

status

• disable use of overridden equals method for types matching given regexes
• disable use of overridden equals method for given types
• disable use of overridden equals method for given fields
• disable use of overridden equals method for all fields
• documentation

Specify a comparison strategy per type or fields

Users should be able to specify a comparator for a given type or fields:

• usingComparatorForType(String.class, caseInsensitiveComparator) for all types
• usingComparatorForFields(caseInsensitiveComparator, "name", "city") a specific given list of fields (vararg)

To keep the API simple once a comparator for a given type is registered it should be used at any level, collection element or collection element field.

Field comparators take precedence over type ones as they are more fine grained.

Once a comparator is registered for a type or a field, it replaces the recursive comparison when these types/fields are being compared.

status

• enable a comparison strategy per type
• enable a comparison strategy per fields
• documentation

Strict/lenient type checking

This will specify if two instances with the same fields but from different classes are considered equals, it allows for example to compare a Person with a PersonDto. If the check is strict the expected object class must be compatible (i.e. extends) actuals, so if Employee inherits Person one can compare a person with an employee (but not an employee with a person. The check is performed on the root objects and their fields.

By default the check is lenient but it can be made strict to fail the comparison.

status

• enable strict type checking
• lenient type checking should allow comparing collections of different type (ex TreeSet vs HashSet).
• documentation

Handle cycles

The recursive comparison should handle cycles, for example when a -> b -> c -> a.

status

• more tests with cycles
• documentation

Map support

Maps entries should be considered as fields.
See if it is possible and relevant to report all compared maps internal differences (size, missing elements, elements differences, order when order is consistent ...) instead just reporting that the maps differ.

status

On hold at the moment as it is not crystal what can be achieved reasonably.

• map support
• documentation

Iterable/array support

At the moment assertThat(Iterable/array) does not expose usingRecursiveComparison() which makes it cumbersome to test them field/field.

The comparison by default fails if we compare an ordered collection to an unordered one but this should be configurable

Nice to have: ignores order in comparison (by default elements are compared in order for ordered collection).
Nice to have: allow to compare ordered collections with unordered one. (this should be disabled by default)
Nice to have: allow to compare array with ordered collections like list. (this should be disabled by default)

Example:

assertThat(fellowshipOfTheRing).usingRecursiveComparison()
                               .contains(frodo, sam, merry, pippin, gandalf, 
                                         legolas, boromir, aragorn, gimli);

status

• provide elements recursive comparison out of the box for iterable assertions
• ignores order in comparison (by default elements are compared in order for ordered collection).
• allow to compare ordered collections with unordered one. (this should be disabled by default)
• allow to compare array with ordered collections like list. (this should be disabled by default)
• documentation

Error reporting

A failure must report the recursive comparison specification:

• comparators used per type
• comparators used per fields
• ignored fields
• when overridden equals are used and not used

The failure should show the number of differences and describe them all.

A difference describes fields by their full path from the root object, the actual value, the expected one and the comparison used if the user specified one.
Values are represented with AssertJ standard representation or the one set by the user.

Actual and expected values type should be displayed when strict type checking is enable.

The differences reported must be ordered by path alphabetically.

Difference report example:

person.sister.name differs: 
- actual value  : "Sansa"
- expected value: "Arya"
- comparison was performed with caseInsensitiveComparator 

The path to field must support maps and adress #1303.

status

• basic error reporting
• error reporting : show comparators used
• error reporting : list ignored fields
• error reporting : show that actual's null field were ignored in the comparison
• error reporting : show which overridden equals were used
• error reporting : the path to field must support maps
• error reporting : report the index of the indexes of ordered collection field element, ex: friends[1].number
• error reporting : the differences reported must be ordered by path alphabetically

Globally configuring the recursive comparison behavior

To avoid repeating the same recursive configuration before each assertion, AssertJ should provide:

• a global way to configure the default recursive comparison behavior
• a way capture easily on the recursive comparison configuration in order to reuse it

Changing the default recursive comparison behavior before all tests

AssertJ will extend the mechanism used to configure Representation.

Capture the recursive comparison setting to reuse it

If users don't want to change the default behavior globally but on a smaller scope, it will possible to capture a recursive comparison specification and reuse it.

Example:

// default behavior
RecursiveComparisonSpecification recursiveComparisonSpecification =  new RecursiveComparisonSpecification();
recursiveComparisonSpecification.ignoreNullFields();
recursiveComparisonSpecification.forceRecursiveComparisonFor(Child.class, Sister.class);

assertThat(person).usingRecursiveComparison(recursiveComparisonSpecification)
                  .isEqualTo(expectedPerson);

assertThat(person2).usingRecursiveComparison(recursiveComparisonSpecification)
                   .isEqualTo(expectedPerson2);

The example above is equivalent to this where we have to repeat call to ignoringNullFields and forcingRecursiveComparisonFor for each assertions:

assertThat(person).usingRecursiveComparison()
                  .ignoringNullFields()
                  .forcingRecursiveComparisonFor(Child.class, Sister.class)
                  .isEqualTo(expectedPerson);

assertThat(person2).usingRecursiveComparison(recursiveComparisonSpecification)
                   .ignoringNullFields()
                   .forcingRecursiveComparisonFor(Child.class, Sister.class)
                   .isEqualTo(expectedPerson2);

status

• provide a way to globally configure the recursive comparison behavior

Tests checklist

The recursive comparison specification must be transferred after calling methods that change the object under test: extracting, asString ...

status

• test recursive comparison is propagated

---

Initial issue description

The current way comparing objects recursively is starting to pollute the api, the methods introduced for the recursive comparison don't apply for the other assertion cluttering the assertj api, ex usingComparatorForType.

I would like to introduce a separate API to use recursive comparison allowing to fine tune the comparison behavior.

API rough draft (please criticize it !)

import static RecursiveComparisonSpecification;

// ignoringNullFields() comes from RecursiveComparisonSpecification
assertThat(person).usingRecursiveComparison(ignoringNullFields()
                                           .ignoringFields("name", "age")                                                 
                                           .forcingRecursiveComparisonFor(Child.class, Sister.class))
                  .isEqualTo(expectedPersont)

[PascalSchumacher]

Another option would be to let usingRecursiveComparison return a special Assert class (e.g. RecursiveComparisonAssert), so you could write

assertThat(person).usingRecursiveComparison()
     .ignoringNullFields()
     .ignoringFields("name", "age")                                                 
     .forcingRecursiveComparisonFor(Child.class, Sister.class)
     .isEqualTo(expectedPersont);

[ylegat]

I do like the design! But there is something that, I believe, is misleading (actually it is already the case with current version of AssertJ): usingRecursiveComparison expression does not say anything about the fact that the recursive comparator will not be used on attributes having a type overidding equals. Of course, it becomes clear if we introduce a forcingRecursiveComparisonFor method, but all users won't see it at first glance.

So, it would be nice to have a way to clarify that for users, either by changing usingRecursiveComparison to something else, either by forcing users to call a specific method in order to get a RecursiveComparisonSpecification instance. Going a step forward: maybe that specific method could be used to specify if the recursive comparison should be forced on any type ?

Here is my proposal:

import static RecursiveComparisonSpecificationFactory;

// forcingRecursiveComparisonForAll() comes from RecursiveComparisonSpecificationFactory
assertThat(person).usingRecursiveComparison(forcingRecursiveComparisonForAll()
                                           .ignoringFields("name", "age"))
                  .isEqualTo(expectedPerson)

// forcingRecursiveComparisonFor() comes from RecursiveComparisonSpecificationFactory
assertThat(person).usingRecursiveComparison(forcingRecursiveComparisonFor(Child.class, Sister.class)
                                           .ignoringFields("name", "age"))
                  .isEqualTo(expectedPerson)

You can only get a RecursiveComparisonSpecification instance by calling one of two methods on RecursiveComparisonSpecificationFactory:

• forcingRecursiveComparisonFor
• forcingRecursiveComparisonForAll

This way it will be clear for users how the comparison will actually be performed. What do you think ?

[PascalSchumacher]

@joel-costigliola usingComparatorForType and usingComparatorForFields are also used by the non-recursive field/property by field/property comparison methods (isEqualToComparingFieldByField, isEqualToIgnoringNullFields, isEqualToComparingOnlyGivenFields and isEqualToIgnoringGivenFields). In order to remove them from ObjectAssert we would have to add a similar API for non-recursive comparison.

[pierrefevrier]

@joel-costigliola I think you are right. Recursive comparison is one of the best feature of AssertJ and having a dedicated api make sense.
It would be nice to be able to explain the way recursive comparison should work for some attributes (for a given depth).
Maybe give a list of rules for each of those specials attributes could be a way to do so ?

[joel-costigliola]

thank you all for your feedback, the api and its behaviour are still opened to discussion and changes, let's continue brainstorming it.

I wonder if the field by field comparison should be the default even for classes having overriding equals, I'd like to have your opinion on this one guys. In that case we could add a useOverriddenEquals to change that maybe with whitelist/blacklist (if that's not much).

I like @PascalSchumacher suggestion, it will make the API easier to discover, we will go that way except if we encounter technical limitations.

@pierrefevrier can you elaborate ? do you refer to add information in the error message explaining how the comparison was performed and (obviously) what went wrong ? (yes we will do this).

[PascalSchumacher]

I think the custom equals of classes should be used by default, because otherwise it is a pain to compare two objects e.g. containing a collection object.

[jomora]

Hey,
what do you think about the possibility to decide whether field-by-field comparison or comparison using equals() for certain types or attributes should be used?

For types one could specify the classes similar to @PascalSchumacher's example:

assertThat(person).usingRecursiveComparison()
     .ignoringNullFields()
     .ignoringFields("name", "age")                                                 
     .forcingRecursiveFieldByFieldComparisonFor(Child.class, Sister.class)
     .forcingRecursiveEqualityComparisonFor(Mother.class, Father.class)
     .isEqualTo(expectedPersont);

For attributes the information in which type the attribute is located might be useful.
What about something like of() or memberOf():

assertThat(person).usingRecursiveComparison()
     .ignoringNullFields()
     .ignoringFields("name", "age")                                                 
//  of() and in() are aliases for memberOf()
     .forcingRecursiveFieldByFieldComparisonForField("sister").of(Person.class)
     .forcingRecursiveEqualityComparisonForField("child").memberOf(Father.class)
     .forcingRecursiveEqualityComparisonForField("mother").in(Child.class)
     .isEqualTo(expectedPersont);

This would result in the equality-comparison being used for the child attribute of class Father, but not for class Mother.
I think sth. in this direction could work out, but I surely haven't thought through it completely.

What's your opinion? Do you see pitfalls?

[GaspardPO]

As explained in #1054 :

when using isEqualToComparingFieldByFieldRecursively or isEqualToComparingFieldByField, the comparison is not done on class type, only on field value (and name).

So two POJO with the same fields, but from different classes are considered as equals.
It could be useful to test the object instance (recursively, or not)

Example

@Test
 public void test() {
     A a = new A(10);
     B b = new B(10);
     assertThat(a).isEqualToComparingFieldByFieldRecursively(b); 
// I would like this line to fail because A and B are different classes
// but it is not failing
 }

 private static class A {
     int x;

     public A(int x) {
         this.x = x;
     }
 }

 private static class B {
     int x;

     public B(int x) {
         this.x = x;
     }

 }

Same too when the object contains some other object :

  private static class Something {
        InnerObject inner;  // can be A or B

        public Something(InnerObject inner) {
            this.inner = inner;
        }
    }


Something someObjectContainingA = new Something(new A(10));
Something someObjectContainingB = new Something(new B(10));
assertThat(someObjectContainingA).isEqualToComparingFieldByFieldRecursively(someObjectContainingB); 

the 2 object have the same class, but they are composed of different object, and it could be interesting to check this field type.