Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

equalToJson ignoreArrayOrder fails sometimes (version wiremock-jre8:2.25.1) #1230

Closed
pjungermann opened this issue Nov 28, 2019 · 5 comments
Closed

equalToJson ignoreArrayOrder fails sometimes (version wiremock-jre8:2.25.1) #1230

pjungermann opened this issue Nov 28, 2019 · 5 comments
Labels
bug

Comments

@pjungermann
Copy link

Bug report

Which version of WireMock you're using

  • wiremock-jre8:2.25.1
  • junit5
  • micronaut

How you're starting and configuring WireMock, including configuration or CLI the command line
slightly "anonymized" version of the used code to create the stub

    private getAll(Set<String> ids, Object response) {
        wireMockServer.givenThat(
             getAllMapping(ids, response)
        );
    }

    private MappingBuilder getAllMapping(Set<String> ids, Object response) throws JsonProcessingException {
        return post("/path")
                .withRequestBody(equalToJson(
                        asJsonString(ids),
                        true,
                        true
                ))
                .willReturn(
                        okJson(asJsonString(response))
                );
    }

A failing test case that demonstrates the problem
The problem is a bit indeterministic as it seems and randomly fails. I tried to debug such an occurrence.

registered stub mappings (serialized as JSON):

{
  "all":[
    {
      "id":"a9203a3f-6a12-47dc-8adc-057237bc9d5d",
      "name":null,
      "request":{
        "url":"/path",
        "urlPattern":null,
        "urlPath":null,
        "urlPathPattern":null,
        "method":"POST",
        "headers":null,
        "queryParameters":null,
        "cookies":null,
        "bodyPatterns":[
          {
            "equalToJson":"[\"b13a0a87-a372-4ed0-b5dc-6f11ac19f97f\",\"1b34af89-9723-47a9-8e4d-b96abec064c0\",\"8cdb54e1-b995-4620-8622-a589213c1c7e\",\"9b028a8b-35f4-4a24-9cd3-cf54d221c7a4\",\"bb2fca7f-5b44-4de2-9fb7-8285dab919b1\",\"838ba1a9-0bec-4187-b401-a699b8e04ac9\",\"7622abde-b9b8-4410-8bf8-396dab79eed7\",\"d2b260eb-297a-4dd9-9be2-c71b142eaf36\"]",
            "ignoreArrayOrder":true,
            "ignoreExtraElements":true,
            "absent":null
          }
        ],
        "customMatcher":null,
        "multipartPatterns":null,
        "basicAuthCredentials":null
      },
      "response":{
        "status":200,
        "statusMessage":null,
        "body":"{\"some\":\"data\"}",
        "jsonBody":null,
        "base64Body":null,
        "bodyFileName":null,
        "headers":{
          "Content-Type":"application/json"
        },
        "additionalProxyRequestHeaders":null,
        "fixedDelayMilliseconds":null,
        "delayDistribution":null,
        "chunkedDribbleDelay":null,
        "proxyBaseUrl":null,
        "fault":null,
        "fromConfiguredStub":null
      },
      "uuid":"a9203a3f-6a12-47dc-8adc-057237bc9d5d",
      "persistent":null,
      "priority":null,
      "scenarioName":null,
      "requiredScenarioState":null,
      "newScenarioState":null,
      "postServeActions":null,
      "metadata":null,
      "insertionIndex":3
    },
    {
      "id":"8fc309f2-9189-4736-9830-c383882c56b7",
      "name":null,
      "request":{
        "url":"/path",
        "urlPattern":null,
        "urlPath":null,
        "urlPathPattern":null,
        "method":"POST",
        "headers":null,
        "queryParameters":null,
        "cookies":null,
        "bodyPatterns":[
          {
            "equalToJson":"[\"a56d8d09-9530-4cb5-a9c9-232b3dac48ea\",\"82d4a5e2-ab86-4390-85f1-e7cdd0a21b5a\",\"0c29c818-fae6-42c4-909a-d2531741bfa6\",\"533d32e5-4457-4841-8540-bfa4674a3914\"]",
            "ignoreArrayOrder":true,
            "ignoreExtraElements":true,
            "absent":null
          }
        ],
        "customMatcher":null,
        "multipartPatterns":null,
        "basicAuthCredentials":null
      },
      "response":{
        "status":200,
        "statusMessage":null,
        "body":"{\"some\":\"data\"}",
        "jsonBody":null,
        "base64Body":null,
        "bodyFileName":null,
        "headers":{
          "Content-Type":"application/json"
        },
        "additionalProxyRequestHeaders":null,
        "fixedDelayMilliseconds":null,
        "delayDistribution":null,
        "chunkedDribbleDelay":null,
        "proxyBaseUrl":null,
        "fault":null,
        "fromConfiguredStub":null
      },
      "uuid":"8fc309f2-9189-4736-9830-c383882c56b7",
      "persistent":null,
      "priority":null,
      "scenarioName":null,
      "requiredScenarioState":null,
      "newScenarioState":null,
      "postServeActions":null,
      "metadata":null,
      "insertionIndex":2
    },
    {
      "id":"d07c20cd-d63b-4e03-9087-186efc6b8350",
      "name":null,
      "request":{
        "url":"/path",
        "urlPattern":null,
        "urlPath":null,
        "urlPathPattern":null,
        "method":"POST",
        "headers":null,
        "queryParameters":null,
        "cookies":null,
        "bodyPatterns":[
          {
            "equalToJson":"[\"182d15a6-7f7f-40bc-9b2c-ab1cdd493c4d\",\"68c56893-e87a-4fc9-be1d-9ccaba199bec\"]",
            "ignoreArrayOrder":true,
            "ignoreExtraElements":true,
            "absent":null
          }
        ],
        "customMatcher":null,
        "multipartPatterns":null,
        "basicAuthCredentials":null
      },
      "response":{
        "status":200,
        "statusMessage":null,
        "body":"{\"some\":\"data\"}",
        "jsonBody":null,
        "base64Body":null,
        "bodyFileName":null,
        "headers":{
          "Content-Type":"application/json"
        },
        "additionalProxyRequestHeaders":null,
        "fixedDelayMilliseconds":null,
        "delayDistribution":null,
        "chunkedDribbleDelay":null,
        "proxyBaseUrl":null,
        "fault":null,
        "fromConfiguredStub":null
      },
      "uuid":"d07c20cd-d63b-4e03-9087-186efc6b8350",
      "persistent":null,
      "priority":null,
      "scenarioName":null,
      "requiredScenarioState":null,
      "newScenarioState":null,
      "postServeActions":null,
      "metadata":null,
      "insertionIndex":1
    },
    {
      "id":"4086da7e-40a9-4ace-8d16-0c6a43c12be9",
      "name":null,
      "request":{
        "url":"/other-path/3b29794d-f451-419e-a519-94a8937bd995",
        "urlPattern":null,
        "urlPath":null,
        "urlPathPattern":null,
        "method":"GET",
        "headers":null,
        "queryParameters":null,
        "cookies":null,
        "bodyPatterns":null,
        "customMatcher":null,
        "multipartPatterns":null,
        "basicAuthCredentials":null
      },
      "response":{
        "status":200,
        "statusMessage":null,
        "body":"{\"some\":\"data\"}",
        "jsonBody":null,
        "base64Body":null,
        "bodyFileName":null,
        "headers":{
          "Content-Type":"application/json"
        },
        "additionalProxyRequestHeaders":null,
        "fixedDelayMilliseconds":null,
        "delayDistribution":null,
        "chunkedDribbleDelay":null,
        "proxyBaseUrl":null,
        "fault":null,
        "fromConfiguredStub":null
      },
      "uuid":"4086da7e-40a9-4ace-8d16-0c6a43c12be9",
      "persistent":null,
      "priority":null,
      "scenarioName":null,
      "requiredScenarioState":null,
      "newScenarioState":null,
      "postServeActions":null,
      "metadata":null,
      "insertionIndex":0
    }
  ],
  "allScenarios":[

  ]
}

request which was unmatched (serialized as JSON):

{
  "method":"POST",
  "port":57971,
  "host":"localhost",
  "scheme":"http",
  "originalRequest":{
    "present":false
  },
  "bodyAsBase64":"WyJiMTNhMGE4Ny1hMzcyLTRlZDAtYjVkYy02ZjExYWMxOWY5N2YiLCIxYjM0YWY4OS05NzIzLTQ3YTktOGU0ZC1iOTZhYmVjMDY0YzAiLCI4Y2RiNTRlMS1iOTk1LTQ2MjAtODYyMi1hNTg5MjEzYzFjN2UiLCI5YjAyOGE4Yi0zNWY0LTRhMjQtOWNkMy1jZjU0ZDIyMWM3YTQiLCI4MzhiYTFhOS0wYmVjLTQxODctYjQwMS1hNjk5YjhlMDRhYzkiLCJiYjJmY2E3Zi01YjQ0LTRkZTItOWZiNy04Mjg1ZGFiOTE5YjEiLCJkMmIyNjBlYi0yOTdhLTRkZDktOWJlMi1jNzFiMTQyZWFmMzYiLCI3NjIyYWJkZS1iOWI4LTQ0MTAtOGJmOC0zOTZkYWI3OWVlZDciXQ==",
  "multipart":false,
  "parts":null,
  "headers":{
    "Accept":"application/json",
    "Connection":"close",
    "Host":"localhost:57971",
    "Content-Length":"313",
    "Content-Type":"application/json"
  },
  "cookies":{

  },
  "body":"WyJiMTNhMGE4Ny1hMzcyLTRlZDAtYjVkYy02ZjExYWMxOWY5N2YiLCIxYjM0YWY4OS05NzIzLTQ3YTktOGU0ZC1iOTZhYmVjMDY0YzAiLCI4Y2RiNTRlMS1iOTk1LTQ2MjAtODYyMi1hNTg5MjEzYzFjN2UiLCI5YjAyOGE4Yi0zNWY0LTRhMjQtOWNkMy1jZjU0ZDIyMWM3YTQiLCI4MzhiYTFhOS0wYmVjLTQxODctYjQwMS1hNjk5YjhlMDRhYzkiLCJiYjJmY2E3Zi01YjQ0LTRkZTItOWZiNy04Mjg1ZGFiOTE5YjEiLCJkMmIyNjBlYi0yOTdhLTRkZDktOWJlMi1jNzFiMTQyZWFmMzYiLCI3NjIyYWJkZS1iOWI4LTQ0MTAtOGJmOC0zOTZkYWI3OWVlZDciXQ==",
  "url":"/path",
  "absoluteUrl":"http://localhost:57971/path",
  "clientIp":"127.0.0.1",
  "browserProxyRequest":false,
  "bodyAsString":"[\"b13a0a87-a372-4ed0-b5dc-6f11ac19f97f\",\"1b34af89-9723-47a9-8e4d-b96abec064c0\",\"8cdb54e1-b995-4620-8622-a589213c1c7e\",\"9b028a8b-35f4-4a24-9cd3-cf54d221c7a4\",\"838ba1a9-0bec-4187-b401-a699b8e04ac9\",\"bb2fca7f-5b44-4de2-9fb7-8285dab919b1\",\"d2b260eb-297a-4dd9-9be2-c71b142eaf36\",\"7622abde-b9b8-4410-8bf8-396dab79eed7\"]",
  "allHeaderKeys":[
    "Accept",
    "Connection",
    "Host",
    "Content-Length",
    "Content-Type"
  ]
}

stubMappings.serverFor(request)

in com.github.tomakehurst.wiremock.core.WireMockApp#serveStubFor returned a ServeEvent which was isNoExactMactch() (no response definition configured / 404)

resulting in an error log

[...] ERROR WireMock - Request was not matched as there were no stubs registered:
{
  "url" : "/contents/bulk/get",
  "absoluteUrl" : "http://localhost:57971/path,
  "method" : "POST",
  "clientIp" : "127.0.0.1",
  "headers" : {
    "Accept" : "application/json",
    "Connection" : "close",
    "Host" : "localhost:57971",
    "Content-Length" : "313",
    "Content-Type" : "application/json"
  },
  "cookies" : { },
  "browserProxyRequest" : false,
  "loggedDate" : 1574982595892,
  "bodyAsBase64" : "WyJiMTNhMGE4Ny1hMzcyLTRlZDAtYjVkYy02ZjExYWMxOWY5N2YiLCIxYjM0YWY4OS05NzIzLTQ3YTktOGU0ZC1iOTZhYmVjMDY0YzAiLCI4Y2RiNTRlMS1iOTk1LTQ2MjAtODYyMi1hNTg5MjEzYzFjN2UiLCI5YjAyOGE4Yi0zNWY0LTRhMjQtOWNkMy1jZjU0ZDIyMWM3YTQiLCI4MzhiYTFhOS0wYmVjLTQxODctYjQwMS1hNjk5YjhlMDRhYzkiLCJiYjJmY2E3Zi01YjQ0LTRkZTItOWZiNy04Mjg1ZGFiOTE5YjEiLCJkMmIyNjBlYi0yOTdhLTRkZDktOWJlMi1jNzFiMTQyZWFmMzYiLCI3NjIyYWJkZS1iOWI4LTQ0MTAtOGJmOC0zOTZkYWI3OWVlZDciXQ==",
  "body" : "[\"b13a0a87-a372-4ed0-b5dc-6f11ac19f97f\",\"1b34af89-9723-47a9-8e4d-b96abec064c0\",\"8cdb54e1-b995-4620-8622-a589213c1c7e\",\"9b028a8b-35f4-4a24-9cd3-cf54d221c7a4\",\"838ba1a9-0bec-4187-b401-a699b8e04ac9\",\"bb2fca7f-5b44-4de2-9fb7-8285dab919b1\",\"d2b260eb-297a-4dd9-9be2-c71b142eaf36\",\"7622abde-b9b8-4410-8bf8-396dab79eed7\"]",
  "scheme" : "http",
  "host" : "localhost",
  "port" : 57971,
  "loggedDateString" : "2019-11-28T23:09:55Z",
  "queryParams" : { }
}

and

[...] ERROR WireMock - 
                                               Request was not matched
                                               =======================

-----------------------------------------------------------------------------------------------------------------------
| Closest stub                                             | Request                                                  |
-----------------------------------------------------------------------------------------------------------------------
                                                           |
POST                                                       | POST
/path                                         | /path
                                                           |
[ "b13a0a87-a372-4ed0-b5dc-6f11ac19f97f",                  | [ "b13a0a87-a372-4ed0-b5dc-6f11ac19f97f",           <<<<< Body does not match
"1b34af89-9723-47a9-8e4d-b96abec064c0",                    | "1b34af89-9723-47a9-8e4d-b96abec064c0",
"8cdb54e1-b995-4620-8622-a589213c1c7e",                    | "8cdb54e1-b995-4620-8622-a589213c1c7e",
"9b028a8b-35f4-4a24-9cd3-cf54d221c7a4",                    | "9b028a8b-35f4-4a24-9cd3-cf54d221c7a4",
"bb2fca7f-5b44-4de2-9fb7-8285dab919b1",                    | "838ba1a9-0bec-4187-b401-a699b8e04ac9",
"838ba1a9-0bec-4187-b401-a699b8e04ac9",                    | "bb2fca7f-5b44-4de2-9fb7-8285dab919b1",
"7622abde-b9b8-4410-8bf8-396dab79eed7",                    | "d2b260eb-297a-4dd9-9be2-c71b142eaf36",
"d2b260eb-297a-4dd9-9be2-c71b142eaf36" ]                   | "7622abde-b9b8-4410-8bf8-396dab79eed7" ]
                                                           |
-----------------------------------------------------------------------------------------------------------------------

as you can see, the last 2 pairs of entries in the array are swapped, but the set of values is the same -- hence ignoreArrayOrder=true should result in a positive match, but wasn't

Profiler data if the issue is performance related
not performance related as far as I can tell
@pjungermann
Copy link
Author

pjungermann commented Nov 29, 2019 •
edited

the indeterministic behavior comes mainly from the test suit generating random test values and some of those will not work due to what is described below.

Test case to reproduce the issue

import com.github.tomakehurst.wiremock.matching.EqualToJsonPattern;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.assertTrue;

class SimpleTest {

    @Test
    void reproduceFailingIgnoreArrayOrder() {
        var expected = "[\"a\",\"b\", \"c\",\"d\",\"e\",\"f\",\"g\",\"h\"]";
        var actual   = "[\"b\",\"a\", \"d\",\"c\",\"e\",\"f\",\"g\",\"h\"]";
        var pattern = new EqualToJsonPattern(expected, true, true);
        var result = pattern.match(actual);
        var exact = result.isExactMatch();
        assertTrue(exact);
    }
}

index 0+1 and 2+3 got swapped, 4..7 are equal values

com.github.tomakehurst.wiremock.matching.MatchResult#getDistance at EqualToJsonPattern.java:115 uses JsonDiff and then filters certain operations for ignoreArrayOrder=true and ignoreExtraElements=true

JsonDiff: JsonDiff.asJson(expected, actual, flags)

[ {
  "op" : "remove",
  "path" : "/0",
  "value" : "a"
}, {
  "op" : "replace",
  "path" : "/1",
  "value" : "a"
}, {
  "op" : "add",
  "path" : "/3",
  "value" : "c"
} ]

com.github.tomakehurst.wiremock.matching.EqualToJsonPattern#diffSize

    private boolean extraElementsIgnoredAndIsAddition(String operation) {
        return operation.equals("add") && shouldIgnoreExtraElements();
    }

    private boolean arrayOrderIgnoredAndIsArrayMove(String operation, List<String> path) {
        return operation.equals("move") && isNumber(getLast(path)) && shouldIgnoreArrayOrder();
    }

for this ignoreArrayOrder=true case, only "move" operations will be ignored. The combination of remove + add + replace resulting in the same object is not supported. JsonDiff does not detect this as 4 move operations (which would require more operations than the resulting diff).

JsonDiff only normalized "add"+"remove" into "move" operations, but here we have a "replace" one.
com.flipkart.zjsonpatch.JsonDiff#compactDiffs

We probably would have expected something like

[ {
  "op" : "move",
  "from" : "/0",
  "path" : "/1"
}, {
  "op" : "move",
  "from" : "/2",
  "path" : "/3"
} ]

which then would work with the current code here.

@pjungermann pjungermann mentioned this issue Nov 29, 2019
Open
@pjungermann
Copy link
Author

pjungermann commented Nov 29, 2019 •
edited

I've also opened flipkart-incubator/zjsonpatch#113 in case the compactDiffs method could be improved in a way that it returns only move operations for such cases which then would only require to update the dependency.

It might not happen though and as alternative, a way to identify or hide the order in array changes would be great then.

@pjungermann
Copy link
Author

a simplistic approach using the sort order of the text representation of the JSON of all entries within all arrays contained, but probably not the most performant approach and also not the most performant way to write it

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.github.tomakehurst.wiremock.common.Json;
import com.github.tomakehurst.wiremock.matching.EqualToJsonPattern;
import org.junit.jupiter.api.Test;

import java.util.List;
import java.util.Map;
import java.util.stream.StreamSupport;

import static java.util.Comparator.comparing;
import static org.junit.jupiter.api.Assertions.assertTrue;

class SimpleTest {

    @Test
    void reproduceFailingIgnoreArrayOrder() {
        var expected = "[\"a\",\"b\",\"c\",\"d\",\"e\",\"f\",\"g\",\"h\"]";
        var actual = "[\"b\",\"a\",\"d\",\"c\",\"e\",\"f\",\"g\",\"h\"]";

        var pattern = new EqualToJsonPattern(expected, true, true);
        var result = pattern.match(actual);
        var exact = result.isExactMatch();
        assertTrue(exact);
    }

    @Test
    void withSortingApplied() {
        var expected = "[\"a\",\"b\",\"c\",\"d\",\"e\",\"f\",\"g\",\"h\"]";
        var actual = "[\"b\",\"a\",\"d\",\"c\",\"e\",\"f\",\"g\",\"h\"]";

        withSortingApplied(expected, actual);
    }

    @Test
    void withSortingApplied_2() {
        var list1 = List.of("c", "d", "a", "b");
        var list2 = List.of("d", "c", "b", "a");

        var data1 = Map.of("a", list1, "b", "text");
        var data2 = Map.of("b", "text", "a", list2);

        var expected = Json.write(data1);
        var actual = Json.write(data2);

        withSortingApplied(expected, actual);
    }

    void withSortingApplied(String expected, String actual) {
        var expectedSorted = Json.write(withDeepSortedArrays(Json.node(expected)));
        var actualSorted = Json.write(withDeepSortedArrays(Json.node(actual)));

        var pattern = new EqualToJsonPattern(expectedSorted, true, true);
        var result = pattern.match(actualSorted);
        var exact = result.isExactMatch();
        assertTrue(exact);
    }

    JsonNode withDeepSortedArrays(JsonNode src) {
        if (src instanceof ArrayNode) {
            var array = (ArrayNode) src;
            var newArray = array.arrayNode();

            StreamSupport.stream(array.spliterator(), false)
                    .map(this::withDeepSortedArrays)
                    .sorted(comparing(Json::write))
                    .forEachOrdered(newArray::add);

            return newArray;
        }

        if (src instanceof ObjectNode) {
            var object = (ObjectNode) src;

            object.fields()
                    .forEachRemaining(entry -> {
                        var newValue = withDeepSortedArrays(entry.getValue());
                        object.set(entry.getKey(), newValue);
                    });

            return object;
        }

        return src;
    }
}

The first test is still failing while the others are not.

@tomakehurst tomakehurst added the bug label Dec 3, 2019
@tomakehurst
Copy link
Member

Fixed by migration to JsonUnit in 7ead912

@YogeshKumbhare
Copy link

YogeshKumbhare commented Sep 24, 2020 •
edited

Hi tomakehurst ,

I need help, in my wiremock stubbing put request I am not passing some extra elements, but I am getting some extra element in response . and hence its shows Verification Exception.
can you please tell me is there any way to ignore those field ...

com.github.tomakehurst.wiremock.client.VerificationException: 3 requests were unmatched by any stub mapping. Requests are:
{
"url" : "v1",
"absoluteUrl" : "http://localhost:9999/v1",
"method" : "PUT",
"clientIp" : "127.0.0.1",
"headers" : {
"Authorization" : "Intuit_IAM_Authentication",
"docid" : "62a5b2ff-260d-489d-a8fb-9e5f3b251yo7",
"User-Agent" : "Vert.x-WebClient/3.9.0",
"Host" : "localhost:9999",
"tid" : "f5f3332e-9848-4783-829c-da8f9e4d141e"
},
"cookies" : { },
"browserProxyRequest" : false,
"loggedDate" : 1600954680338,
"bodyAsBase64" : "",
"body" : "",
"scheme" : "http",
"host" : "localhost",
"port" : 9999,
"loggedDateString" : "2020-09-24T13:38:00Z",
"queryParams" : { }
}

these are the extra fields want to ignore ...
"cookies" : { },
"browserProxyRequest" : false,
"loggedDate" : 1600954680338,
"bodyAsBase64" : "",
"body" : "",
"scheme" : "http",
"host" : "localhost",
"port" : 9999,
"loggedDateString" : "2020-09-24T13:38:00Z",
"queryParams" : { }

Could you please tell me is there any possibility to handle that issue .

@MrAolen MrAolen mentioned this issue Oct 6, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tomakehurst @pjungermann @YogeshKumbhare