Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request verification with JSON Schema doesn't work as expected #871

Closed
cjosepha opened this issue Sep 22, 2020 · 5 comments
Closed

Request verification with JSON Schema doesn't work as expected #871

cjosepha opened this issue Sep 22, 2020 · 5 comments
Assignees

Comments

@cjosepha
Copy link

cjosepha commented Sep 22, 2020

Describe the issue
When using JSON Schema to verify requests, mockserver finds a match (i.e. call the success callback of the verify() function) while no request matching the JSON schema has been sent to it.

What you are trying to do
I'm able to create and use expectations without problem, but the verification using JSON Schema seems to provide false positives.

MockServer version
5.11.1

To Reproduce
Steps to reproduce the issue:

In my case the mockserver is controlled by a Node.js app which use mockserver-node and mockserver-client-node.

1. Start mock server on port 1080

mockServer.start_mockserver({
    serverPort: 1080,
    trace: true
  });

2. Create expectation on path /api/logs:

mockServerClient("localhost", 1080).mockAnyResponse({
    id: "logs",
    httpRequest: {
      method: "POST",
      path: "/api/logs"
    },
    httpResponse: {
      statusCode: 200,
      body: JSON.stringify({
        success: true
      })
    }
  });

4. Send request to /api/logs from our mobile app

Extracted from mockserver logs:

------------------------------------
2020-09-22 17:52:52.864 - returning response:

  {
    "statusCode" : 200,
    "body" : "{\"success\":true}"
  }

 for request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "headers" : {
      "apicontext" : [ "dev" ],
      "Accept" : [ "application/json" ],
      "Content-Type" : [ "application/x-www-form-urlencoded" ],
      "User-Agent" : [ "Dalvik/2.1.0 (Linux; U; Android 10; ART-L29 Build/HUAWEIART-L29)" ],
      "Host" : [ "192.168.0.28:1080" ],
      "Connection" : [ "Keep-Alive" ],
      "Accept-Encoding" : [ "gzip" ],
      "Content-Length" : [ "351" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "body" : "{\"deviceId\":\"8c76ead9a9223dd8\",\"sdkPlatform\":\"Android\",\"sdkVersion\":\"6.6.1-SNAPSHOT\",\"requestDate\":\"2020-09-22T17:52:53.149+0200\",\"packageName\":\"com.followapps.android.sdkdemo.alpha.debug\",\"sessions\":[{\"sessionId\":\"SESSION_1\",\"logs\":[{\"logDate\":\"2020-09-22T17:52:53.128+0200\",\"logType\":1,\"logDetails\":\"World\",\"logName\":\"Hello\"}]}]}"
  }

 for action:

  {
    "statusCode" : 200,
    "body" : "{\"success\":true}"
  }

 from expectation:

  logs

------------------------------------

3. Verify request with JSON Schema:

This schema has been tested OK on https://www.jsonschemavalidator.net/, meaning it rejects the request sent on step 4:

mockServerClient("localhost", 1080).verify({
    "method": "POST",
    "path": "/api/logs",
    "body": {
      "type": "JSON_SCHEMA",
      "jsonSchema": {
        "type": "object",
        "properties": {
          "sessions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "sessionId": {
                  "type": "string"
                },
                "logs": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "logType": {
                        "type": "integer",
                        "enum": [1]
                      },
                      "logName": {
                        "type": "string",
                        "enum": ["Error"]
                      },
                      "logDetails": {
                        "type": "string",
                        "enum": ["Verify"]
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }, 1, 1);

-> The success callback is called instead of the error one. Note that the mockserver logs below mention the following for this verify request : request: ... found exactly once.

Complete Mockserver logs

logs_verify.txt

@jamesdbloom
Copy link
Collaborator

I can't reproduce this error which either 5.11.1 or the latest SNAPSHOT version as I get the following error:

2020-10-30 08:15:01 5.11.2-SNAPSHOT FINEST json schema match failed expected:

  {
    "type" : "object",
    "properties" : {
      "sessions" : {
        "type" : "array",
        "items" : {
          "type" : "object",
          "properties" : {
            "sessionId" : {
              "type" : "string"
            },
            "logs" : {
              "type" : "array",
              "items" : {
                "type" : "object",
                "properties" : {
                  "logType" : {
                    "type" : "integer",
                    "enum" : [ 1 ]
                  },
                  "logName" : {
                    "type" : "string",
                    "enum" : [ "Error" ]
                  },
                  "logDetails" : {
                    "type" : "string",
                    "enum" : [ "Verify" ]
                  }
                }
              }
            }
          }
        }
      }
    }
  }

 found:

  {
      "deviceId": "8c76ead9a9223dd8", 
      "sdkPlatform": "Android", 
      "sdkVersion": "6.6.1-SNAPSHOT", 
      "requestDate": "2020-09-22T17:52:53.149+0200", 
      "packageName": "com.followapps.android.sdkdemo.alpha.debug", 
      "sessions": [
          {
              "sessionId": "SESSION_1", 
              "logs": [
                  {
                      "logDate": "2020-09-22T17:52:53.128+0200", 
                      "logType": 1, 
                      "logDetails": "World", 
                      "logName": "Hello"
                  }
              ]
          }
      ]
  }

 failed because:

  2 errors:
   - field: "/sessions/0/logs/0/logDetails" for schema: "/properties/sessions/items/properties/logs/items/properties/logDetails" has error: "instance value ("World") not found in enum (possible values: ["Verify"])"
   - field: "/sessions/0/logs/0/logName" for schema: "/properties/sessions/items/properties/logs/items/properties/logName" has error: "instance value ("Hello") not found in enum (possible values: ["Error"])"

I am using the following code to start MockServer and create the expectation:

require('mockserver-node')
    .start_mockserver({
        serverPort: 1081,
        verbose: true,
        trace: true
    })
    .then(
        function () {
            require('mockserver-client')
                .mockServerClient("localhost", 1080)
                .mockAnyResponse({
                    id: "logs",
                    httpRequest: {
                        method: "POST",
                        path: "/api/logs"
                    },
                    httpResponse: {
                        statusCode: 200,
                        body: JSON.stringify({
                            success: true
                        })
                    }
                })
                .then(
                    function () {
                        console.log("created expectation");
                    },
                    function (error) {
                        console.log(error);
                    }
                );
        },
        function (error) {
            console.log(JSON.stringify(error, null, "  "));
        }
    );

I am using this curl request to simulate the request:

curl -i -X POST \
   -H "Content-Type:application/json" \
   -d \
'{
    "deviceId": "8c76ead9a9223dd8", 
    "sdkPlatform": "Android", 
    "sdkVersion": "6.6.1-SNAPSHOT", 
    "requestDate": "2020-09-22T17:52:53.149+0200", 
    "packageName": "com.followapps.android.sdkdemo.alpha.debug", 
    "sessions": [
        {
            "sessionId": "SESSION_1", 
            "logs": [
                {
                    "logDate": "2020-09-22T17:52:53.128+0200", 
                    "logType": 1, 
                    "logDetails": "World", 
                    "logName": "Hello"
                }
            ]
        }
    ]
}' \
 'http://localhost:1080/api/logs'

I am using the following code to verify the expectation:

require('mockserver-client')
    .mockServerClient("localhost", 1080)
    .verify({
        "method": "POST",
        "path": "/api/logs",
        "body": {
            "type": "JSON_SCHEMA",
            "jsonSchema": {
                "type": "object",
                "properties": {
                    "sessions": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "sessionId": {
                                    "type": "string"
                                },
                                "logs": {
                                    "type": "array",
                                    "items": {
                                        "type": "object",
                                        "properties": {
                                            "logType": {
                                                "type": "integer",
                                                "enum": [1]
                                            },
                                            "logName": {
                                                "type": "string",
                                                "enum": ["Error"]
                                            },
                                            "logDetails": {
                                                "type": "string",
                                                "enum": ["Verify"]
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }, 1, 1)
    .then(
        function () {
            console.log("verified expectation");
        },
        function (error) {
            console.log(error);
        }
    );

The following is the full log: issue_871.log

Can you please retry using the latest SNAPSHOT version as follows:

require('mockserver-node')
    .start_mockserver({
        serverPort: 1081,
        verbose: true,
        trace: true,
        mockServerVersion: "5.11.2-SNAPSHOT"
    })
    .then(
        function () {
            require('mockserver-client')
                .mockServerClient("localhost", 1080)
                .mockAnyResponse({
                    id: "logs",
                    httpRequest: {
                        method: "POST",
                        path: "/api/logs"
                    },
                    httpResponse: {
                        statusCode: 200,
                        body: JSON.stringify({
                            success: true
                        })
                    }
                })
                .then(
                    function () {
                        console.log("created expectation");
                    },
                    function (error) {
                        console.log(error);
                    }
                );
        },
        function (error) {
            console.log(JSON.stringify(error, null, "  "));
        }
    );

@cjosepha
Copy link
Author

cjosepha commented Nov 2, 2020

@jamesdbloom the use case is working OK when the request body is sent as a JSON (Content-Type:application/json), so we changed our Android code to do so and all is OK now.

It is not working when the request body is sent as a FORM (Content-Type:application/x-www-form-urlencoded), as you can see in my initial post:

4. Send request to /api/logs from our mobile app

Extracted from mockserver logs:

------------------------------------
2020-09-22 17:52:52.864 - returning response:

  {
    "statusCode" : 200,
    "body" : "{\"success\":true}"
  }

 for request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "headers" : {
      "apicontext" : [ "dev" ],
      "Accept" : [ "application/json" ],
      "Content-Type" : [ "application/x-www-form-urlencoded" ],
      "User-Agent" : [ "Dalvik/2.1.0 (Linux; U; Android 10; ART-L29 Build/HUAWEIART-L29)" ],
      "Host" : [ "192.168.0.28:1080" ],
      "Connection" : [ "Keep-Alive" ],
      "Accept-Encoding" : [ "gzip" ],
      "Content-Length" : [ "351" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "body" : "{\"deviceId\":\"8c76ead9a9223dd8\",\"sdkPlatform\":\"Android\",\"sdkVersion\":\"6.6.1-SNAPSHOT\",\"requestDate\":\"2020-09-22T17:52:53.149+0200\",\"packageName\":\"com.followapps.android.sdkdemo.alpha.debug\",\"sessions\":[{\"sessionId\":\"SESSION_1\",\"logs\":[{\"logDate\":\"2020-09-22T17:52:53.128+0200\",\"logType\":1,\"logDetails\":\"World\",\"logName\":\"Hello\"}]}]}"
  }

 for action:

  {
    "statusCode" : 200,
    "body" : "{\"success\":true}"
  }

 from expectation:

  logs

------------------------------------

@cjosepha
Copy link
Author

@jamesdbloom did you try to reproduce with the information I provided?

@cjosepha
Copy link
Author

@jamesdbloom ?

@jamesdbloom
Copy link
Collaborator

@cjosepha I have just tried to reproduce the issue you are having but I don't seem to be able to do that, please see the logs below demonstrating this. I'm going to close this ticket but if you still have the issue with the latest SNAPSHOT version (which will be released very shortly) then please provide some code so I can reproduce your issue. The JSON schema validation library has been upgraded in the latest code to improve performance and provide support for draft 4, 6, 7 & 2019-09 so this may have fixed any issue you had.

2022-01-28 19:58:50 5.11.3-SNAPSHOT INFO 1080 creating expectation:

  {
    "httpRequest" : {
      "method" : "POST",
      "path" : "/api/logs"
    },
    "httpResponse" : {
      "statusCode" : 200,
      "body" : {
        "type" : "JSON",
        "json" : {
          "success" : true
        }
      }
    },
    "id" : "33629085-880b-495e-a23d-1701e9056800",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }

 with id:

  33629085-880b-495e-a23d-1701e9056800
 
2022-01-28 19:58:56 5.11.3-SNAPSHOT INFO 1080 received request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "headers" : {
      "sec-ch-ua-mobile" : [ "?0" ],
      "sec-ch-ua" : [ "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\"" ],
      "content-length" : [ "563" ],
      "User-Agent" : [ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36" ],
      "Sec-Fetch-Site" : [ "none" ],
      "Sec-Fetch-Mode" : [ "cors" ],
      "Sec-Fetch-Dest" : [ "empty" ],
      "Host" : [ "localhost:1080" ],
      "DNT" : [ "1" ],
      "Content-Type" : [ "application/x-www-form-urlencoded" ],
      "Connection" : [ "keep-alive" ],
      "Accept-Language" : [ "en-GB,en;q=0.9,fa;q=0.8" ],
      "Accept-Encoding" : [ "gzip, deflate, br" ],
      "Accept" : [ "*/*" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "body" : "{\n    \"deviceId\": \"8c76ead9a9223dd8\", \n    \"sdkPlatform\": \"Android\", \n    \"sdkVersion\": \"6.6.1-SNAPSHOT\", \n    \"requestDate\": \"2020-09-22T17:52:53.149+0200\", \n    \"packageName\": \"com.followapps.android.sdkdemo.alpha.debug\", \n    \"sessions\": [\n        {\n            \"sessionId\": \"SESSION_1\", \n            \"logs\": [\n                {\n                    \"logDate\": \"2020-09-22T17:52:53.128+0200\", \n                    \"logType\": 1, \n                    \"logDetails\": \"World\", \n                    \"logName\": \"Hello\"\n                }\n            ]\n        }\n    ]\n}"
  }
 
2022-01-28 19:58:56 5.11.3-SNAPSHOT INFO 1080 request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "headers" : {
      "sec-ch-ua-mobile" : [ "?0" ],
      "sec-ch-ua" : [ "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\"" ],
      "content-length" : [ "563" ],
      "User-Agent" : [ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36" ],
      "Sec-Fetch-Site" : [ "none" ],
      "Sec-Fetch-Mode" : [ "cors" ],
      "Sec-Fetch-Dest" : [ "empty" ],
      "Host" : [ "localhost:1080" ],
      "DNT" : [ "1" ],
      "Content-Type" : [ "application/x-www-form-urlencoded" ],
      "Connection" : [ "keep-alive" ],
      "Accept-Language" : [ "en-GB,en;q=0.9,fa;q=0.8" ],
      "Accept-Encoding" : [ "gzip, deflate, br" ],
      "Accept" : [ "*/*" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "body" : "{\n    \"deviceId\": \"8c76ead9a9223dd8\", \n    \"sdkPlatform\": \"Android\", \n    \"sdkVersion\": \"6.6.1-SNAPSHOT\", \n    \"requestDate\": \"2020-09-22T17:52:53.149+0200\", \n    \"packageName\": \"com.followapps.android.sdkdemo.alpha.debug\", \n    \"sessions\": [\n        {\n            \"sessionId\": \"SESSION_1\", \n            \"logs\": [\n                {\n                    \"logDate\": \"2020-09-22T17:52:53.128+0200\", \n                    \"logType\": 1, \n                    \"logDetails\": \"World\", \n                    \"logName\": \"Hello\"\n                }\n            ]\n        }\n    ]\n}"
  }

 matched expectation:

  {
    "httpRequest" : {
      "method" : "POST",
      "path" : "/api/logs"
    },
    "httpResponse" : {
      "statusCode" : 200,
      "body" : {
        "type" : "JSON",
        "json" : {
          "success" : true
        }
      }
    },
    "id" : "33629085-880b-495e-a23d-1701e9056800",
    "priority" : 0,
    "timeToLive" : {
      "unlimited" : true
    },
    "times" : {
      "unlimited" : true
    }
  }
 
2022-01-28 19:58:56 5.11.3-SNAPSHOT INFO 1080 returning response:

  {
    "statusCode" : 200,
    "body" : {
      "success" : true
    }
  }

 for request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "headers" : {
      "sec-ch-ua-mobile" : [ "?0" ],
      "sec-ch-ua" : [ "\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\"" ],
      "content-length" : [ "563" ],
      "User-Agent" : [ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36" ],
      "Sec-Fetch-Site" : [ "none" ],
      "Sec-Fetch-Mode" : [ "cors" ],
      "Sec-Fetch-Dest" : [ "empty" ],
      "Host" : [ "localhost:1080" ],
      "DNT" : [ "1" ],
      "Content-Type" : [ "application/x-www-form-urlencoded" ],
      "Connection" : [ "keep-alive" ],
      "Accept-Language" : [ "en-GB,en;q=0.9,fa;q=0.8" ],
      "Accept-Encoding" : [ "gzip, deflate, br" ],
      "Accept" : [ "*/*" ]
    },
    "keepAlive" : true,
    "secure" : false,
    "body" : "{\n    \"deviceId\": \"8c76ead9a9223dd8\", \n    \"sdkPlatform\": \"Android\", \n    \"sdkVersion\": \"6.6.1-SNAPSHOT\", \n    \"requestDate\": \"2020-09-22T17:52:53.149+0200\", \n    \"packageName\": \"com.followapps.android.sdkdemo.alpha.debug\", \n    \"sessions\": [\n        {\n            \"sessionId\": \"SESSION_1\", \n            \"logs\": [\n                {\n                    \"logDate\": \"2020-09-22T17:52:53.128+0200\", \n                    \"logType\": 1, \n                    \"logDetails\": \"World\", \n                    \"logName\": \"Hello\"\n                }\n            ]\n        }\n    ]\n}"
  }

 for action:

  {
    "statusCode" : 200,
    "body" : {
      "success" : true
    }
  }

 from expectation:

  33629085-880b-495e-a23d-1701e9056800
 
2022-01-28 19:59:02 5.11.3-SNAPSHOT INFO 1080 verifying requests that match:

  {
    "httpRequest" : {
      "method" : "POST",
      "path" : "/api/logs",
      "body" : {
        "type" : "JSON_SCHEMA",
        "jsonSchema" : {
          "type" : "object",
          "properties" : {
            "sessions" : {
              "type" : "array",
              "items" : {
                "type" : "object",
                "properties" : {
                  "sessionId" : {
                    "type" : "string"
                  },
                  "logs" : {
                    "type" : "array",
                    "items" : {
                      "type" : "object",
                      "properties" : {
                        "logType" : {
                          "type" : "integer",
                          "enum" : [ 1 ]
                        },
                        "logName" : {
                          "type" : "string",
                          "enum" : [ "Error" ]
                        },
                        "logDetails" : {
                          "type" : "string",
                          "enum" : [ "Verify" ]
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    "times" : {
      "atLeast" : 1,
      "atMost" : 1
    }
  }
 
2022-01-28 19:59:02 5.11.3-SNAPSHOT INFO request:

  {
    "method" : "POST",
    "path" : "/api/logs",
    "body" : {
      "type" : "JSON_SCHEMA",
      "jsonSchema" : {
        "type" : "object",
        "properties" : {
          "sessions" : {
            "type" : "array",
            "items" : {
              "type" : "object",
              "properties" : {
                "sessionId" : {
                  "type" : "string"
                },
                "logs" : {
                  "type" : "array",
                  "items" : {
                    "type" : "object",
                    "properties" : {
                      "logType" : {
                        "type" : "integer",
                        "enum" : [ 1 ]
                      },
                      "logName" : {
                        "type" : "string",
                        "enum" : [ "Error" ]
                      },
                      "logDetails" : {
                        "type" : "string",
                        "enum" : [ "Verify" ]
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }

 found exactly once 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants