Ability to test the body schema

[mboudreau]

In our case, we need to be able to make sure that the json schema returned back by an API is current (right properties, right data types), but the data itself is not really important.

[bethesque]

I believe this will solve your problem if you are using ruby (wrapper) on both sides (consumer and provider):

https://github.com/DiUS/pact-consumer-js-dsl/wiki/Flexible-matching

If you are not, then you'll have to wait for v2 matching which is a WIP.

[mboudreau]

@bethesque I've finally used the somethingLike function within our project and I found a major flaw; There's no concept of 'optional' variables, or extra keys. For instance, I was testing for an object that has, at the very least, the property 'id' as a string and 'schema' as an array. In that same object, I also had a key of 'label'. Because of this label, the request didn't work since the parsing was failing (wasn't matching). I see json schema matching as a 'minimum requirement', not a hard requirement.

We definitely also need for optional params in there. For instance, our schema has a parent child relationship which can be 3 deep at most, but is optional. With the somethingLike matcher, all children would have to be there or an error would occur. This is problematic for us since we now have to create a mock match for every single 'variability' of our data.

[bethesque]

Hi Michael,

This design is quite deliberate, as we follow Postel's law of saying "we're going to be very strict with what we send out, but liberal with what we accept". We don't want our real code sending out data that our test code does not know about, as this would create a difference between the test scenarios and real life scenarios that would allow some code paths to not be properly tested, and might hide the existence of bugs.

Having said that, I know that sometimes it's a real pain! If you are finding this too painful and your use case is low risk enough that you're willing to risk differences between test/prod requests, there is an option you can add to the request setup to let it ignore extra keys. It's not currently supported by the js library, but you could add it quite easily. The following hash would need to be included to the request setup.

{
  "request" : {
    "options" : {
      "allow_unexpected_keys_in_body" : true
    }
  }
}

[mboudreau]

Aren't you contradicting yourself if the design to be 'liberal with was we accept' but creating an error when I send a key that wasn't specified in pact?

Our case is this: we have form data that we want to save. The 'value' of the fields within the form can either change from string, to number, to array, to object or not have one at all. How can we specify this using somethingLike?

I really believe that somethingLike should be a schema DSL of sorts, a way to describe how it's suppose to look, what it's suppose to accept, if parameters are optional or required (and throw an error if an unspecified one is sent over).

[bethesque]

Pact is strict with what the Consumer sends out - that's why we don't, by default, allow extra keys in the requests bodies. It is liberal with what the Consumer accepts from the Provider - that's why extra keys are allowed.

Pact is "specification by example", not "specification by schema", so you would just set up a different interaction for each variation you expect at that key. Your use case is quite unusual - it is not common to have a key that will accept multiple types of values, and I'd suggest that it actually indicates there might be some improvements that could be made to the design.

[mboudreau]

Not that unusual. I'd say that at the very least, the optional data points are something that I've seen happen many times. The different values is something a bit more rare, but definitely not unheard of.