Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
What about Swagger Specification aka OpenAPI? #28
Hi @elgalu, this is a great discussion point. For starters, Pact pre-dates OpenAPI (not Swagger), so I like to think we're not reinventing the wheel! But as I understand it there are some key differences and if anything, they are complimentary technologies and could be used together.
The differences can be summarised below:
Swagger / OpenAPI specification aims to standardise the description and structure of an API. It can tell you what APIs are available and what fields/structure it expects and can generate documentation/UI to interact with one. What it is not, is a testing framework.
Pact on the other hand, is essentially a unit testing framework using specification by example. It just so happens that to be able to run those tests on the API consumer and provider side, it needs to generate an intermediate format to be able to communicate that structure - this is the specification. Now we need a lot more information that just the structure (matching rules, provider states and so on) that OpenAPI documents in its spec.
In fact, the authors of the OpenAPI specification predicted such use cases by announcing:
Potentially, for example, we could use vendor extensions to document this extra metadata that is captured in our spec. This is one way the two projects could come together.
So for me, I would like to explore further how we could use existing Swagger/OpenAPI definitions to streamline Pact usage - for example to generate stub tests or as part of our specification. Suggestions most welcome :)
Does this help?
Awesome answer @mefellows, my question should have been more like
That sounds great! maybe we can use this thread to communicate our findings.
I have done quite a bit with both Swagger (or OpenAPI) and RAML. Although they seem to serve the same purpose, their use cases are slightly different. Swaggers main use is as a design tool, allowing the APIs to be specified and then the server and client parts to be generated off the specification. At my current client, all APIs have to have their solution design signed off before they can be built, and swagger serves this purpose well.
Pact supports a more evolutionary development process, and I have used it in many Agile projects where the APIs have evolved from the requirements of the user stories. It was always designed from the start to have the contracts generated from actual running code, so the contracts are always current. Swagger and RAML work the other way round, the code is generated off the specification and there is no automatic way to keep them in sync.
The other problem is that Pact is "Specification by Example". It is a record of example request response pairs that determine the specification. Swagger, OpenAPI and RAML are all based on paths, and normally have a single request specification and then multiple response specifications for each different kind of response. I have had discussions on how to generate the specification from the pact files, but it turned out to be difficult because with pact you can have multiple examples for the same API path, so which one do you use for the specification?
To contribute to the conversation - our team is experimenting with validating the Consumer-generated Pacts against a Provider-published Swagger/RAML specification within the Consumer tests.
We've found it can help catch a class of invalid expectations very early (bad paths, incorrect parameter formats, request/response schema validation etc) and shorten the feedback loop (you don't have to wait for the Provider tests to run to realise you've made a mistake setting up your expectations).
It does come at the expense of requiring Consumers to set up expectations that include all required fields in a request/response schema rather than just the ones they care about (breaking Postel's law), but at the moment that hasn't been a big problem for us.
We're also looking at the possibility of generating REST clients from the Swagger/RAML spec that include a 'mock' mode powered by Pact files (havent got very far with that spike though...)
referenced this issue
Jun 16, 2016
Had another sort of similar-ish project to add to the conversation.
The short version is that you write tests describing your endpoints, and as a side-effect of running those tests new documentation is created.
@uglyog - Ive (finally) open sourced our swagger-request-validator library that includes a Pact module for validating Pact interactions against a Swagger/OpenAPI spec.
There's some examples in there on usage etc. The library is still very much under development - if you find any issues raise a ticket on the project and I'll try to look at it.
We'll be sharing some details during Atlassian Summit 2016 so come join us if you can!
Wow, that's awesome! Maybe we'll see if DiUS can fly us to California to
On a side note, would you be interested in talking (privately) to us about
On Fri, Sep 2, 2016 at 3:50 PM, Ben email@example.com wrote: