New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What about Swagger Specification aka OpenAPI? #28

Open
elgalu opened this Issue May 19, 2016 · 14 comments

Comments

Projects
None yet
7 participants
@elgalu
Copy link

elgalu commented May 19, 2016

Maybe we are kind of re-inventing the wheel here? see OpenAPI

@mefellows mefellows added the question label May 19, 2016

@mefellows

This comment has been minimized.

Copy link
Member

mefellows commented May 19, 2016

Hi @elgalu, this is a great discussion point. For starters, Pact pre-dates OpenAPI (not Swagger), so I like to think we're not reinventing the wheel! But as I understand it there are some key differences and if anything, they are complimentary technologies and could be used together.

The differences can be summarised below:

Swagger / OpenAPI specification aims to standardise the description and structure of an API. It can tell you what APIs are available and what fields/structure it expects and can generate documentation/UI to interact with one. What it is not, is a testing framework.

Pact on the other hand, is essentially a unit testing framework using specification by example. It just so happens that to be able to run those tests on the API consumer and provider side, it needs to generate an intermediate format to be able to communicate that structure - this is the specification. Now we need a lot more information that just the structure (matching rules, provider states and so on) that OpenAPI documents in its spec.

In fact, the authors of the OpenAPI specification predicted such use cases by announcing:

Additional utilities can also take advantage of the resulting files, such as testing tools.

Potentially, for example, we could use vendor extensions to document this extra metadata that is captured in our spec. This is one way the two projects could come together.

So for me, I would like to explore further how we could use existing Swagger/OpenAPI definitions to streamline Pact usage - for example to generate stub tests or as part of our specification. Suggestions most welcome :)

Does this help?

@elgalu

This comment has been minimized.

Copy link

elgalu commented May 20, 2016

Awesome answer @mefellows, my question should have been more like
How Swagger and Pacts could complement each other

To add some background, at Zalando we are pushing Swagger quite a lot and we have some related open source projects: connexion , play-swagger , swagger-mock , codegen-tooling

explore further how we could use existing Swagger/OpenAPI definitions to streamline Pact usage

That sounds great! maybe we can use this thread to communicate our findings.

@mefellows

This comment has been minimized.

Copy link
Member

mefellows commented May 20, 2016

Glad to hear we're on the same page. I'm OK with the conversation staying here, but would like to here thoughts of @uglyog and others. It might be best to move to a Google groups discussion.

@uglyog

This comment has been minimized.

Copy link
Member

uglyog commented May 23, 2016

I have done quite a bit with both Swagger (or OpenAPI) and RAML. Although they seem to serve the same purpose, their use cases are slightly different. Swaggers main use is as a design tool, allowing the APIs to be specified and then the server and client parts to be generated off the specification. At my current client, all APIs have to have their solution design signed off before they can be built, and swagger serves this purpose well.

Pact supports a more evolutionary development process, and I have used it in many Agile projects where the APIs have evolved from the requirements of the user stories. It was always designed from the start to have the contracts generated from actual running code, so the contracts are always current. Swagger and RAML work the other way round, the code is generated off the specification and there is no automatic way to keep them in sync.

The other problem is that Pact is "Specification by Example". It is a record of example request response pairs that determine the specification. Swagger, OpenAPI and RAML are all based on paths, and normally have a single request specification and then multiple response specifications for each different kind of response. I have had discussions on how to generate the specification from the pact files, but it turned out to be difficult because with pact you can have multiple examples for the same API path, so which one do you use for the specification?

@jfnavin

This comment has been minimized.

Copy link

jfnavin commented Jun 1, 2016

To contribute to the conversation - our team is experimenting with validating the Consumer-generated Pacts against a Provider-published Swagger/RAML specification within the Consumer tests.

We've found it can help catch a class of invalid expectations very early (bad paths, incorrect parameter formats, request/response schema validation etc) and shorten the feedback loop (you don't have to wait for the Provider tests to run to realise you've made a mistake setting up your expectations).

It does come at the expense of requiring Consumers to set up expectations that include all required fields in a request/response schema rather than just the ones they care about (breaking Postel's law), but at the moment that hasn't been a big problem for us.

We're also looking at the possibility of generating REST clients from the Swagger/RAML spec that include a 'mock' mode powered by Pact files (havent got very far with that spike though...)

@uglyog

This comment has been minimized.

Copy link
Member

uglyog commented Jun 4, 2016

@jfnavin I'll be really interested to see what you can come up with.

@cah-andrewfitzgerald

This comment has been minimized.

Copy link

cah-andrewfitzgerald commented Jun 20, 2016

Had another sort of similar-ish project to add to the conversation.
Spring REST Docs feels like it's somewhere in between swagger and pact consumer tests.

The short version is that you write tests describing your endpoints, and as a side-effect of running those tests new documentation is created.

@jfnavin

This comment has been minimized.

Copy link

jfnavin commented Aug 3, 2016

@uglyog - Ive (finally) open sourced our swagger-request-validator library that includes a Pact module for validating Pact interactions against a Swagger/OpenAPI spec.

https://bitbucket.org/atlassian/swagger-request-validator

There's some examples in there on usage etc. The library is still very much under development - if you find any issues raise a ticket on the project and I'll try to look at it.

@uglyog

This comment has been minimized.

Copy link
Member

uglyog commented Aug 3, 2016

Yay! 🤘 I'll definetly give it a look.

@mefellows

This comment has been minimized.

Copy link
Member

mefellows commented Aug 27, 2016

@jfnavin this is interesting! Also, are Atlassian using Pact? If so, we'd love to hear about how/what/etc.!

Can discuss here or better yet: https://gitter.im/realestate-com-au/pact

@BenSayers

This comment has been minimized.

Copy link

BenSayers commented Sep 2, 2016

We'll be sharing some details during Atlassian Summit 2016 so come join us if you can!
https://www.atlassian.com/summit/sessions?tab=build-%26-deploy&sessionid=54094

@mefellows

This comment has been minimized.

Copy link
Member

mefellows commented Sep 2, 2016

Wow, that's awesome! Maybe we'll see if DiUS can fly us to California to
see it in person ;)

On a side note, would you be interested in talking (privately) to us about
your travels with Pact at Atlassian? We'd love to hear about what you're
doing and what we can do to improve the product.

On Fri, Sep 2, 2016 at 3:50 PM, Ben notifications@github.com wrote:

We'll be sharing some details during Atlassian Summit 2016 so come join us
if you can!
https://www.atlassian.com/summit/sessions?tab=build-%26-
deploy&sessionid=54094


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#28 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADSjB96ay9qBbb2bmdQhDhaKuOEiVQcks5ql7kWgaJpZM4IiRod
.

Matt Fellows

@fitzoh

This comment has been minimized.

Copy link

fitzoh commented Mar 27, 2017

@BenSayers is there a recording or some slides out there?

@BenSayers

This comment has been minimized.

Copy link

BenSayers commented Mar 28, 2017

@fitzoh here is the recording - https://www.youtube.com/watch?v=-6x6XBDf9sQ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment