Deprecating swagger-tools

[whitlockjc]

For over a year now, the need to rewrite swagger-tools has been apparent. Not only that but swagger-tools doesn't exactly adhere to the NPM best practices where each module do one specific thing since swagger-tools is an API, a CLI and Middleware. Lastly, swagger-tools started with Swagger 1.2 support and was adapted to support Swagger 2.0. This last piece is what has caused a lot of code cruft, architectural hardships, etc. It is also this last piece that is making supporting swagger-tools harder.

That being said, the plan is to deprecate swagger-tools as-is and replace each of the individual components of swagger-tools with specific NPM modules:

• API: https://github.com/apigee-127/sway _(This is a _real* API that makes Swagger integrations simpler. Sway is already used by swagger-editor and swagger-node, although they use very old versions, and 1.0.0 has been ready for weeks and is just waiting on sway-connect to be completed to flush out any remaining bugs.)*
• CLI: https://github.com/whitlockjc/swagger-lint (Breaking the CLI out allows us to use Sway's custom validators to do ESLint style rules.)
• Middleware: https://github.com/apigee-127/sway-connect (The plan is to have middleware that could be a drop-in replacement for swagger-tools' middleware but architected from the start as it works now, but with the swagger-tools bugs/features fixed/implemented)

I realize that this might come as a shock to some but I've been talking about this for quite some time. Issues like #327, #227 and others have mention of this plan in it as well. Without a rewrite, swagger-tools just can't fix some of these bugs and rewriting software that's already been replaced makes no sense. I welcome your feedback.

[whitlockjc]

When this issue is closed, that means all of the replacement projects are in place and ready to be used. Until then, this is just a place for discussion on the topic.

[firefoxNX]

Thanks @whitlockjc for posting this. The deprecation did come as bit of of a surprise. We have had no major issues with swagger-tools. Only few times we had issues with new versions not being compatible with Trireme. Did not know about sway-connect at all. Looking forward to it!

[whitlockjc]

I'm not saying I won't support swagger-tools for those using it. :) It's just that swagger-tools started out as one thing and became three. So while I use the term "deprecated", I'm really breaking the project up and rewriting where necessary since that would be a good time.

As for swagger-tools not having any major issues, there are a number that bug me and I hate that they aren't fixed already, especially since they are fixed in Sway. But to fix them would require breaking the existing APIs which means rewriting things, etc. I just figured breaking the project up into separate repos was a good time to revisit this.

I hope this all makes sense, I am by no means walking away from swagger-tools and I hope that those projects that replace it are better than what we have now.

[steve-gray]

Sounds reasonable.

Maybe a swagger-tools org needs to rise up - I've got some ideas around codegen for this. For example where I am, we are generating from the YAML a set of server-side entities, and also controller classes that map to methods that are called with just the parameter values, like a 'real' function (rather than having to deep-dive into req.swagger-parms) - and then automatically mapping back to the appropriate HTTP response code based on the type of JSON object being returned.

Theres probably other better ideas.

-Steve

[whitlockjc]

I may create a sway organization, that's a good idea.

[steve-gray]

If there's room for a gulp-task in there, let me know. I've written some codegen stuff to wrap around the controller classes. It's effectively generating definitions and controllers that are supposed to plug into swaggerRouter, and then call a much simpler-looking implementation class.

It's published up at:

• https://www.npmjs.com/package/gulp-swagger-codegen

ES6 (I know some folks will go 'ick!' but cross-testing on <= 4.0 didn't feel overly worthwhile immediately).

[whitlockjc]

If you use the swagger-tools middleware, feel free to help guide its replacement: https://github.com/whitlockjc/sway-connect/issues

[hsablonniere]

Hey @whitlockjc!

From what I've understood in this thread you're going to split the project in multiple modules and correct some of the APIs. I need a project like swagger-tools (or its replacement) so I looked around and found this : https://github.com/kogosoftwarellc/express-openapi.

I'm wondering where the split is going and how it will differ from express-openapi? Do you have any details on the roadmap and usage examples?

Thx...

[whitlockjc]

Hello @hsablonniere!

swagger-tools is nearly 2 years old and has been heavily used/tested. The official swagger-node project from the original Swagger owners uses swagger-tools for its middleware and so do many others. I tell you this because swagger-tools already got an established track record and is in use by many people. I also do not see any problem with using swagger-tool as-is as the new sway-connect replacement will have migration documentation from swagger-tools to sway-connect.

As for express-openapi, I've never heard it until you brought it up. It seems that the project only 1.5 months old. I'm not sure why someone would want to reinvent the wheel by creating another Swagger/OpenAPI driven middleware but competition is always good and to each their own. That being said, I've no opinion on that project as I've not used it.

As for swagger-tools, there are a number of middlewares provided by swagger-tools, all documented here, and I will make sure that the new module containing these middlewares are least replace the functionality swagger-tools provides. As far as how it's going, it's coming along quite nice but it's taking longer than it could because I'm doing my best to solicit community feedback: https://github.com/whitlockjc/sway-connect/issues All of this being said, I hope to make sway-connect, and the rest of sway-* middlewares, as good or better than what swagger-tools provides. I hope to use the 2 years of feedback and learning to make it the best middleware it can possibly be.

Just know that swagger-tools being "deprecated" is really not the case, we're just breaking our project up into specific npm modules and using that time to revisit design decisions and to make things better. Any feature in swagger-tools now will be available in the replacement modules albeit in a better shape.

I hope this helps.

[hsablonniere]

@whitlockjc Thanks very much for the feedbacks and details.

[whitlockjc]

@hsablonniere I'm watching that thread to see if I can get some information as well. I want you to have enough information to use what makes sense for you, even if it's not my project.

[hsablonniere]

@whitlockjc Fair enough, thanks very much.

[amcdnl]

Any updates on this @whitlockjc ?

[whitlockjc]

I apologize @amcdnl but I've been assigned to other things at work and I've not had much time to get the middleware done. I've got this started here and it shouldn't take much work but I've just not had the time recently.

I am working on some improvements and bug fixes for json-refs which will mean a new sway release, with improvements and bug fixes. So if you need an API for loading/validating/... Swagger document, sway is more than ready. If you need the middleware, I'm still not done but I will be working on it soon.

[chadxz]

@whitlockjc If I were looking at starting a fresh project today, would you recommend using swagger-express-mw (as is pushed by the swagger-node cli generator) or continue to use swagger-tools ?

[whitlockjc]

swagger-express-mw uses sway for the loading/validating of the input document(s) but does not use sway for the middleware, it still uses the swagger-tools middleware. That being said, most of the reasons this project is being deprecated is not because of the middleware but really about loading/processing the input document(s). But on the other hand, some of the swagger-tools middleware do use the swagger-tools APIs which are somewhat outdated as part of the deprecation.

I say, if it works use it because I will be fixing the middleware bugs either here or in the new project.

[chadxz]

Thank you for your thoughts on the subject. I'll stick with using swagger-tools for now as that is what some of our other projects already use, and keep an eye on this ticket. When the full migration path is ready I'll make the jump then.

[thecodejunkie]

@whitlockjc would you say that the stack, that you mention in the initial post, is ready to be used in production now? I.e would you recommend people migrating of swagger-tool and onto that stack? I am currently looking at stating a new project and have been playing around with swagger-tools and only found this post because of you comment on #443

Do you know of an examples/tutorials that use the mentioned replacement stack / workflow?

Thanks!

[whitlockjc]

sway is most definitely ready for production. It's on the cusp of a release that includes some major json-refs improvements but it should be out this week or next. The CLI and middleware repositories need some love but I did get some help recently so when sway gets released, we're rolling over to those repos.

[liviuignat]

@whitlockjc using swagger-tools was very good for us and congrats on building such tools for the community.

We have a few APIs using it, and especially one is becoming quite large (I think @140 endpoints already). Of course, we are going to split it in smaller APIs, but it does not make sense for now.

To manage this API, we've split the swagger yaml in multiple files. Eg: we have a models folder, we have one swagger file defining the route per controller folder and in the end it all comes together in one main file, via $refs.

To combine all the refs we used json-schema-ref-parser package. The final swagger.JSON has @100kb on disk.

Now, I didn't spend time to investigate if it's from swagger-tools or not, but one issue I've seen is that it's using quite some memory on start-up to build the final swagger.JSON file. Any thoughts on this and if it's improved in sway?

Thanks and keep up the great work!

[whitlockjc]

swagger-tools uses json-refs to resolve references. The version in swagger-tools is old and has a number of bugs/performance issues. sway uses the latest json-refs (3.0.0) and these issues have been resolved. When benchmarking json-refs@v3.0.0, it was shown to be the fastest resolver available. And since I'm not in the business of bad-mouthing other projects, I'll just give you the benchmark details in this form against a well-known JSON References resolver:

Time spent (name-omitted): 43544 ms
Time spent (json-refs): 831 ms

The benchmark was ran on a document that use to take ~48 minutes json-refs to resolve, the version of json-refs in swagger-tools. (The issue with json-refs was years of bug fixes and monkey patches on top of a brute force algorithm. The new version is very clean and uses a graph traversal for processing.)

So yes, the time spent resolving references in swagger-tools is a known issue and has been fixed in the underlying library but has not been merged to swagger-tools.

[whitlockjc]

To quantify the document used for benchmarking:

• Lines: 28396,
• Reference Count: 3967
• Referenced Items: 355
• Circular References: 22 (deeply circular)

[liviuignat]

@whitlockjc are there any examples about how to use sway together with express framework?

[whitlockjc]

I'm embarrassed to point you to sway-connect due to its dormancy but it was the planned project for migrating the swagger-tools based middleware to sway. You can also look at the sway API documentation to get a feel for what APIs you'd use in your middlewares.

[liviuignat]

No worries, for now we are happy with swagger-tools, but we wanted to make sure we are close to the latest version of our dependencies and also get a small boost in performance. My hope is that swagger-tools will still be maintained for a while. I think it would be great to provide some middleware for express on top of sway in the future, but I understand that time is not working for us 🙂.

Also, the documentation is very helpful to get an in-depth view, but I think it would be great to have also some "getting started" code examples, to be able to get started and test the framework as fast as possible.

Cheers

[whitlockjc]

Thanks for the feedback. I am currently working on the connect middleware for sway.

[ramirogm]

Hi I'm not sure if this the right place to ask about the current roadmap. I'm using swagger-tools with a Swagger 2.0 API, have integrated OAuth2, and I'm very happy with the result so thanks for the package! Any recommendations on what to use for OpenAPI3? I see that there hasn't been much activity on the sway-connect.