[Meta Issue]: OpenAPI Support

[sennyeya]

🔖 Feature description

This is a collection of issues stemming from #2566. The goal of this ticket is to break apart the different types of work from that original ticket into separate camps and define a clearer roadmap for OpenAPI support in Backstage.

What do you mean by OpenAPI support in Backstage?

This can mean a multitude of things, which is partially why we want to replace the old ticket. Here it will mean

1. OpenAPI specs for plugins.
2. An OpenAPI spec-first developer experience, using the spec to drive server-side code writing.
3. A Backstage-only client generated using the spec.
4. Other language clients like Java, Python for interfacing with Backstage plugins.
5. new API fuzzing of OpenAPI documented plugins

Plugin Specs

• catalog-backend ([PRFC] OpenAPI Typings for Routers #15667, feat(catalog-backend): Add OpenAPI 3.1 yaml spec. #16238)
• search-backend (feat(search-backend): Add spec and update to use ApiRouter typing. #17511)

OpenAPI Specification Updates

• Add links to existing specs.
• Figure out how to deduplicate refs to common properties (Entity, User, etc)

Spec-first Development

• Typed express router methods ([PRFC] OpenAPI Typings for Routers #15667)
• Request, params and response validation (feat(openapi-tooling): Add support for request validation. #17875)
• Auto-generated server stubs (feat(openapi-tooling): Add support for request validation. #17875)
• Test-driven feedback of OpenAPI specs (feat: add support for validating OpenAPI specs against test runs #19955)
• Document how to write an OpenAPI spec

Internal Client Autogeneration

• Custom generator from openapi-generator (feat(catalog-client): Support generating a front end client based on an OpenAPI spec #17470)
• Migrate existing clients to use autogenerated versions
  • catalog-backend
  • others

Additional Client Autogeneration

• Support for additional languages
  • Java
  • Python
  • Typescript

API Fuzzing

Context:

• Integrate with Schemathesis (feat(openapi-tooling): add support for fuzzing with schemathesis #23084).

Open Questions

• Should we have a single monolithic client that includes all plugins and specs or should each plugin have its own autogenerated client?

Others

• Serving these specs in Backstage for meta discovery. ([PRFC] API Docs for the whole backend #17535)
• openapi.json endpoint with a runtime generated API spec ([PRFC] API Docs for the whole backend #17535)

[sennyeya]

Also, found this library for jest that adds spec validation into the responses, https://github.com/openapi-library/OpenAPIValidators/tree/master/packages/jest-openapi#readme. Would be useful for verifying response values in test cases as a precursor to turning on validateResponses in the middleware.

[dannysheridan]

@sennyeya as you think about the client autogeneration, you ought to check out Fern

[Rugvip]

@dannysheridan apart from discussion on the technical solution, this part of the experience is a blocker for us to adopt Fern as a default tool in the Backstage OSS project:

[dsinghvi]

@dannysheridan apart from discussion on the technical solution, this part of the experience is a blocker for us to adopt Fern as a default tool in the Backstage OSS project:

@Rugvip we can add a --skip-login option on init/generate. Would that unblock adopting fern? (So far enforcing login has been great for analytics/support but I see why that's extremely limiting in an OSS project).

[Rugvip]

@dsinghvi If login is not required and there are no usage limitations for the core use-cases we're likely at a point where we wouldn't dismiss it. We'd still ofc need to evaluate whether it's a desirable solution and what level of lock-in there is.

[dannysheridan]

@Rugvip at Fern, we're going to make login not required if you download files locally. There would be no usage limitations for the core use-cases. Are you up to hop on a call to discuss further? here's my cal

[Stranger6667]

@sennyeya if you need anything from the Schemathesis side, feel free to tag me and I’ll be happy to provide support on integrating it :)

[sennyeya]

@Stranger6667 That was quick! Will definitely keep you in the loop as I start exploring. Thanks for the great tool!

[vinzscam]

hey have we considered finding a way to avoid running the openapi generate-client script when developing?

I felt the pain of writing yaml, then running a script to generate a .ts file which is then consumed by our typescript bundler.
I found the process really slow and is yet another thing that contributors need to learn before contributing to Backstage.

I would love to just write the openapi definition in typescript instead.

[sennyeya]

@vinzscam Reached out via DM, happy to help but I'd like a little more info on your use case.

[vinzscam]

@vinzscam Reached out via DM, happy to help but I'd like a little more info on your use case.

nothing fancy I am just developing a plugin which integrates with OpenAPI and found the process a bit complicated 😅

I'm thinking that writing the definition directly in typescript (for example by using some zod to openapi library) might simplify the development process.