Move Scaffolder API to OpenAPI by solimant · Pull Request #27771 · backstage/backstage

solimant · 2024-11-21T16:51:16Z

Hey, I just made a Pull Request!

Server and client generated using:

yarn backstage-repo-tools package schema openapi generate --server --client-package plugins/scaffolder-common

✔️ Checklist

A changeset describing the change and affected packages. (more info)
Added or updated documentation
Tests for new functionality and regression tests for bug fixes
Screenshots attached (for UI changes)
All your commits have a Signed-off-by line in the message. (more info)

backstage-goalie · 2024-11-21T16:51:45Z

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@backstage/backend-openapi-utils	packages/backend-openapi-utils	minor	`v0.5.5`
@backstage/repo-tools	packages/repo-tools	patch	`v0.15.0`
@backstage/plugin-scaffolder-backend	plugins/scaffolder-backend	minor	`v2.1.0`
@backstage/plugin-scaffolder-common	plugins/scaffolder-common	minor	`v1.6.0`
@backstage/plugin-scaffolder-node	plugins/scaffolder-node	minor	`v0.10.0`
@backstage/plugin-scaffolder-react	plugins/scaffolder-react	minor	`v1.18.1-next.0`
@backstage/plugin-scaffolder	plugins/scaffolder	minor	`v1.33.1-next.0`

backstage-goalie · 2024-11-21T16:57:01Z

Thanks for the contribution!
All commits need to be DCO signed before they are reviewed. Please refer to the the DCO section in CONTRIBUTING.md or the DCO status for more info.

Rugvip

Thank you for looking at this! 👍

Is the goal to wire up the router etc with this as well in this PR? I'd really be needed to ensure that the schema is correct and remains so.

Rugvip · 2024-11-22T09:19:44Z

+          type: object
+          properties:
+            entity:
+              $ref: '../../../catalog-backend/src/schema/openapi.yaml#/components/schemas/UserEntity'


To be honest I think it's best to keep this one opaque. As an adopter of Backstage you can extend the UserEntity schema anyway, so there's kinda no point aiming to model this imo.

Fair point. I'll adjust here, and remove the addition from the Catalog OpenAPI schema.

solimant

Is the goal to wire up the router etc with this as well in this PR? I'd really be needed to ensure that the schema is correct and remains so.

@Rugvip - Yes, that's the goal, and I'm working on making more commits for that. And, yes, the schema needs to be correct 100%. That's the reason why I started with that since everything else is pretty much a matter of code generation.

solimant · 2024-11-22T11:44:14Z

+          type: object
+          properties:
+            entity:
+              $ref: '../../../catalog-backend/src/schema/openapi.yaml#/components/schemas/UserEntity'


Fair point. I'll adjust here, and remove the addition from the Catalog OpenAPI schema.

solimant · 2024-12-06T14:50:45Z

@aramissennyeydd - I'm cleaning up a few things, and I noticed that we've got some schemas defined in OpenAPI and the corresponding generated TypeScript type, and also defined separately (manually) in other places. For example, Entity defined in packages/catalog-model/src/entity/Entity.ts and in plugins/catalog-backend/src/schema/openapi/generated/models/Entity.model.ts. Is that intentional?

solimant · 2024-12-06T14:51:01Z

@benjdlambert - On a separate note, I'm seeing an issue with properties that have special characters, such as backstage:permissions and EXPERIMENTAL_recovery, where the generated TypeScript type is omitting those special characters, which seems related to OpenAPITools/openapi-generator/issues/15261. How do we feel about manually adjusting those properties to match?

solimant · 2024-12-06T14:57:59Z

@Rugvip - because I've come across a few Scaffolder types that use JsonObject and similar types from @backstage/types (e.g. TaskStep), how do you feel about defining the types in @backstage/types in OpenAPI spec, and then repurpose *.ts files under `packages/types/src to export the generated TypeScript types? The benefit here is that we can cross-reference those common types in other OpenAPI spec YAMLs across the application.

CC: @benjdlambert, @aramissennyeydd

aramissennyeydd · 2024-12-06T15:54:37Z

@solimant

For example, Entity defined in packages/catalog-model/src/entity/Entity.ts and in plugins/catalog-backend/src/schema/openapi/generated/models/Entity.model.ts. Is that intentional?

Yup this is intentional, the duplication here is minimal and the OpenAPI version should be looser than the catalog one.

@Rugvip - because I've come across a few Scaffolder types that use JsonObject and similar types from @backstage/types (e.g. TaskStep), how do you feel about defining the types in @backstage/types in OpenAPI spec, and then repurpose *.ts files under `packages/types/src to export the generated TypeScript types? The benefit here is that we can cross-reference those common types in other OpenAPI spec YAMLs across the application.

Similarly here, I don't think we gain much if we deduplicate the types and could start running into issues where we expect x version of the shared types and get y.

@benjdlambert - On a separate note, I'm seeing an issue with properties that have special characters, such as backstage:permissions and EXPERIMENTAL_recovery, where the generated TypeScript type is omitting those special characters, which seems related to OpenAPITools/openapi-generator#15261. How do we feel about manually adjusting those properties to match?

I can take a look into a fix here - in the meantime, can you just add these as an additionalProperties: {type: string} to allow them on requests and responses. That will give us an initial bit of schema definition and we can add them back once this is fixed.

solimant · 2024-12-06T16:11:36Z

+      description: Step that is part of a Template Entity.
+    TemplateEntityV1beta3:
+      allOf:
+        - $ref: '../../../catalog-backend/src/schema/openapi.yaml#/components/schemas/Entity'


Thank you very much for the response, @aramissennyeydd. Would like to get your feedback on this as well. Do we wanna cross-reference types from other OpenAPI specs like this one? For this one, the TypeScript type I'm matching is defined here, and is used in the Scaffolder router here.

nope - just copy this over from the other spec. I haven't figured out the stitching between specs yet. This kind of route may fail if you were you install this package on your local instance.

solimant · 2024-12-11T19:31:38Z

@aramissennyeydd - for the Scaffolder client, I'm thinking to do the following:

Create plugins/scaffolder-common/src/ScaffolderClient.ts where we export class ScaffolderClient implements ScaffolderApi {...}
Generate the client code under plugins/scaffolder-common/client/

What do you think? Or should we create a new package called packages/scaffolder-client similar to packages/catalog-client in structure?

solimant · 2024-12-11T19:49:47Z

I should've clarified the part about ScaffolderClient. What I meant is to relocate plugins/scaffolder/src/api.ts to plugins/scaffolder-common/src/ScaffolderClient.ts, and also plugins/scaffolder-react/src/api/types.ts to plugins/scaffolder-common/src/api.ts.

aramissennyeydd · 2024-12-12T00:53:22Z

@solimant The ScaffolderClient idea makes sense, we should just use whatever folder the yarn backstage-repo-tools package schema openapi generate --client <folder> outputs to (I believe /client).

scaffolder-common also makes sense, and we can re-export from scaffolder-react so it isn't breaking. For the CatalogClient, I kept the same API interface for the handwritten client, but changed out the internals to use the generated client - we can probably do the same here. Also helps to ensure the types match up fully.

github-actions · 2024-12-26T01:23:57Z

This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution!

solimant · 2024-12-26T01:25:25Z

Please remove the "stale" label.

awanlin · 2024-12-26T01:35:03Z

Hi @solimant, the best way to avoid this going stale while you wait for reviews is to rebase. Your branch is currently 690 commits behind.

Signed-off-by: solimant <solimant@users.noreply.github.com>

benjdlambert

Just some minor bits and then we can ship I think.

benjdlambert · 2025-07-29T07:39:25Z

@@ -0,0 +1,10 @@
+---
+'@backstage/backend-openapi-utils': minor


Suggested change

'@backstage/backend-openapi-utils': minor

This should probably be in another changeset, and also only a patch

There's also a changeset needed for repo-tool as well?

benjdlambert · 2025-07-29T07:46:45Z

  | 'failed'
  | 'open'
-  | 'processing';
+  | 'processing'


Yeah, think these types might need a cleanup, but happy to take that before the mainline release if that's OK? Going to try and ship this ASAP.

benjdlambert · 2025-07-29T07:48:32Z

    if (taskId) {
      analytics.captureEvent('retried', 'Template has been retried');
-      await scaffolderApi.retry?.(taskId);
+      await scaffolderApi.retry?.({ taskId });


I think that this is a breaking change to the client which we can't make yet, this needs updating in our wrapping client to take just taskId.

There's a bigger piece of work which is to make sure that all the functions of the scaffolder API follow this pattern ({ taskId }) for example, but rather make that change as a bigger breaking change instead.

Signed-off-by: benjdlambert <ben@blam.sh>

benjdlambert · 2025-07-29T11:34:00Z

@solimant I created a separate changeset for the -react package with some additional callouts to all the deprecations.

I think that covers everything for now, but if you're willing to cleanup the types in a followup PR those deprecations will need to also be called out explicitly in another changeset too 🙏

Approving and auto-merge on! Thanks a lot for the effort here all round from everyone involved 🎉 @aramissennyeydd too 👍

Signed-off-by: benjdlambert <ben@blam.sh>

github-actions · 2025-07-29T12:37:17Z

Thank you for contributing to Backstage! The changes in this pull request will be part of the 1.42.0 release, scheduled for Tue, 19 Aug 2025.

solimant requested review from a team as code owners November 21, 2024 16:51

solimant requested review from benjdlambert, camilaibs and jhaals November 21, 2024 16:51

github-actions Bot added area:catalog Related to the Catalog Project Area area:scaffolder Everything and all things related to the scaffolder project area labels Nov 21, 2024

solimant mentioned this pull request Nov 21, 2024

🚀 Feature: Backend client for Scaffolder #23819

Closed

2 tasks

solimant force-pushed the 23819_scaffolder_openapi branch from a04277c to 0cd8958 Compare November 21, 2024 17:12

Rugvip reviewed Nov 22, 2024

View reviewed changes

solimant commented Nov 22, 2024

View reviewed changes

solimant force-pushed the 23819_scaffolder_openapi branch from 0cd8958 to 93cc245 Compare December 2, 2024 16:18

github-actions Bot removed the area:catalog Related to the Catalog Project Area label Dec 2, 2024

solimant commented Dec 6, 2024

View reviewed changes

github-actions Bot added the stale label Dec 26, 2024

github-actions Bot removed the stale label Dec 26, 2024

solimant added 21 commits July 28, 2025 20:59

Fix dependencies

fcd82b1

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix prettier

bd7ae85

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix API report

68eca7b

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix OpenAPI spec linting

ac5613f

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix prettier

317fa46

Signed-off-by: solimant <solimant@users.noreply.github.com>

Move deps

b7e9812

Signed-off-by: solimant <solimant@users.noreply.github.com>

Skip SSE

845e8a1

Signed-off-by: solimant <solimant@users.noreply.github.com>

Revert limit and offset test change

c475f8c

Signed-off-by: solimant <solimant@users.noreply.github.com>

Revert ResponseBodyParser changes

58d1c08

Signed-off-by: solimant <solimant@users.noreply.github.com>

Ignore type for SSE route

3c03a0a

Signed-off-by: solimant <solimant@users.noreply.github.com>

Relocate client

e8e0f8e

Signed-off-by: solimant <solimant@users.noreply.github.com>

Refactor type for SSE route

8d78543

Signed-off-by: solimant <solimant@users.noreply.github.com>

Add changeset

c08cbc4

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix API reports

67caec2

Signed-off-by: solimant <solimant@users.noreply.github.com>

Add ListTemplatingExtensions

563ff4d

Signed-off-by: solimant <solimant@users.noreply.github.com>

Clean up openapi.yaml

d9a0381

Signed-off-by: solimant <solimant@users.noreply.github.com>

Rebase tests

182e77b

Signed-off-by: solimant <solimant@users.noreply.github.com>

Remove deprecated ActionExample

0afb6e8

Signed-off-by: solimant <solimant@users.noreply.github.com>

Add generate script

90bc740

Signed-off-by: solimant <solimant@users.noreply.github.com>

Fix modelEnum.mustache

7127ac9

Signed-off-by: solimant <solimant@users.noreply.github.com>

Capture 403 errors

2ea0997

Signed-off-by: solimant <solimant@users.noreply.github.com>

benjdlambert requested changes Jul 29, 2025

View reviewed changes

benjdlambert added 3 commits July 29, 2025 13:24

chore: cleanup breaking change

df41cf9

Signed-off-by: benjdlambert <ben@blam.sh>

chore: changeset for repo tools

225d7c2

Signed-off-by: benjdlambert <ben@blam.sh>

chore: added a seperate callout for -react package deprecations

4f99e10

Signed-off-by: benjdlambert <ben@blam.sh>

benjdlambert approved these changes Jul 29, 2025

View reviewed changes

chore: fixing implementation of the retry

4dd4fa6

Signed-off-by: benjdlambert <ben@blam.sh>

benjdlambert mentioned this pull request Aug 19, 2025

Version Packages #30868

Merged

		@@ -0,0 +1,10 @@
		---
		'@backstage/backend-openapi-utils': minor

Conversation

solimant commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Hey, I just made a Pull Request!

✔️ Checklist

Uh oh!

backstage-goalie Bot commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changed Packages

Uh oh!

backstage-goalie Bot commented Nov 21, 2024

Uh oh!

Rugvip left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

solimant left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

solimant commented Dec 6, 2024

Uh oh!

solimant commented Dec 6, 2024

Uh oh!

solimant commented Dec 6, 2024

Uh oh!

aramissennyeydd commented Dec 6, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aramissennyeydd Dec 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

solimant commented Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

solimant commented Dec 11, 2024

Uh oh!

aramissennyeydd commented Dec 12, 2024

Uh oh!

github-actions Bot commented Dec 26, 2024

Uh oh!

solimant commented Dec 26, 2024

Uh oh!

awanlin commented Dec 26, 2024

Uh oh!

benjdlambert left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benjdlambert commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

solimant commented Nov 21, 2024 •

edited

Loading

backstage-goalie Bot commented Nov 21, 2024 •

edited

Loading

aramissennyeydd Dec 6, 2024 •

edited

Loading

solimant commented Dec 11, 2024 •

edited

Loading

benjdlambert commented Jul 29, 2025 •

edited

Loading