Skip to content

ci: fail when the published OpenAPI spec is stale - #43841

Open
shivamgoel wants to merge 2 commits into
apache:masterfrom
shivamgoel:ci/openapi-spec-drift
Open

ci: fail when the published OpenAPI spec is stale#43841
shivamgoel wants to merge 2 commits into
apache:masterfrom
shivamgoel:ci/openapi-spec-drift

Conversation

@shivamgoel

Copy link
Copy Markdown
Contributor

SUMMARY

docs/static/resources/openapi.json is generated from the running app but committed by hand, so it drifts silently. The last genuine regeneration before #43788 was #33378, sixteen months earlier. This regenerates the spec on every PR and fails on a dirty diff.

Two generation inputs are pinned deliberately, because getting either wrong was a real failure while preparing #43788. The job installs requirements/base.txt: apispec 6.10.0 renders marshmallow 4's unknown=RAISE as "additionalProperties": false where the pinned 6.6.1 does not, so an unpinned guard would fail every PR. And it passes no config file, so generation reflects the routes a default deployment registers — a config with feature flags enabled documents paths that 404 for everyone who hasn't enabled them.

No paths: filter, matching enforce-single-migration-head: a required check that never runs on a PR blocks it forever. The job is ~10s.

TESTING INSTRUCTIONS

Passes on unmodified master — the committed spec reproduces byte-identically on the pinned requirements. To see it fail, add a field to any marshmallow schema without regenerating:

SUPERSET__SQLALCHEMY_DATABASE_URI='sqlite:///:memory:' \
  FLASK_APP='superset.app:create_app()' superset update-api-docs
git diff --stat -- docs/static/resources/openapi.json

Adding one schema renumbers apispec's auto-generated suffixes (User1, Subject1), so diffs are routinely larger than the change.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
  • Introduces new feature or API
  • Removes existing feature or API

@bito-code-review

bito-code-review Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Bito Automatic Review Skipped - Files Excluded

Bito didn't auto-review this change because all changed files are in the exclusion list for automatic reviews. No action is needed if you didn't intend for the agent to review it. Otherwise, to manually trigger a review, type /review in a comment and save.
You can change the excluded files settings here, or contact your Bito workspace admin at evan@preset.io.

@github-actions github-actions Bot added the github_actions Pull requests that update GitHub Actions code label Sep 4, 2026
The spec is generated from the running app but committed by hand, so it drifts
silently — the last genuine regeneration was apache#33378, sixteen months before the
one in apache#43788. Regenerate on every PR and fail on a dirty diff.

The job installs requirements/base.txt and passes no config file, because both
inputs change the output: apispec 6.10.0 renders marshmallow 4's unknown=RAISE
as "additionalProperties": false where the pinned 6.6.1 does not, and a config
with feature flags enabled documents paths that 404 by default.
@shivamgoel
shivamgoel force-pushed the ci/openapi-spec-drift branch from c8fa99d to 1fd71eb Compare September 4, 2026 01:38
@netlify

netlify Bot commented Sep 4, 2026

Copy link
Copy Markdown

Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit da98ec0
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6a9a25365a9b18000833ba2a
😎 Deploy Preview https://deploy-preview-43841--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

# paths that would 404 for everyone who has not enabled them.
SUPERSET__SQLALCHEMY_DATABASE_URI: "sqlite:///:memory:"
FLASK_APP: "superset.app:create_app()"
run: superset update-api-docs

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new required check is already failing because regenerating the spec changes openapi.json (110 additions, 46 deletions), so the PR cannot merge as written. Should the regenerated artifact be included with this guard?

@bito-code-review

Copy link
Copy Markdown
Contributor

Yes, the regenerated artifact should be included in the PR. The current workflow is designed to enforce that the checked-in openapi.json matches the output generated by the project's pinned dependencies and configuration. If the spec changes during the check, it indicates that the committed file is stale or was generated in an environment that does not match the project's requirements. To resolve the failure, you should regenerate the spec locally using the exact environment specified in the workflow steps and commit the updated openapi.json to the PR.

.github/workflows/check-openapi-spec-drift.yml

- name: Assert the published spec is up to date
        run: |
          if ! git diff --quiet -- docs/static/resources/openapi.json; then
            echo "::error::docs/static/resources/openapi.json is stale."
            echo "Regenerate it on the pinned requirements, with no config file:"
            echo "  SUPERSET__SQLALCHEMY_DATABASE_URI='sqlite:///:memory:' \"
            echo "    FLASK_APP='superset.app:create_app()' superset update-api-docs"
            git diff --stat -- docs/static/resources/openapi.json
            exit 1
          fi

@pull-request-size pull-request-size Bot added size/L and removed size/M labels Sep 4, 2026
@github-actions github-actions Bot added the doc Namespace | Anything related to documentation label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Namespace | Anything related to documentation github_actions Pull requests that update GitHub Actions code size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants