Skip to content

OpenAPI docs with webhooks + schema refs fail with filtering #712

Open
@MahdiBM

Description

@MahdiBM
Contributor

Description

Considering target 'GitHubAPI':
- Trying OpenAPI code generation.
Error: Failed to satisfy: JSONSchema reference can be found in components/schemas at path: .webhooks.branch-protection-configuration-disabled.post.requestBody.content['application/json'].schema
Failed to satisfy: JSONSchema reference can be found in components/schemas at path: .webhooks.branch-protection-configuration-enabled.post.requestBody.content['application/json'].schema

There are a lot more similar Failed to satisfy errors.

Reproduction

Try to use the OpenAPI Generator on https://github.com/github/rest-api-description/blob/main/descriptions-next/api.github.com/api.github.com.yaml

Package version(s)

generator 1.6.0
runtime 1.7.0

Expected behavior

work.

Environment

Swift 6.0.3

Additional information

No response

Activity

added
kind/bugFeature doesn't work as expected.
status/triageCollecting information required to triage the issue.
on Jan 3, 2025
MahdiBM

MahdiBM commented on Jan 3, 2025

@MahdiBM
ContributorAuthor

commenting out all paths.webhooks make things work, though there are still a lot of these warnings:

warning: Schema "null" is not supported, reason: "schema type", skipping [context: foundIn=Components.Schemas.ContributorActivity (#/components/schemas/contributor-activity)/author]
warning: Schema "array ()" is not supported, reason: "not an object-ish schema (object, ref, allOf)", skipping [context: foundIn=Operations.ReposGetContent.Output.Ok.Body (#/paths/repos/{owner}/{repo}/contents/{path}/GET/responses/200/content)]
warning: A property name only appears in the required list, but not in the properties map - this is likely a typo; skipping this property. [context: foundIn=Operations.ReposCreatePagesSite.Input.Body.JsonPayload.Value1Payload (#/paths/repos/{owner}/{repo}/pages/POST/requestBody/json/value1)/source]
czechboy0

czechboy0 commented on Jan 3, 2025

@czechboy0
Contributor

Can you confirm the exact commit of the GitHub OpenAPI doc you're using? I just pulled the latest and it generates successfully for me (with warnings, but without errors).

MahdiBM

MahdiBM commented on Jan 4, 2025

@MahdiBM
ContributorAuthor

@czechboy0 didn't use any special documents, just the one linked which is the latest committed to the main branch. This to be precise.

I simply substituted this file with the one in Penny's Lambdas/GitHubAPI folder, and reran the command plugin.

MahdiBM

MahdiBM commented on Jan 4, 2025

@MahdiBM
ContributorAuthor

I also noticed the problems only happen when using the 3.1.0 OpenAPI document, not the 3.0.3 one.

MahdiBM

MahdiBM commented on Jan 4, 2025

@MahdiBM
ContributorAuthor

Also these all happened before the latest PR ... so if you want to try something in Penny, you should checkout https://github.com/vapor/penny-bot/tree/b3fea03e431d81edf0edc4b04ca78bb83b7450e6.

I went through updating the openapi file and i used the filter feature to only create stuff that we need, so it might not try to create those stuff that result in errors.

czechboy0

czechboy0 commented on Jan 4, 2025

@czechboy0
Contributor

I'm saying I can't reproduce this issue on a new empty sample project. And are you saying the error isn't occurring once you've filtered only to the parts of the document you use? That'd be a good hint at which part of the pipeline is failing

MahdiBM

MahdiBM commented on Jan 4, 2025

@MahdiBM
ContributorAuthor

@czechboy0 surely yes ... the generator doesn't totally fail, it just stops since it sees some things that it thinks are errors.
If you comment out the stuff that the generator doesn't like, then it does work to some extent.

Though now that we're talking I also want to mention that even after the comment-outs and the generator working, it was super confusing to me that with the 3.1.0 OpenAPI document there were a lot more of those e.g. warning: Schema warnings, and how the generator just completely skipped generating some important properties that Penny was using. I thought the OpenAPI spec has just changed that much but after 1-2 hours of debugging noticed that the problem is that the generator is just skipping some properties since they seem erroneous to it.
This can be its own issue, but just wanted to mention it.

czechboy0

czechboy0 commented on Jan 4, 2025

@czechboy0
Contributor

Please file individual issues if you find anything that's well-formed under the OpenAPI specification (we do skip malformed content in some places), the generator claims to support it, but isn't generating it. That'd be considered a bug.

Let me take a look at the commit hash you shared of Penny to try to reproduce it, I wasn't able to repro on a clean project yesterday.

czechboy0

czechboy0 commented on Feb 7, 2025

@czechboy0
Contributor

I tried again today, with the latest GitHub OpenAPI doc, and the latest Swift OpenAPI Generator, and I can successfully generate code from it for both client and server. I'm using the 3.1 docs, from here: https://raw.githubusercontent.com/github/rest-api-description/main/descriptions-next/api.github.com/api.github.com.yaml

Are you still able to reproduce the issue? If so, can you reproduce it outside of Penny?

MahdiBM

MahdiBM commented on Feb 9, 2025

@MahdiBM
ContributorAuthor

Just tried. Still getting the same errors in Penny. Haven't tried outside Penny but I would be surprised if it's a Penny thing.
This is the Penny openapi config file but I don't think that matters either?

czechboy0

czechboy0 commented on Feb 9, 2025

@czechboy0
Contributor

Might be your version of the tools or something. Can you try with any of the hello world projects in Swift OpenAPI Generator's Examples directory? Just drop in the OpenAPI doc. Does it successfully generate? (It does for me)

MahdiBM

MahdiBM commented on Feb 9, 2025

@MahdiBM
ContributorAuthor

Simply replacing the openapi.yaml file in hello-world-async-http-client-example with the GitHub one, emits a lot of warnings but does not fail (Swift 6.0.3 RELEASE, generator 1.7.0, runtime 1.8.0).

Using Penny's openapi config file does make it throw errors.

Removing filter for some reason makes things work (so warnings but not errors). So:

generate:
  - types
  - client
accessModifier: package
namingStrategy: idiomatic
filter:
-  operations:
-    - apps/create-installation-access-token
-    - issues/create
-    - issues/get
-    - projects/create-card
-    - projects/delete-card
-    - projects/list-cards
-    - projects/move-card
-    # Need to manually mark auto_merge.commit_title and .commit_message as nullable.
-    # Also pull_request.assignee, .body, .closed_at, .merge_commit_sha, .merged_at, .milestone, .additions, .changed_files, .comments, .commits, .deletions, .mergeable, .mergeable_state, .merged, .maintainer_can_modify, .merged_by, .review_comments.
-    - pulls/get
-    - pulls/list-files
-    - pulls/list-reviews
-    - pulls/update
-    # Need to manually mark all schemas.repository date fields as accepting both string and number.
-    # For example "pushed_at", "created_at", "updated_at".
-    # Look at other "pushed_at" fields in other schemas.
-    - repos/compare-commits
-    - repos/create-release
-    - repos/get-latest-release
-    - repos/list-commits
-    - repos/list-contributors
-    - repos/list-pull-requests-associated-with-commit
-    - repos/list-releases
-    - repos/list-tags
-    - users/get-by-username
-  schemas:
-    - installation-token
-    # Need to manually mark label.description as nullable
-    - label
-    # Need to manually mark simple-commit.committer.email as nullable
-    - simple-commit
-    # Need to manually mark simple-installation.node_id as nullable
-    - simple-installation

Removing only one of filter.schemas and filter.operations is not enough, you need to remove both to avoid getting errors.
You don't need to remove filter: apparently, as long as it's empty.

changed the title [-]Problems with GitHub's OpenAPI 3.1 spec[/-] [+]OpenAPI docs with webhooks + schema refs fail with filtering[/+] on Feb 9, 2025

5 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/generatorAffects: plugin, CLI, config file.kind/bugFeature doesn't work as expected.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      Participants

      @czechboy0@MahdiBM

      Issue actions

        OpenAPI docs with webhooks + schema refs fail with filtering · Issue #712 · apple/swift-openapi-generator