Invalid request body reference passes validation, but throws NPE in codegen

[wheezil]

Description

A spec with an invalid request body reference will pass validation, but get NullPointerException during codegen.

The NPE happens in DefaultCodegen.fromRequestBody:

    public CodegenParameter fromRequestBody(RequestBody body, Map<String, Schema> schemas, Set<String> imports, String bodyParameterName) {
...
        Schema schema = ModelUtils.getSchemaFromRequestBody(body);
        if (StringUtils.isNotBlank(schema.get$ref())) {

schema is not found, and ModelUtils.getSchemaFromRequestBody returns null.

openapi-generator version

3.3.0-SNAPSHOT

OpenAPI declaration file content or url

{
   "openapi": "3.0.0",
   "info": {
      "description": "blah",
      "version": "1.0.0",
      "title": "blah"
   },
   "paths": {
      "/test1": {
         "post": {
            "tags": [ "test" ],
            "operationId": "testOp1",
            "responses": {
               "405": { "description": "Invalid input" }
            },
            "requestBody": {
               "$ref": "#/components/requestBodies/Missing"
            }
         }
      }
   },
   "components": {
      "requestBodies": {
         "RequestBody": {
            "content": {
               "application/json": {
                  "schema": {
                     "$ref": "#/components/schemas/Request"
                  }
               }
            }
         }
      },
      "schemas": { "Request": { "properties": { "status": { "type": "string" } } } }
   }
}

Config file:

{
	"declspec" : "DL_GLOBAL_CLASS",
	"appName" : "RPDM",
	"appDescription" : "RedPoint Data Management services"
}

Command line used for generation

java -jar OPENAPIVERSION.jar generate -v -o genjava -g java -i invalid_request_body_ref.json -c rpdmcpp.config

Steps to reproduce

Run the generate command above

Related issues/PRs

None I could find.

Suggest a fix/enhancement

Returning null from not-found references seems like a bad idea. Is this ever valid? If not, throw an exception:

    public static RequestBody getReferencedRequestBody(OpenAPI openAPI, RequestBody requestBody) {
        if (requestBody != null && StringUtils.isNotEmpty(requestBody.get$ref())) {
            String name = getSimpleRef(requestBody.get$ref());
            RequestBody referencedRequestBody = getRequestBody(openAPI, name);
            if (referencedRequestBody != null) {
                return referencedRequestBody;
            }
else {
>>>> throw an exception
}
        }
        return requestBody;
    }

Overall, the "return null if not found" convention seems to be poorly documented at best, and probably just wrong, given how easy it is to cause an NPE.

[adamdecaf]

I'm having the same problem, but my schema objects aren't missing. Any advice?

I've tried 3.3.1 and the 3.4.0-SNAPSHOT

openapi: "3.0.2"
info:
  description: test
  version: "v1"
  title: "Moov API"
  contact:
    email: security@moov.io
    url: "https://groups.google.com/forum/#!forum/moov-users"
  license:
    name: "Apache 2.0"
    url: "http://www.apache.org/licenses/LICENSE-2.0.html"
servers:
  - url: https://api.moov.io
    description: Production server
tags:
  - name: User
    description: User represents an entity that can create api auth tokens used to make requests.

paths:
  /v1/users/{user_id}:
    patch:
      tags:
        - User
      summary: Update a User's profile information
      operationId: updateUserProfile
      security:
        - cookieAuth: []
      parameters:
        - $ref: '#/components/parameters/cookie'
        - name: user_id
          in: path
          description: Moov API User ID
          required: true
          schema:
            type: string
            example: 3f2d23ee214
      requestBody:
        description: "test" # TODO(adam)
        content:
          application/json:
            schema:
              $ref: '#/components/requestBodies/UserProfile'
      responses:
        '200':
          description: User profile updated
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
        '400':
          description: Invalid request body, check error(s).
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
components:
  schemas:
    Error:
      required:
        - error
      properties:
        error:
          type: string
          description: An error message describing the problem intended for humans.
          example: Validation error(s) present.
    User:
      properties:
        id:
          description: Moov API user ID
          type: string
          example: c05ad98a
        firstName:
          type: string
          example: Taylor
  requestBodies:
    UserProfile:
      description: 'User profile information'
      # required: true
      content:
        application/json:
          schema:
            properties:
              firstName:
                type: string
                description: Legal first name
                example: Jane
  parameters:
    cookie:
      name: cookie
      in: cookie
      schema:
        type: string
      description: moov_auth Cookie
  securitySchemes:
    cookieAuth:
      type: apiKey
      in: header
      name: Cookie
      description: moov_auth Cookie header

$ make
chmod +x ./openapi-generator
rm -rf ./client
OPENAPI_GENERATOR_VERSION=3.4.0-SNAPSHOT ./openapi-generator generate -i openapi.yaml -g go -o ./client
[main] WARN  o.o.c.ignore.CodegenIgnoreProcessor - Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[main] INFO  o.o.c.languages.AbstractGoCodegen - Environment variable GO_POST_PROCESS_FILE not defined so Go code may not be properly formatted. To define it, try `export GO_POST_PROCESS_FILE="/usr/local/bin/gofmt -w"` (Linux/Mac)
[main] INFO  o.o.codegen.AbstractGenerator - writing file /Users/adam/code/src/github.com/moov-io/go-client/./client/model_error.go
[main] INFO  o.o.codegen.AbstractGenerator - writing file /Users/adam/code/src/github.com/moov-io/go-client/./client/docs/Error.md
[main] INFO  o.o.codegen.AbstractGenerator - writing file /Users/adam/code/src/github.com/moov-io/go-client/./client/model_user.go
[main] INFO  o.o.codegen.AbstractGenerator - writing file /Users/adam/code/src/github.com/moov-io/go-client/./client/docs/User.md
Exception in thread "main" java.lang.RuntimeException: Could not process operation:
  Tag: class Tag {
    name: User
    description: User represents an entity that can create api auth tokens used to make requests.
    externalDocs: null
}
  Operation: updateUserProfile
  Resource: patch /v1/users/{user_id}
  Schemas: {Error=class Schema {
    title: null
    multipleOf: null
    maximum: null
    exclusiveMaximum: null
    minimum: null
    exclusiveMinimum: null
    maxLength: null
    minLength: null
    pattern: null
    maxItems: null
    minItems: null
    uniqueItems: null
    maxProperties: null
    minProperties: null
    required: [error]
    type: null
    not: null
    properties: {error=class StringSchema {
        class Schema {
            title: null
            multipleOf: null
            maximum: null
            exclusiveMaximum: null
            minimum: null
            exclusiveMinimum: null
            maxLength: null
            minLength: null
            pattern: null
            maxItems: null
            minItems: null
            uniqueItems: null
            maxProperties: null
            minProperties: null
            required: null
            type: null
            not: null
            properties: null
            additionalProperties: null
            description: An error message describing the problem intended for humans.
            format: null
            $ref: null
            nullable: null
            readOnly: null
            writeOnly: null
            example: Validation error(s) present.
            externalDocs: null
            deprecated: null
            discriminator: null
            xml: null
        }
        type: string
        _default: null
        _enum: null
    }}
    additionalProperties: null
    description: null
    format: null
    $ref: null
    nullable: null
    readOnly: null
    writeOnly: null
    example: null
    externalDocs: null
    deprecated: null
    discriminator: null
    xml: null
}, User=class Schema {
    title: null
    multipleOf: null
    maximum: null
    exclusiveMaximum: null
    minimum: null
    exclusiveMinimum: null
    maxLength: null
    minLength: null
    pattern: null
    maxItems: null
    minItems: null
    uniqueItems: null
    maxProperties: null
    minProperties: null
    required: null
    type: null
    not: null
    properties: {id=class StringSchema {
        class Schema {
            title: null
            multipleOf: null
            maximum: null
            exclusiveMaximum: null
            minimum: null
            exclusiveMinimum: null
            maxLength: null
            minLength: null
            pattern: null
            maxItems: null
            minItems: null
            uniqueItems: null
            maxProperties: null
            minProperties: null
            required: null
            type: null
            not: null
            properties: null
            additionalProperties: null
            description: Moov API user ID
            format: null
            $ref: null
            nullable: null
            readOnly: null
            writeOnly: null
            example: c05ad98a
            externalDocs: null
            deprecated: null
            discriminator: null
            xml: null
        }
        type: string
        _default: null
        _enum: null
    }, firstName=class StringSchema {
        class Schema {
            title: null
            multipleOf: null
            maximum: null
            exclusiveMaximum: null
            minimum: null
            exclusiveMinimum: null
            maxLength: null
            minLength: null
            pattern: null
            maxItems: null
            minItems: null
            uniqueItems: null
            maxProperties: null
            minProperties: null
            required: null
            type: null
            not: null
            properties: null
            additionalProperties: null
            description: null
            format: null
            $ref: null
            nullable: null
            readOnly: null
            writeOnly: null
            example: Taylor
            externalDocs: null
            deprecated: null
            discriminator: null
            xml: null
        }
        type: string
        _default: null
        _enum: null
    }}
    additionalProperties: null
    description: null
    format: null
    $ref: null
    nullable: null
    readOnly: null
    writeOnly: null
    example: {firstName=Taylor, id=c05ad98a}
    externalDocs: null
    deprecated: null
    discriminator: null
    xml: null
}}
  Exception: null
	at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:996)
	at org.openapitools.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:892)
	at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:489)
	at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:848)
	at org.openapitools.codegen.cmd.Generate.run(Generate.java:349)
	at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:62)
Caused by: java.lang.NullPointerException
	at org.openapitools.codegen.utils.ModelUtils.isMapSchema(ModelUtils.java:325)
	at org.openapitools.codegen.DefaultCodegen.fromRequestBody(DefaultCodegen.java:4444)
	at org.openapitools.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:2402)
	at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:964)
	... 5 more
make: *** [client] Error 1

[adamdecaf]

It looks like there's a problem if you $ref requestBodies? This commit doesn't NPE and generates the Go client fine.

https://github.com/moov-io/api/pull/24/commits/5486832ce4fa8eed614b0fa8beb76b6b0db4e99d

[adamdecaf]

I figured out my problem. It was missing components.schemas for a components.requestBodies.

https://github.com/moov-io/api/pull/43 was the fix for my project

[mindhaq]

I have a similar problem, but no NPE. Instead:

[main] WARN  o.o.codegen.DefaultCodegen - codegenModel is null. Default to UNKNOWN_BASE_TYPE

I moved a requestBody definition to components/requestBodies and added a $ref (I would consider this a refactoring), and then got that error.

[adamdecaf]

I had the same problem and after moving them into components.schemas rather than components.requestBodies (as a $ref towards components.schemas) that problem went away.

See: https://github.com/moov-io/api/pull/43/files#diff-fe030a7c1568b3decf599edf399be7f4

[shuninghuang]

so the conclusion is like openapi-generator doesnt deal with requestBodies well?
i am having the same problem

[alikhanz]

Same problem with requestBodies

[lornajane]

I also ran into this issue - the validation step passes, as does my usual validation tooling (spectral) - but the generated code has the UNKNOWN_BASE_TYPE in it and the warning message about a v2 URL. In my case, I had a schema with properties but no type inside it. The validators don't seem to mind but adding type: object to all my schemas helped a lot! Hopefully it helps someone else too :)

[ybelenko]

This bug is really annoying I got completely broken Javascript client build because of that. I also do $refs to #/components/requestBodies/foobar.

In generated API class:

import ApiClient from "../ApiClient";
import ModelApplication from '../model/ModelApplication';
import UNKNOWN_BASE_TYPE from '../model/UNKNOWN_BASE_TYPE';

Then compile error:

ERROR in ./out/api-client/src/api/MessagesApi.js 20:0-59
Module not found: Error: Can't resolve '../model/UNKNOWN_BASE_TYPE' in '/Users/ybelenko/Sites/front_react/out/api-client/src/api'

[adelcsc]

First of all, you have to override the void processOpenAPI(OpenAPI openAPI) function and implement a way of extracting the requestBodies from openAPI.getComponents().getRequestBodies() to openAPI.getComponents().getSchemas()

here is mine :

    @Override
    public void processOpenAPI(OpenAPI openAPI) {
        //Include Request bodies into models
        if(openAPI.getComponents().getRequestBodies()!=null)
            openAPI.getComponents().getRequestBodies().forEach((requestName, requestBody) -> {
                if(openAPI.getComponents().getSchemas()!=null)
                    openAPI.getComponents().getSchemas().put(requestName.contains("Request")?requestName:requestName+"Request",requestBody.getContent().get("application/json").getSchema());
            });
        //Include Responses into models
        if(openAPI.getComponents().getResponses()!=null)
            openAPI.getComponents().getResponses().forEach((responseName, apiResponse) -> {
                openAPI.getComponents().getSchemas().put(responseName.contains("Response")?responseName:responseName+"Response",apiResponse.getContent().get("application/json").getSchema());
            });
        super.processOpenAPI(openAPI);
    }

this code will add all requestBodies and responseBodies to Models with a suffix Of Request and Response respectively.

then you need to add this name = schema.getTitle(); at the start of (i think this is what's missing) protected void addBodyModelSchema(...) that is located in DefaultCodegen.java class

[yazumoto]

I had same problem. And I fixed (a bit troublesome though...)
Use $ref in components.requestBodies.content.<type>.schema.

openapi: 3.0.0
info:
  title: title
  version: 1.0.0
  description: desc
paths:
  /test/update:
    post:
      summary: test
      operationId: updateTest
      requestBody:
        $ref: '#/components/requestBodies/updateTestBody'
      responses:
        200:
          description: Success
components:
  schemas:
    UpdateTest:
      type: object
      required:
        - id
        - name
      properties:
        id:
          type: integer
        name:
          type: string
  requestBodies:
    updateTestBody:
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/UpdateTest' # This is important!!