Unable to generate anything with multiple files yaml spec

[JFCote]

Description

I'm trying to generate the JavaPlayFramework with a very simple multiple files yaml spec but it doesn't work. I also tried with other server and I receive the same error. I was advised to try to validate the spec using the brand new "validate" function but I still receive the same error.

What is strange is that I am able to look at this spec using swagger-editor in local. There is not error at all. Just put both file in the swagger-editor folder and see for yourself.

There are 2 files (find the content of them below):

• swagger.yaml
• definitions.yaml

Here is the error I get every time

[main] INFO io.swagger.parser.Swagger20Parser - reading from swagger-editor/swagger.yaml
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -7
        at java.lang.String.substring(Unknown Source)
        at io.swagger.models.refs.GenericRef.computeSimpleRef(GenericRef.java:86)
        at io.swagger.models.refs.GenericRef.<init>(GenericRef.java:29)
        at io.swagger.models.properties.RefProperty.set$ref(RefProperty.java:56)
        at io.swagger.models.properties.RefProperty.<init>(RefProperty.java:18)
        at io.swagger.util.PropertyDeserializer.propertyFromNode(PropertyDeserializer.java:214)
        at io.swagger.util.PropertyDeserializer.propertyFromNode(PropertyDeserializer.java:286)
        at io.swagger.util.PropertyDeserializer.deserialize(PropertyDeserializer.java:137)
        at io.swagger.util.PropertyDeserializer.deserialize(PropertyDeserializer.java:39)
        at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3584)
        at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3508)
        at io.swagger.parser.util.SwaggerDeserializer.property(SwaggerDeserializer.java:974)
        at io.swagger.parser.util.SwaggerDeserializer.definition(SwaggerDeserializer.java:789)
        at io.swagger.parser.ResolverCache.loadRef(ResolverCache.java:135)
        at io.swagger.parser.processors.ExternalRefProcessor.processRefToExternalDefinition(ExternalRefProcessor.java:34)
        at io.swagger.parser.processors.ExternalRefProcessor.processRefProperty(ExternalRefProcessor.java:119)
        at io.swagger.parser.processors.ExternalRefProcessor.processRefToExternalDefinition(ExternalRefProcessor.java:83)
        at io.swagger.parser.processors.PropertyProcessor.processRefProperty(PropertyProcessor.java:34)
        at io.swagger.parser.processors.PropertyProcessor.processProperty(PropertyProcessor.java:21)
        at io.swagger.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:21)
        at io.swagger.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:45)
        at io.swagger.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:101)
        at io.swagger.parser.SwaggerResolver.resolve(SwaggerResolver.java:50)
        at io.swagger.parser.SwaggerParser.read(SwaggerParser.java:67)
        at io.swagger.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:420)
        at io.swagger.codegen.cmd.Generate.run(Generate.java:232)
        at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:43)

Swagger-codegen version

Latest master

Swagger declaration file content or url

swagger.yaml

swagger: '2.0'

info:
  version: "0.1.0"
  title: Title
  description: ""
  contact:
    name: me
    email: hello@foo.com
    url: http://www.foo.com
schemes:
  - http
  - https
basePath: /api
host: "localhost:9000"
consumes:
  - application/json
produces:
  - application/json
paths:
  /actors:
    delete:
      tags:
        - Actor
      summary: Kill an actor in the actor system (MAW)
      description: ""
      operationId: killActor
      parameters:
        - in: body
          name: body
          required: true
          schema:
            $ref: "./definitions.yaml#/ActorInfo"
      responses:
        200:
          description: OK
          schema:
            $ref: "./definitions.yaml#/DefaultMetaOnlyResponse"
    get:
      tags:
        - Actor
      summary: Request the status of an actor
      description: ""
      operationId: requestPrintStatus
      parameters:
        - in: body
          name: body
          required: true
          schema:
            $ref: "./definitions.yaml#/ActorInfo"
      responses:
        200:
          description: OK
          schema:
            $ref: "./definitions.yaml#/DefaultMetaOnlyResponse"

definitions.yaml

EmptyObject:
  type: object
  description: "Sample Description"
Error:
  type: object
  description: "Sample Description"
  required:
    - type
  properties:
    type:
      type: string
      description: "Sample Description"
      default: "GenericException"
      example: "GenericException"
    itemInfo:
      type: object
      description: "Sample Description"
      items:
        $ref: '#/ItemId'
    details:
      type: string
      description: "Sample Description"
      example: "Could not update that field"
    exception:
      type: string
      description: "Sample Description"
      example: "DBException + stack trace"
ItemId:
  type: object
  description: "Sample Description"
  required:
    - id
    - type
  properties:
    id:
      type: string
      description: "Sample Description"
      example: "45"
    type:
      type: string
      description: "playlist"
      example: "5667"
ResponseMeta:
  type: object
  description: "Sample Description"
  required:
    - code
  properties:
    code:
      type: string
      description: "Sample Description"
      default: Ok
      enum:
      - Ok
      example: Ok
    detail:
      type: string
      description: "Sample Description"
      example: "this is some detail about the error or the success"
    exception:
      type: string
      description: "Sample Description"
      example: "IOException + stack trace"
    errors:
      type: array
      description: "Sample Description"
      items:
        $ref: '#/Error'
ActorInfo:
  type: object
  description: "Sample Description"
  required:
    - actorPath
  properties:
    actorPath:
      type: string
      example: "/user/synchronizer"
DefaultMetaOnlyResponse:
  type: object
  required:
    - meta
  properties:
    meta:
      $ref: '#/ResponseMeta'

Command line used for generation

java -jar swagger-codegen.jar generate -i swagger-editor/swagger.yaml -l java-play-framework -o generatedServer -DhideGenerationTimestamp=true

Steps to reproduce

Just run the command line above

[jimschubert]

I'd never seen external refs defined outside of a definitions key. I checked swagger-parser, and it should be supported.

I ran the following against the csharp generator with a parent definitions key in your definitions.yaml, and it generated for me.

swagger.yaml:

swagger: '2.0'

info:
  version: "0.1.0"
  title: Title
  description: ""
  contact:
    name: me
    email: hello@foo.com
    url: http://www.foo.com
schemes:
  - http
  - https
basePath: /api
host: "localhost:9000"
consumes:
  - application/json
produces:
  - application/json
paths:
  /actors:
    delete:
      tags:
        - Actor
      summary: Kill an actor in the actor system (MAW)
      description: ""
      operationId: killActor
      parameters:
        - in: body
          name: body
          required: true
          schema:
            $ref: "./definitions.yaml#/definitions/ActorInfo"
      responses:
        200:
          description: OK
          schema:
            $ref: "./definitions.yaml#/definitions/DefaultMetaOnlyResponse"
    get:
      tags:
        - Actor
      summary: Request the status of an actor
      description: ""
      operationId: requestPrintStatus
      parameters:
        - in: body
          name: body
          required: true
          schema:
            $ref: "./definitions.yaml#/definitions/ActorInfo"
      responses:
        200:
          description: OK
          schema:
            $ref: "./definitions.yaml#/definitions/DefaultMetaOnlyResponse"

definitions.yaml:

definitions:
  EmptyObject:
    type: object
    description: "Sample Description"
  Error:
    type: object
    description: "Sample Description"
    required:
      - type
    properties:
      type:
        type: string
        description: "Sample Description"
        default: "GenericException"
        example: "GenericException"
      itemInfo:
        type: object
        description: "Sample Description"
        items:
          $ref: '#/definitions/ItemId'
      details:
        type: string
        description: "Sample Description"
        example: "Could not update that field"
      exception:
        type: string
        description: "Sample Description"
        example: "DBException + stack trace"
  ItemId:
    type: object
    description: "Sample Description"
    required:
      - id
      - type
    properties:
      id:
        type: string
        description: "Sample Description"
        example: "45"
      type:
        type: string
        description: "playlist"
        example: "5667"
  ResponseMeta:
    type: object
    description: "Sample Description"
    required:
      - code
    properties:
      code:
        type: string
        description: "Sample Description"
        default: Ok
        enum:
        - Ok
        example: Ok
      detail:
        type: string
        description: "Sample Description"
        example: "this is some detail about the error or the success"
      exception:
        type: string
        description: "Sample Description"
        example: "IOException + stack trace"
      errors:
        type: array
        description: "Sample Description"
        items:
          $ref: '#/definitions/Error'
  ActorInfo:
    type: object
    description: "Sample Description"
    required:
      - actorPath
    properties:
      actorPath:
        type: string
        example: "/user/synchronizer"
  DefaultMetaOnlyResponse:
    type: object
    required:
      - meta
    properties:
      meta:
        $ref: '#/definitions/ResponseMeta'

When I run without the parent definitions key, I also receive an error:

$ ./bin/csharp-petstore.sh
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0
[main] INFO io.swagger.parser.Swagger20Parser - reading from /Users/jim/temp/issues/swagger-codegen/5753/swagger.yaml
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -7
	at java.lang.String.substring(String.java:1919)
	at io.swagger.models.refs.GenericRef.computeSimpleRef(GenericRef.java:86)
	at io.swagger.models.refs.GenericRef.<init>(GenericRef.java:29)
	at io.swagger.models.properties.RefProperty.set$ref(RefProperty.java:56)
	at io.swagger.models.properties.RefProperty.<init>(RefProperty.java:18)
	at io.swagger.util.PropertyDeserializer.propertyFromNode(PropertyDeserializer.java:214)
	at io.swagger.util.PropertyDeserializer.propertyFromNode(PropertyDeserializer.java:286)
	at io.swagger.util.PropertyDeserializer.deserialize(PropertyDeserializer.java:137)
	at io.swagger.util.PropertyDeserializer.deserialize(PropertyDeserializer.java:39)
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:3584)
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:3508)
	at io.swagger.parser.util.SwaggerDeserializer.property(SwaggerDeserializer.java:974)
	at io.swagger.parser.util.SwaggerDeserializer.definition(SwaggerDeserializer.java:789)
	at io.swagger.parser.ResolverCache.loadRef(ResolverCache.java:135)
	at io.swagger.parser.processors.ExternalRefProcessor.processRefToExternalDefinition(ExternalRefProcessor.java:34)
	at io.swagger.parser.processors.ExternalRefProcessor.processRefProperty(ExternalRefProcessor.java:119)
	at io.swagger.parser.processors.ExternalRefProcessor.processRefToExternalDefinition(ExternalRefProcessor.java:83)
	at io.swagger.parser.processors.PropertyProcessor.processRefProperty(PropertyProcessor.java:34)
	at io.swagger.parser.processors.PropertyProcessor.processProperty(PropertyProcessor.java:21)
	at io.swagger.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:21)
	at io.swagger.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:45)
	at io.swagger.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:101)
	at io.swagger.parser.SwaggerResolver.resolve(SwaggerResolver.java:50)
	at io.swagger.parser.SwaggerParser.read(SwaggerParser.java:67)
	at io.swagger.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:431)
	at io.swagger.codegen.cmd.Generate.run(Generate.java:239)
	at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:43)

It seems to me like this would be covered by the tests hitting relative-file-references in swagger-parser, although those are json-only and this could be a yaml issue.

You may want to track swagger-api/swagger-parser#342, as it looks like it's missing functionality (or perhaps a bug) in the YAML processing.

[jimschubert]

I should mention, current master was updated today (9 hours ago, issue reported 7 hours ago) to use swagger-parser 1.0.29. I tested this both with 1.0.28 and 1.0.29 and received the same error when there's no parent definitions key in the externally referenced file.

[fehguy]

Could be the case that we need to add a test to the parser for this. It should be handled...

[JFCote]

@jimschubert and @fehguy : I'm pretty sure it should be supported by swagger-codegen because it seems to be supported by the swagger-editor and swagger-ui (and probably by the OpenApi spec itself).

I followed this article on how to split a single yaml file into multiple files and as you can see, there is no more "definitions" key once it is split and it make sense.

https://apihandyman.io/writing-openapi-swagger-specification-tutorial-part-8-splitting-specification-file/

I'm pretty sure more and more people will start to split their files and will follow this blog post and swagger-codegen must be compliant.

Is this an easy fix? I don't know anything about the parser unfortunately so I don't think I can contribute myself.

[jimschubert]

@JFCote I don't think it's a swagger-codegen issue. The third line from the bottom of my stack trace is in CodegenConfigurator, which calls

Swagger swagger = new SwaggerParser().read(inputSpec, authorizationValues, true);

The exception that's thrown is in swagger-core, and looks like it has to do with getting the name of a reference. As @fehguy said, it should be covered.

I'm heading out on vacation today, but I'll try to look later.

[jimschubert]

@fehguy I tracked down the problem. This is an issue with swagger-parser. I'm on vacation for a week, so I was wondering if you could look into a fix?

The issue is that the code is getting into GenericRef#computeSimpleRef with a RefFormat.INTERNAL and RefType.DEFINITION for which the prefix is defined as #/definitions/ by the RefType enum. If an internal type definition is a shorter length than #/definitions/ count, line 82 of GenericRef:

result = ref.substring(prefix.length());

Throws an error. For example, n the sample case of this issue, it's doing "#/Error".substring("#/definitions/".length()).

It's weird to me that swagger-parser loosely allows external refs to be defined as #/Error rather than #/definitions/Error, and after stepping through the code I can understand why I've only ever seen it as #/definitions/Error.

If I rename #/Error in the sample above to #/ErrorErrorError, swagger-codegen generates successfully.

I haven't worked in the swagger-parser codebase, so I wouldn't really know where to start with the fix. Intuitively, I'd think the fix would be to just strictly enforce the prefix. However, I'm not too familiar with the spec on external refs and it looks like the test files aren't prefixed with #/definitions/. Maybe the issue is that RefFormat needs an EXTERNAL option which can be special cased in computeSimpleRef?

[2ricecrackerfolder]

Hi Jim,

I added the "definitions:" in the parent hierarchy as mentioned above. In the UI I was able to display my api, but when I ran the code generator in java, everything ran and the response models were not created. They were referenced in the other models but not created. They would have this name: SnapAdYamldefinitionssnapAd. Is this related? Does the code-gen not support multiple files?

Greg

[jimschubert]

@2ricecrackerfolder the external yaml generated for me in the local filesystem with the examples above. I didn't test it with hosted specs and external definitions Is that how you're generating?

[2ricecrackerfolder]

@jimschubert Thanks for your quick reply! I also modified my configuration to what they have above. The yaml did generate with no errors but was missing the response model files when i dug through the generated src.

[2ricecrackerfolder]

I have a large api with a lot of definitions. I am splitting the files, and it seems like the response models would not generate.

[jimschubert]

@2ricecrackerfolder my assumption would be that you may have a validation error somewhere.

You can check with the online validator by appending your url:
https://online.swagger.io/validator/debug?url=

For example, here's a swagger definition with a validation error: https://online.swagger.io/validator/debug?url=https://quay.io/api/v1/discovery

I don't think the above definition would have a problem when generating client code, but if you have lots of errors it's more likely.

[fehguy]

@jimschubert looks like a bug in swagger-core is triggering this. Bummer, we just did a release today! But please see here:

swagger-api/swagger-core#2255

[fehguy]

Updating to swagger-parser 1.0.30-SNAPSHOT to see if this fixes the issue