Scanner SubTypesScanner was not configured

[clemi2408]

Hey,
first of all your jsonschema-generator is really great thing!
I love it!

I am jusing maven to generate json schemas out of my Pojo classes.
this works like a charm when using it like this:

<classNames>
    <className>model.entity.user.User</className>
    <className>model.entity.group.Group</className>
</classNames>

if i want to use it like this:

<packageNames>
    <packageName>mode</packageName>
</packageNames>

i get an exception from

org.reflections

Caused by: org.reflections.ReflectionsException: Scanner SubTypesScanner was not configured

Can you explain how to configure the SubTypesScanner via maven?
Regards and Dankeschön
Clem

[JanLabrie]

Hi Clem,

I will try to reproduce your problem and come up with a solution.

Regards,

Jan

[JanLabrie]

Hi Clem,

I have found and solved the issue. It was related to the setting of the proper classpath.

I will make a branch with the fix and discuss with Carsten about releasing.

Regards,

Jan

[clemi2408]

Hey coool!
Thanks for that quick support!
Hope the branch is merged soon and the plugin available on Central.

Another thing/feature: Have you thought about include- and exclude (Regex/glob) pattern for further filtering?
Regards and thank you
Clem

[CarstenWickner]

1. @JanLabrie thanks for taking this on. I wouldn't have had time for it.
2. @clemi2408 Jan's fix has been merged v4.12.1 has been released. It may take a few hours until it has been propagated through the various Maven repositories.
3. We're open to additional ideas. Feel free to create a PR for Jan and me to take a look.

[clemi2408]

Issue still not fixed :(

Failed to execute goal com.github.victools:jsonschema-maven-plugin:4.12.1:generate (default) on project schemafuck: Execution default of goal com.github.victools:jsonschema-maven-plugin:4.12.1:generate failed: Scanner SubTypesScanner was not configured

[JanLabrie]

Mmmmm...not good. It worked for me. :-)

I will investigate and get back to you.

[JanLabrie]

Clem,

I retried with the official release 4.12.1 and in my project I am able to generate schema's for a package.

Would it possible to share your setup, so I can examine it?

Jan

[clemi2408]

Hey Jan,
sorry for the late response - had to switch out the device over the weekend :-/
Here is the part of my maven pom:

            <plugin>
                <groupId>com.github.victools</groupId>
                <artifactId>jsonschema-maven-plugin</artifactId>
                <version>${victool-json-schema-generator-maven-plugin.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <!--     <packageNames>
                             <packageName>model.dto</packageName>
                         </packageNames>-->
                    <classNames>
                        <className>model.entity.group.Group</className>
                        <className>model.entity.user.User</className>
                    </classNames>
                    <schemaVersion>DRAFT_2019_09</schemaVersion>
                    <schemaFileName>{0}.json</schemaFileName>
                    <schemaFilePath>${schema.json.folder}</schemaFilePath>
                    <options>
                        <enabled>
                            <option>DEFINITIONS_FOR_ALL_OBJECTS</option>
                            <option>NULLABLE_FIELDS_BY_DEFAULT</option>
                            <option>FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT</option>
                            <option>ADDITIONAL_FIXED_TYPES</option>
                            <option>DEFINITIONS_FOR_ALL_OBJECTS</option>
                            <option>PLAIN_DEFINITION_KEYS</option>
                            <option>ALLOF_CLEANUP_AT_THE_END</option>
                        </enabled>
                    </options>
                    <modules>
                        <module>
                            <name>Jackson</name>
                            <options>
                                <option>FLATTENED_ENUMS_FROM_JSONVALUE</option>
                            </options>
                        </module>
                        <module>
                            <name>JavaxValidation</name>
                        </module>
                        <module>
                            <name>Swagger15</name>
                        </module>
                    </modules>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>javax.validation</groupId>
                        <artifactId>validation-api</artifactId>
                        <version>${javax-validation.version}</version>
                    </dependency>
                    <dependency>
                        <groupId>io.swagger</groupId>
                        <artifactId>swagger-annotations</artifactId>
                        <version>${swagger.annotations.version}</version>
                    </dependency>
                </dependencies>
            </plugin>

U need something else?
Regards
Clem

[JanLabrie]

Thanks for the content. I will have a look at it.

[JanLabrie]

Hi Clem,

I discovered that the Reflections library gives the error you mentioned in case it can not find any classes on the specified path.

Can it be that you do not have any class files in the model.dto package?

It is difficult to understand the error message. I will investigate whether it can be made friendlier.

Regards,

Jan

[clemi2408]

There are files in that path.
But these files are in the target folder because they are generated out of xsds. If i reference them directly by class name they are resolvable. But maybe there is an issue in class path scanning in relation to the target folder?

[clemi2408]

How can i support you?

[CarstenWickner]

This might just be a timing issue then.
Can you check in what Maven build phase the code generation occurs?
If that happens only after the schema generation, you know why it’s not finding anything.

[clemi2408]

Might be possible but to be honest i dont believe in that assumption.
This is based on the fact that its working using this:

<classNames>
  <className>model.entity.user.User</className>
</classNames>

[CarstenWickner]

Fair enough. Might still be a classloader issue when we look-up packages, that it somehow ignores generated class files.

@clemi2408 if you have the capacity, it'd be great to have a minimal sample project somewhere to allow us to reproduce this locally while playing around with potential solutions.

[JanLabrie]

A reproduction set will definitly help. The classloader that is used to load individual classes is also used for loading packages. So we should investigate further to find the difference. Besides the reproduction set it would be good if you could try to use the class mechanism to one of your classes in model.dto i.e: <classNames> <className>model.dto.OneOfYourClasses</className> </classNames> This way we know whether it is a generic classloader thing, i.e. model.dto is not in the classpath, or that it is specifically related to the package loading. Regards, Jan Op vr 5 jun. 2020 16:02 schreef Carsten Wickner <notifications@github.com>:

…

Fair enough. Might still be a classloader issue when we look-up packages, that it somehow ignores generated class files. @clemi2408 <https://github.com/clemi2408> if you have the capacity, it'd be great to have a minimal sample project somewhere to allow us to reproduce this locally while playing around with potential solutions. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#102 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPRADAOC4TYBC4LK5JGBIDRVD3GJANCNFSM4NNEICFQ> . [image: image.gif]

[clemi2408]

Hi guys,
have created github repo and invited you two.
Dont be scared, i try to do some stunts with that project:

Input:

1. XSD
2. OpenApi def (without datatypes)

Output:

1. Json Schema
2. Complete OpenAPI file

Its WIP
regards
Clem

[JanLabrie]

Hi Clem,

Thank you for providing the reproduction repo.
It indeed builds successfully by default, and when I enable the <packageNames> it starts failing.

I am looking for the model.dto package that you mention in the pom. But I can not find it in the target folder, and neither are there any class files for the model.dto package.

In the commented java code you write:
import model.dto.UserDTO
Which can not be resolved.

Can you please clarify where the model.dto classes are?

Thank you!

Jan

[clemi2408]

hey, look, if i execute mvn compile they get stored here:

[JanLabrie]

Aha...When I remove the <packageNames> element, the model.dto classes are generated.

This means it is an ordering issue. The json schema generator is invoked before the model.dto classes are generated.

This is where we must look for the solution. Either generate the model.dto classes earlier, or the schema later.

Jan

[clemi2408]

i didnt get your point to be honest.
if i use
<classNames> <className>model.dto.OneOfYourClasses</className> </classNames>

its working, so it cant be a matter of order...

[JanLabrie]

Hi,

It does not for me when you run mvn clean first:

I'll drop some extra info in a sec.

Regards,

Jan

[JanLabrie]

Hi Clem!

1. Without referring model.dto this is the OK result.
   
   GroupDTO is generated.

2. When enabling schema generation for model.dto, the result is like this:
   

Generation fails and there are no classes generated in the target/generated-sources/openapi/schema/model/dto folder.

The schemas that are generated for User and Group come from target/classes.

For me this can be explained when the model.dto package is generated after the json-schema generation.

Regards,

Jan

[clemi2408]

yep exactly if you refer to the classes directly (User and Group) they get generated.

Can you explain why this is working when using
<classNames> <className>model.dto.OneOfYourClasses</className> </classNames>
and it fetches classes from target/classes

and when using
<packageNames> its not picking the classes from target/classes?

i think changing to does not change anything in the overall maven build order... or?

[JanLabrie]

Clem,

I am out of time now. Will investigate further at a later time.

Bye for now,

Jan

[CarstenWickner]

I had a brief look now as well and there seem to be at least two plugins configured to the compile phase that have circular dependencies to one another:

• jsonschema-maven-plugin (produces schemas under resources/schema/generated/json from classes under model.dto)
• openapi-generator-maven-plugin (produces classes in model.dto and reads schemas from resources/schema/generated/json)

That means, the jsonschema-maven-plugin needs to be executed twice, which can be done via separate entries under <executions> with the second one assigned to a later phase like this:

<plugin>
    <groupId>com.github.victools</groupId>
    <artifactId>jsonschema-maven-plugin</artifactId>
    <version>${victool-json-schema-generator-maven-plugin.version}</version>
    <executions>
        <execution>
            <id>schemas-for-existing-sources</id>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <classNames>
                    <className>model.entity.group.Group</className>
                    <className>model.entity.user.User</className>
                </classNames>
            </configuration>
        </execution>
        <execution>
            <id>schemas-for-generated-sources</id>
            <phase>process-classes</phase>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <packageNames>
                    <packageName>model.dto</packageName>
                </packageNames>
            </configuration>
        </execution>
    </executions>
    <configuration>
        <!-- common configurations for all executions go here -->
    </configuration>
</plugin>

With this setup, I can see that the model.dto classes have been generated before the second execution of the jsonschema-maven-plugin is being triggered, but they are still not being picked up correctly.

My assumption right now is, that the output folder for the model.dto classes is wrong. They declare the correct package but are in the schema/model/dto folder. That mismatch may prevent them being recognised.

You might have to explicitly add the appropriate folder as source folder via yet another plugin as per this StackOverflow answer. Though the following did not do the trick for me (the source folder was added, but the classes weren't picked up). Just something to further investigate I guess:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>3.1.0</version>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>add-source</goal>
            </goals>
            <configuration>
                <sources>
                    <source>${project.build.directory}/generated-sources/openapi/schema</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

[clemi2408]

Thanks for your quick response and the detailed explanation.

But the solution works like this

1. generate pojos with annotations from xsd
2. generate json schema out of pojos
3. assemble openapi.json with generated schema in 2)
4. generate pojos and controller delegates for spring out of assembled openapi.json.

Its planned to have two inputs:
a) xsd files
b) openapi file with empty ref to the bodies

Solution should genereate: xsd—>pojo —> schema ( as there isnt a proper way to create xsd to json schema without intermediate pojos), delete all the intermediate classes (currently they are called entity) and after that generate from json schema and via the openapi file the end result, the spring delegates and model classes.

Yes i know the example does it slightly different, but it was ment to show the error i am faced!

Thanks for your great help and wish you a nice weekend!
Regards
Clem

[CarstenWickner]

I don’t fully follow, but it sounds like nothing needs to be changed on this side of the fence then. 😉

[clemi2408]

Ya the key goal is to have full open api spec to generate models, spring Delegates and so on.

Openapi specs consists out of the openapi stuff and refers to json schema payloads for requests and responses.

So my input in the system is an open api spec without models (only with referenced) as the first half. The second half is the json schema part.

As json schema to pojo generators do not support markup like annox is doing (lombok, hibernate stuff),so i am doing the stunt to define xsd for each model class, with annotations and xml restrictions in place and generate java classes with their annotations. These classes i convert to json schema (with all the validation stuff your cool plugin supports) and use this schema in the open api spec..

I know this is a huge stunt but just reflecting the fact that the xsd to json schema conversion plugins are not cool, due to the fact that json schema has Afaik nothing like annox to enrich pojos....

I know this is not nice at all but leads to a code gen approach that goes a little more into the contract and data governance way...

If you have ideas, to improve or to reduce the huge stunt, lets have a call :-)

[clemi2408]

... but still, why is the package scanner not working? Can you provide a readme.md example how this can work?

[CarstenWickner]

In regards to generating an OpenAPI specification, there is already the option via SchemaGenerator.buildMultipleSchemaDefinitions() to collect the definitions for multiple schemas (e.g. payloads and parameters):

• You'd first call SchemaBuilder.createSchemaReference() for each schema node within your OpenAPI specification to produce the respective ObjectNode for each (assuming you use Jackson to build the OpenAPI specification.
• In the end, call SchemaBuilder.collectDefinitions("components/schemas") to create the schemas node under components in your OpenAPI specification – with the correct "$ref" values everywhere (instead of the "#/definitions/ prefix).

Since you're using the Maven Plugin, the above is not available to you.

---

As to your sample project, I'm afraid I don't have the time to dive deeper into the investigation. And I cannot tell whether @JanLabrie has still some capacity.
I'm happy to fix any particular issue that can be identified – but as-is I don't know what needs to be done/fixed. Consequently, I wouldn't know what to document. I'm open for suggestions.

[clemi2408]

Thanks for the clarification. Maybe i change the generation to a console application.
The example could cover how to do maven json schema generation out of a package.

[CarstenWickner]

Well, the Maven Plugin can generate JSON Schemas out of a package in the way you did it – that is already documented in its README.

The challenge you are encountering here is related to the fact that the classes in the targeted package are being generated just before the schema generation. I still believe they are simply not being added correctly to the plugin's classpath then.

[JanLabrie]

There is an issue. The following proves it:

The following configuration will generate the Group schema directly as class and from the package:

                        <packageName>model.entity.group</packageName>
                    </packageNames>
                    <classNames>
                        <className>model.entity.group.Group</className>
                        <className>model.entity.user.User</className>
                    </classNames>

The output is as follows:

[INFO] --- jsonschema-maven-plugin:4.12.1:generate (default) @ schema ---
[INFO] Generating JSON Schema for class model.entity.group.Group
[INFO] - Writing schema to file: C:\Users\jlabrie\IdeaProjects\schema_test\src\main\resources\schema\generated\json\Group.json
[INFO] Generating JSON Schema for class model.entity.user.User
[INFO] - Writing schema to file: C:\Users\jlabrie\IdeaProjects\schema_test\src\main\resources\schema\generated\json\User.json
[INFO] Generating JSON Schema for package model.entity.group
[INFO] - Writing schema to file: C:\Users\jlabrie\IdeaProjects\schema_test\src\main\resources\schema\generated\json\package-info.json
[INFO] - Writing schema to file: C:\Users\jlabrie\IdeaProjects\schema_test\src\main\resources\schema\generated\json\ObjectFactory.json

Observation:

1. Group.json is generated when from the <className> entry
2. Group.json is not generated from the <packageName> entry
3. For the two other classes in the package schemas are generated

Conclusion:
The classpath is correct, but there is a reason why the Group class is filtered out of the package content.
I will investigate.

[JanLabrie]

It is a weird problem. Feels like a bug in reflections. But we'll have to see.

When you would ask Reflections for the content of the model.entity.group package, it will say:
[model.entity.group.package-info, model.entity.group.ObjectFactory]

When you would ask for the content of the parent package model.entity, it will say:
[model.entity.user.package-info, model.entity.group.package-info, model.entity.basic.BasicEntity, model.entity.group.ObjectFactory, model.entity.user.User$Groups, model.entity.user.User, model.entity.basic.ObjectFactory, model.entity.user.ObjectFactory, model.entity.basic.package-info, model.entity.group.Group]

The latter one does have model.entity.group.Group, where the first has not.

My next step would be to debug into Reflections. But need to find some time.

Any other suggestions?

Jan

[JanLabrie]

Pffff...I found the problem. In case a class has a baseclass that is outside the scanned area, Reflections will not take it into account. The Group class extends BasicEntity, which is not in the model.entity,Group package. It is therefore excluded.

So...that is the problem. I will sleep over the solution. :-)

[JanLabrie]

Ahh...why sleep?. I made a fix, tested it on the Clem's project and it works.

The solution is not elegant, but it works. We'll have to discuss something better. But in the mean time I will make a PR so we have the issue fixed. Tomorrow. Bye.

Jan

[CarstenWickner]

Hi @clemi2408,

Release v4.12.2 contains the latest fix provided by @JanLabrie.

[fanf]

Pffff...I found the problem. In case a class has a baseclass that is outside the scanned area, Reflections will not take it into account. The Group class extends BasicEntity, which is not in the model.entity,Group package. It is therefore excluded.

So...that is the problem. I will sleep over the solution. :-)

Hey, just a little message to say a BIG THANK YOU. I started to had that problem on totally unrelated lib upate (not in the scanned path, etc) and I think I would have never found the problem without that thread and that message in particular.

So: thank you, you saved my day :)