Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ K6 Generator ] Support for extracting examples defined at parameter level of Swagger/OpenAPI specification, plus minor fixes #9750

Merged
merged 3 commits into from
Aug 22, 2021

Conversation

ideas-into-software
Copy link
Contributor

This addresses: #8378

Overview

If the Swagger/OpenAPI specification used as the input spec contains examples at parameter level, those will be extracted and utilized as parameter values.

The handleParamValue custom Mustache lambda, registered for use in the K6 script.mustache template, handles the conditional checks, formatting, and outputting of parameter values.

If a given parameter has value specified – either in example or examples field, defined at the parameter level – that value will be used.

For list (examples), entire list will be output in the generated script and the first element from that list will be assigned as parameter value.

If a given parameter does not have an example defined, a placeholder value with TODO_EDIT_THE_ prefix will be generated for that parameter, and you will have to assign a value before you can run the script.

In other words, you can now generate K6 test scripts which are ready to run, provided the Swagger/OpenAPI specification used as the input spec contains examples for all of the path/query parameters; see modules/openapi-generator/src/test/resources/3_0/examples.yaml for an example of such specification, and https://swagger.io/docs/specification/adding-examples/ for more information about adding examples.

Thanks to these enhancements, OpenAPI / Swagger spec can be utilized as the single source of truth for automated test generation, including extracting of test data from examples embedded in spec.

Data entered once can then be re-used by OpenAPI code generation tools such as the K6 OpenAPI generator. This allows for full round-trip flow and ensures REST API tests are always up to date.

Generate script based on Swagger/OpenAPI specification with examples

Generate script based on the modules/openapi-generator/src/test/resources/3_0/examples.yaml spec:

$ ./mvnw clean install

$ java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate \
  -i modules/openapi-generator/src/test/resources/3_0/examples.yaml \
  -g k6 \
  -o /var/tmp/k6_examples \
  --skip-validate-spec

Minor fixes

Aside from the above mentioned enhancements, following minor fixes were applied:

  • LoggerFactory.getLogger – logger named after K6ClientCodegen.class, instead of JavascriptClientCodegen.class

  • Inside org.openapitools.codegen.languages.K6ClientCodegen.preprocessOpenAPI(OpenAPI) method, List<CodegenParameter> formParameters instead of List<CodegenParameter> formParameteres

  • modules/openapi-generator/src/test/resources/3_0/examples.yaml – fixed problems which caused errors when using as input spec

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples:
    ./mvnw clean package 
    ./bin/generate-samples.sh
    ./bin/utils/export_docs_generators.sh
    
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    For Windows users, please run the script in Git BASH.
  • File the PR against the correct branch: master, 5.1.x, 6.0.x
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.

@wing328
Copy link
Member

wing328 commented Jun 13, 2021

@ideas-into-software thanks for the PR

cc @mostafa

Copy link
Contributor

@mostafa mostafa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ideas-into-software for the PR and well done! I just left a few comments.

@@ -16,7 +16,7 @@ export default function() {
{{#requestGroups}}
group("{{{groupName}}}", () => {
{{#variables}}
let {{{key}}} = "TODO_EDIT_THE_{{{value}}}";
let {{{key}}} = {{#lambda.handleParamValue}}{{value}}{{/lambda.handleParamValue}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this doesn't need the extra indentation.

@@ -39,7 +39,7 @@ export default function() {
sleep(SLEEP_DURATION);
});
group("/pet/findByStatus", () => {
let status = "TODO_EDIT_THE_STATUS";
let status = 'TODO_EDIT_THE_STATUS'; // specify value as there is no example value for this parameter in OpenAPI spec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extra indentation in the template is reflected here, which I think is unnecessary.

Comment on lines 164 to 165
let password = 'TODO_EDIT_THE_PASSWORD'; // specify value as there is no example value for this parameter in OpenAPI spec
let username = 'TODO_EDIT_THE_USERNAME'; // specify value as there is no example value for this parameter in OpenAPI spec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong, but I expect these values to be filled with examples? Or is it another example script altogether? If so, please regenerate that example alongside this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi

Thank you for the feedback. My IDE does not understand mustache (yet), so apologies for the indentation problems; I’ll fix it ASAP.

Regarding the samples/client/petstore/k6/script.js, please keep in mind samples are generated based on whatever is defined in the config, and for K6 generator, the config file https://github.com/OpenAPITools/openapi-generator/blob/master/bin/configs/other/k6.yaml uses modules/openapi-generator/src/test/resources/2_0/petstore.yaml as the input spec; unfortunately, that OpenAPI spec (modules/openapi-generator/src/test/resources/2_0/petstore.yaml) does not have any examples, and since it’s used by many other generators I did not want to introduce any changes in such place; this is also the reason why in the docs for this PR I provided example (“Generate script based on Swagger/OpenAPI specification with examples”) based on modules/openapi-generator/src/test/resources/3_0/examples.yaml.

Whether modules/openapi-generator/src/test/resources/2_0/petstore.yaml itself should be updated with examples – perhaps that’s a good question to ask the OpenAPI maintainers?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wing328 Any comments?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modules/openapi-generator/src/test/resources/2_0/petstore.yaml

We usually keep it as it's and create a new one based on it with more edge cases per generator.

@wing328
Copy link
Member

wing328 commented Aug 22, 2021

I'll merge this into master and replace all tabs with 4 spaces in the java files.

@wing328 wing328 merged commit 6844737 into OpenAPITools:master Aug 22, 2021
@ideas-into-software
Copy link
Contributor Author

I'll merge this into master and replace all tabs with 4 spaces in the java files.

@wing328 Hi. I already made the requested changes (i.e. formatting) ~ 2 months ago; please see commit from June 17: c32657a

@wing328
Copy link
Member

wing328 commented Aug 22, 2021

Please have a look at f7bc2aa that I made in the master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants