Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG][DART] Fields named json get shadowed by ToJson and become unserializable due to cyclic pointer reference #12127

Closed
5 of 6 tasks
0xNF opened this issue Apr 13, 2022 · 7 comments · Fixed by #12138
Closed
5 of 6 tasks

Comments

@0xNF
Copy link
Contributor

0xNF commented Apr 13, 2022

Bug Report Checklist

  • Have you provided a full/minimal spec to reproduce the issue?
  • Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
  • [Optional] Sponsorship to speed up the bug fix or feature request (example)
Description

Here's a repo for the issue: https://github.com/0xNF/openapi_gen_dart_json_shadow

The Dart SDK generator will incorrectly shadow field variables named json in the generated toJson() method, resulting in a cyclic pointer loop when running the toJson() method.

openapi-generator version

jar name: openapi-generator-cli-6.0.0-20220412.074015-127.jar
jar version: 6.0.0-SNAPSHOT

OpenAPI declaration file content or url
openapi: 3.0.3
info:
  version: "1.0"
  title: Dart Shadow Json Demo
servers:
  - url: 'localhost'
    variables:
      host:
        default: localhost
components:
  schemas:
    ItemWithFieldNamedJson:
      type: object
      properties:
        json:
          type: object
          additionalProperties: {}
paths:
  /:
    get:
      operationId: shadow
      responses:
        '200':
          description: produces a shadowed json field when generated in dart
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ItemWithFieldNamedJson'
Generation Details

See: https://github.com/0xNF/openapi_gen_dart_json_shadow/blob/master/shadowed/lib/model/item_with_field_named_json.dart

class ItemWithFieldNamedJson {
  ItemWithFieldNamedJson({
    this.json = const {},
  });

  Map<String, Object> json;

  Map<String, dynamic> toJson() {
    final json = <String, dynamic>{};
      json[r'json'] = json;
    return json;
  }
Steps to reproduce
  1. Run the generator command
  • java -jar openapi-generator-cli-6.0.0-20220412.074015-127.jar generate -i .\shadowspec.yaml -g dart -o shadowed
  1. Examine the shadowed/lib/model/item_with_field_named_json.dart
  2. See the shadowed variable on line 34, and the infinite reference on line 35
Related issues/PRs

Use another parameter name to stop variable shadowing. related to #10263

Suggest a fix

The hard fix:

  • For any variables that are generated by the Generator, use static analysis to determine if there are any conflicting variable names in the namespace in the generated functions, and then modify the desired generated variable name appropriately, repeating as many times as necessary to ensure truly unique variable names within the functions scope.

The easier fix:

  • The Dart generator already knows about reserved keywords. For fields that are named int, generated fields are named int_ with an underscore. Extend this idea to json. This is a half measure though, as if I name my field variable json_, then we're back to being shadowed.
@kuhnroyal
Copy link
Contributor

Yea probably can go with _json inside the method. Feel free to create a PR, this should be pretty easy.

Or something like:

  Map<String, dynamic> toJson() {
    return <String, dynamic>{
	  r'json': json,
	};
  }

CC @agilob

0xNF pushed a commit to 0xNF/openapi-generator that referenced this issue Apr 14, 2022
…to avoid shadowing fields named 'json' (OpenAPITools#12127)

* [Dart][client] Updated dart samples
wing328 pushed a commit that referenced this issue Apr 16, 2022
…to avoid shadowing fields named 'json' (#12127) (#12138)

* [Dart][client] Updated dart samples

Co-authored-by: 0xnf <0xnf>
@0xNF
Copy link
Contributor Author

0xNF commented Apr 16, 2022

@kuhnroyal
I found another instance of the same bug -- this time with variables named path where the calls to Future<Response> xWithHttpInfo({ String? path }); have the path variable shadowed by the generated path string:

  Future<Response> getFolderWithHttpInfo({ String? path, }) async {
    final path = r'/some_random_path';
    [...]
}

I can submit a fix for this newly found path bug, but it would probably be better if we could agree to a bigger fix -- for instance, renaming every generated variable to have a leading underscore if its scope comes in contact with any open-api specified variable names.

As an example, in the same withHttpInfo method, I see the following generated variables:

    Object? postBody;

    final queryParams = <QueryParam>[];
    final headerParams = <String, String>{};
    final formParams = <String, String>{};


    const authNames = <String>[];
    const contentTypes = <String>[];

Having these precludes any openapi variables from being defined with these names lest they get shadowed.
Admittedly, adding an underscore to these variables is just kicking the can down the road, but I guess having a leading underscore as a public variable name is quite a bit less likely than not.

@kuhnroyal
Copy link
Contributor

kuhnroyal commented Apr 16, 2022

A lot of these variables can be avoided by nesting, spread operators etc. - that's what I did in the Dio generator. Automatically renaming them is not really feasible, would have to keep additional reserved word lists which will ultimately be out of sync. Not sure what other generators do in these cases.

@0xNF
Copy link
Contributor Author

0xNF commented Apr 16, 2022

Is it acceptable if I rewrite most of these generated methods to use spreads, etc to avoid these naming conflicts?

I don't know how big of a commit the repo maintainers (e.g, you) are willing to accept at once.

@kuhnroyal
Copy link
Contributor

It is ok to rewrite. We can review, it would be great if we could add some tests. It is also good to get this in before 6.0.0 release.

@0xNF
Copy link
Contributor Author

0xNF commented Apr 16, 2022

What's the timeline for the 6.0 release?

@kuhnroyal
Copy link
Contributor

What's the timeline for the 6.0 release?

~3 weeks

rk0n pushed a commit to rk0n/openapi-generator that referenced this issue Apr 24, 2022
…to avoid shadowing fields named 'json' (OpenAPITools#12127) (OpenAPITools#12138)

* [Dart][client] Updated dart samples

Co-authored-by: 0xnf <0xnf>
spacether added a commit that referenced this issue Apr 26, 2022
* Add source folder variable to fastapi fix 12118

* Add generated sample files for python-fastapi fix 12118

* [python-experimental] fixes json + charset use case (#12114)

* Adds code to detect json content type when charset is also set

* Updates template to properly render content type, regenerates samples

* Adds test_json_with_charset

* Reverts version file

* Fixes typo

* Add example allOf with single ref (#10948)

* Add example allOf with single ref

* fix dart-dio-next handling of that case

* Refactor without vendor extension

* Regenerate newer samples

* Add a sample of an enum model array in query params (#12107)

* [typescript-fetch] drop support typescript under v4.0 (#12102)

* [typescript-fetch] drop support typescript under v4.0

* [typescript-fetch] update docs

* [typescript-fetch] update package-lock.json manually & fix test

* [typescript-fetch] fix test

* update samples

* [dart] Remove old dio generator (to be replaced with dart-dio-next) (#12109)

* remove old `dart-dio` generator which will be replaced by `dart-dio-next` in a seperate PR
* remove left-over `DartJaguarClinetCodegen` class which was sunset a while ago and is unused

* fix: respect configured generator URL in swagger config (#12064)

* fix: respect configured generator URL in swagger config

The generated OpenAPI spec does not reflect the GENERATOR_HOST which causes wrong generated code and non-functional snippets in the UI.

This PR improves that by adding the relevant parts to the spec.

* style: use `OpenAPI` instead of `Swagger`

* refactor: make Set creation Java 8 compatible

* fix: add missing import

* [typescript*] drop support typescript below 4.0 (#12123)

* [typescript-axios] drop support typescript below 4.0 & update samples

* [typescript-axios] update package.json & package-lock.json

* [typescript-node] drop support typescript below 4.0 & update samples

* [typescript-nestjs] drop support typescript below 4.0 & update samples

* [typescript-redux-query] drop support typescript below 4.0 & update samples

* [typescript-aurelia] drop support typescript below 4.0 & update samples

* [typescript-jquery] drop support typescript below 4.0 & update samples

* [typescript] drop support typescript below 4.0 & update samples

* Upgrade haskell-servant to latest LTS (#12092)

* [C++][Qt] update petstore to 3.0 spec (#12124)

* test update sampels

* update samples

* add file

* update readme with onesignal (#12126)

* [typescript-fetch] Removed functions that are unused when withoutRuntime is true. (#12101)

* [typescript-fetch] remove unused function when withoutRuntimeCheks option to true

* [typescript-fetch] update samples

* [kotlin][client] fix encoding of individual parts of a multipart request (#11911)

* [kotlin][client] fix encoding (and Content-Type headers) of individual parts of a multipart request

* [kotlin][client] fix incorrect handling of binary downloads

* [kotlin][client] update samples

* [python-experimental] Allow response media types to omit schema (#12135)

* Adds issue spec file and attemts to generate code from it

* Adds missing schema definitions

* Skips fromProperty invocation if the passed in schema is none in getContent

* Makes MediaType.schema optional

* Adds checking that the content type is in self.content

* Sets ApiResponse body type as Unset if there is no schema for it

* Handles schema = None case

* Adds endpoint without response schema

* Reverts version files

* Adds test_response_without_schema

* improve errorObjectType to avoid regression (#12131)

* [php-slim4] Add monolog package as default logger (#12137)

* Add monolog to templates

* Remove default values from DI\get helper

It turned out \DI\get expects only single argument, current method call
doesn't throw any errors but it should be corrected anyway.

* Refresh samples

* Bump async from 2.6.3 to 2.6.4 in /website (#12148)

Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4.
- [Release notes](https://github.com/caolan/async/releases)
- [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md)
- [Commits](caolan/async@v2.6.3...v2.6.4)

---
updated-dependencies:
- dependency-name: async
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* do not put the invalid value of the enum to a JSON structure (#12133)

* [Java] Ignore return value for Java file assert classes (#12145)

* Add @mkusaka to TS technical committee (#12150)

* Fix documentation for users of AbstractJavaJAXRSServerCodegen (#12142)

The default for `sourceFolder` in the documentation was wrong.

* [Dart][client] Adjust toJson method to use '_json' instead of 'json' to avoid shadowing fields named 'json' (#12127) (#12138)

* [Dart][client] Updated dart samples

Co-authored-by: 0xnf <0xnf>

* [dart] Support/Fix sourceFolder parameter and docs (#12113)

* [dart] Support/Fix sourceFolder parameter and docs

* allow passing the sourceFolder to generators
* not sure how this translates to the dart2 generator due to the `part` files
* fix doc generation not showing default value of CLI options

* [dart] Delete old generator doc files

* [Ruby] Add support for faraday 2.x (#12112)

* [ruby] Add faraday 2.x support

* Remove redundant params_encoder config

* Memoize Faraday connection and refactor

* [Java/Spring] all-of and one-of Improvements and Fixes (was #12075) (#12089)

* Fix Bug in OneOfImplementorAdditionalData pulling in wrong vars to one-of-implementors.
Support parentVars in order to support fluent setter with inherited properties.

Squashed commit of the following:

commit f945c943777a1a496d7de8fc0a188842d9efb1ac
Author: Lars Uffmann <lars.uffmann@gmail.com>
Date:   Thu Apr 7 18:22:54 2022 +0200

    Polishing

commit 23ce1d0ff1faff53e85ca4362f33660962aa6a92
Author: Lars Uffmann <lars.uffmann@vitroconnect.de>
Date:   Thu Apr 7 17:15:28 2022 +0200

    Add JavaDoc

commit fee70fde5709afa67f3aabd4f48ba496df63a884
Author: Lars Uffmann <lars.uffmann@vitroconnect.de>
Date:   Thu Apr 7 17:11:17 2022 +0200

    Add imports for inherited Properties

commit 29509aaac51750fbd33c00a57d32cac34cbcbb90
Author: Lars Uffmann <lars.uffmann@vitroconnect.de>
Date:   Thu Apr 7 13:40:36 2022 +0200

    Generate Samples

commit 1d19d5465137d3af712f2fd3b4ae4474c58af15e
Author: Lars Uffmann <lars.uffmann@vitroconnect.de>
Date:   Thu Apr 7 13:21:23 2022 +0200

    SpringCodegen: Support parentVars in order to support fluent setter with inherited properties.

commit 2217a77bb747d0b07ef17407a6b5dd5c624a2551
Author: Lars Uffmann <lars.uffmann@gmail.com>
Date:   Thu Apr 7 07:18:50 2022 +0200

    Add allVars to omit list in OneOfImplementorAdditionalData

commit 90499a3b0a187971bfe25deb6355c3444dcf89a7
Author: Lars Uffmann <lars.uffmann@gmail.com>
Date:   Wed Apr 6 16:40:23 2022 +0200

    Works exactly as needed for oneOf/allOf in Java/Spring

commit b6d496d772e0d0a8d87a3b8cdba8fd3ca4db7f3f
Author: Lars Uffmann <lars.uffmann@gmail.com>
Date:   Wed Apr 6 15:16:27 2022 +0200

    Debug Session: identify critical codep path

commit 85722360038107f15841d5acc448d93dee513a06
Author: Lars Uffmann <lars.uffmann@vitroconnect.de>
Date:   Tue Apr 5 09:56:38 2022 +0200

    Add test case to reproduce issue.

commit 14acc5cd974bb5260f3751015558807e2eb1a8a1
Author: Lars Uffmann <lars.uffmann@gmail.com>
Date:   Tue Apr 5 06:57:30 2022 +0200

    Add config to reproduce the issue

* Adjust indentation.

Co-authored-by: Lars Uffmann <lars@wintermute.local>

* [REQ][Ruby] Ruby Allow Follow Redirect (#12047)

* Add `follow_location` option

Implementation of #10028

* regenerated clients

* set follow_location default to true

* Adds UUID to python-experimental (#12153)

* Adds UUID to python-exp, allows uuid models to be generated

* Adds test_UUIDString uuid model test

* Fixes uuid properties in python-exp, changes maps to object data type, adds uuid data type

* Adds maps data type back in

* Adds missing Null and AnyType definitions and adds them to python-experimental

* Generator docs updated, added missing uuid, null, anytype, and object

* Adds uuid support description

* Docs updated

* update url to travis ci

* Adds not to CodegenComposedSchemas and uses it in python-exp (#12146)

Updates docs

* [python-experimental] fixes bug where some singleton representations raised a RecursionError  (#12157)

* Adds issue components and endpoint

* Regenerates samples

* Fixes singleton repr, removes issue components and endpoint

* Removes unused endpoint

* Reverts file

* Adds tests of enum, boolean, and none representations

* Uses super repr for singletons that ere not none, true, or false

* [swift5] Abstract away URLSession (#11651) (#12110)

* [Java][OkHTTP] fix empty request body handling (#12172)

* Better inline model resolver to handle inline schema in array item (#12104)

* better support of inline schema in array item

* update tests

* update samples

* regenerate samples

* fix allof naming, remove files

* add files

* update samples

* update readme

* fix tests

* update samples

* update samples

* add new files

* update test spec

* add back tests

* remove unused files

* comment out python test

* update js test using own spec

* remove files

* remove unused files

* remove files

* remove unused files

* better handling of allOf with a single type

* comment out go test

* remove test_all_of_with_single_ref_single_ref_type.py

* fix inline resolver, uncomment go test

* [Inline model resolver] minor enhancements/refactoring (#12175)

* better code format

* better code format, minor refactor

* [python-experimental] Fixes enum is comparison (#12176)

* Fixes enum is comparison

* Reverts file

* [php] make ObjectSerializer::toString actually return a string (#12158)

* update php samples

* Allow selection of MP REST API version for MicroProfile REST client g… (#12043)

* Allow selection of MP REST API version for MicroProfile REST client generation

* fix typo in pom.xml

* fix typo in pom.xml, update samples

* add exception when incorrect MP Rest Client version is chosen

* [Java][microprofile] update API test template to work with v3.0 (#12177)

* update microprofile api test to work with 3.0

* minor format change

* update samples

* Return type for Azure funcs (#12115)

* Azure func return type

* Changed to Task<IActionResult<T>

* Readme

* update doc

* Improvements to csharp-netcore-function generator (#12183)

* improvements to csharp-netcore-function generator

* update samples

* update doc, samples

* [java-micronaut] Support Optional for non-required properties (#12144)

The Micronaut generator by default adds the @nullable annotation to
non-required properties and allows using the Jackson JsonNullable
wrapper but it is not possible to use java.util.Optional as a wrapper
for optional properties.

This change adds support for using the Optional wrapper for non-required
properties.

* update java samples

* [typescript-fetch] allow initOverrides with async function (#12098)

* [typescript-fetch] allow initOverrides with async function

* [typescript-fetch] update samples

* [typescript-fetch] refactoring initFnction apply

* [typescript-fetch] update samples

* [typescript-fetch] refactoring create body function

* [typescript-fetch] update samples

* [typescript-fetch] make interface more flexible

* [typescript-fetch] update samples

* [typescript-fetch] support 2.x version of typescript & update samples

* [typescript-axios] update samples

* [typescript-fetch] refactor: add type alias

* [typescript-fetch] override with init params even if initOverrides is function

* [typescript-fetch] update samples

* [Wsdl] Adding cli-option for generating different versions of WSDL-files regarding Media type versioning (content negotiation) (#12206)

* add wsdl version generation

* add option to use specified operationId

* update samples

* update cli description

* [python-flask] Fix return type too strict (#12190)

Flask (and connexion by extension) allows the return type to be either
just the body, or the body & status code, or the body & status code &
headers.

This commit fixes the stated `rtype` to allow the latter two cases.

* update samples, docs

* Fixing bug in Kotlin Client with BigDecimal default value (#12213)

* Fix duplication of "Api" when structPrefix is set (#12128)

This fixes a minor duplication of the word "Api", which is already part
of the classname template parameter and doesn't need to be repeated when
structPrefix is set.

* add samples/client/petstore/kotlin-bigdecimal-default to kotlin ci tests

* remove spring-mvc samples (#12222)

* [Micronaut] Add option to describe response wrappers (#12186)

* Minor refactor for Micronaut generators

* Add support for security roles in micronaut server generator

* Micronaut Server Generator refactor the x-roles String variable

* Add support for Micronaut HttpResponse wrapper

* Generate samples

* Optimize the usage of context-path for Micronaut server

* Emit default values for aspnetcore 3 value types (#11280)

Fixes #10772 for aspnetcore 3+

This allows numbers to be set to zero, and booleans to be set to false.
It may make sense to port this fix to the other C# generators,
though it was partially fixed (for booleans only) in the netcore client in PR9042.

* update samples

Co-authored-by: Justin Black <spacether@users.noreply.github.com>
Co-authored-by: Peter Leibiger <kuhnroyal@gmail.com>
Co-authored-by: mkusaka <hinoshita1992@gmail.com>
Co-authored-by: William Cheng <wing328hk@gmail.com>
Co-authored-by: Florian Greinacher <florian@greinacher.de>
Co-authored-by: Tom Bärwinkel <dev@baerwinkel.org>
Co-authored-by: Anton Koscejev <koscejev@users.noreply.github.com>
Co-authored-by: Yuriy Belenko <yura-bely@mail.ru>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hui Yu <ityuhui@gmail.com>
Co-authored-by: Oleh Kurpiak <oleh.kurpiak@gmail.com>
Co-authored-by: Eric Wolf <1983821+typetetris@users.noreply.github.com>
Co-authored-by: 0xNF <0xNF@users.noreply.github.com>
Co-authored-by: Yohei Kitamura <ykitamura@mdsol.com>
Co-authored-by: cachescrubber <5127753+cachescrubber@users.noreply.github.com>
Co-authored-by: Lars Uffmann <lars@wintermute.local>
Co-authored-by: Connor Moore <cmoore@aurorasolar.com>
Co-authored-by: leszek-s <1277962+leszek-s@users.noreply.github.com>
Co-authored-by: fengelniederhammer <92720311+fengelniederhammer@users.noreply.github.com>
Co-authored-by: Andrii Serkes <74911628+aserkes@users.noreply.github.com>
Co-authored-by: Abrhm7786 <82376602+Abrhm7786@users.noreply.github.com>
Co-authored-by: Auke Schrijnen <auke-@users.noreply.github.com>
Co-authored-by: adessoDpd <83429301+adessoDpd@users.noreply.github.com>
Co-authored-by: Oliver Ford <dev.github@ojford.com>
Co-authored-by: Johan Sjöblom <sjoblomj88@gmail.com>
Co-authored-by: Noah Fontes <noah.fontes@puppet.com>
Co-authored-by: Andriy Dmytruk <80816836+andriy-dmytruk@users.noreply.github.com>
Co-authored-by: matt beary <1661988+hauntingEcho@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants