Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make moduleObject.mustache confirms to JSONEncodable. #11202

Merged
merged 19 commits into from
Jan 9, 2022

Conversation

0x0c
Copy link
Contributor

@0x0c 0x0c commented Dec 30, 2021

This PR resolves #8925. Classes that is generated from moduleObject.mustache now confirms JSONEncodable to send user-defined model as multipart/form-data. Please let me know if any issue to merge this PR. Thank you :)

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples:
    ./mvnw clean package 
    ./bin/generate-samples.sh
    ./bin/utils/export_docs_generators.sh
    
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    For Windows users, please run the script in Git BASH.
  • File the PR against the correct branch: master (5.3.0), 6.0.x
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.


extension JSONEncodable where Self: Encodable {
func encodeToJSON() -> Any {
let encoder = JSONEncoder()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c here instead of creating a new JSONEncoder, please use CodableHelper.jsonEncoder

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with 1c06b4d

guard let data = try? encoder.encode(self) else {
fatalError("Could not encode to json: \(self)")
}
return data.base64EncodedString(options: Data.Base64EncodingOptions())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c I'm a little worried about this.
When do we want to encode json as base64?
Only in multipart/form-data requests?
Can't we just return the json in Data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@4brunu According to generated code, Data can encode to json as follows.

extension Data: JSONEncodable {
    func encodeToJSON() -> Any {
        return self.base64EncodedString(options: Data.Base64EncodingOptions())
    }
}

This code is implemented here. Therefore, I followed this code and encode json as base 64.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.
Could you please reuse that extension please?
return data.encodeToJSON()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated line 216 so please check 7836e27

@4brunu
Copy link
Contributor

4brunu commented Jan 4, 2022

Hey @0x0c,
thanks for opening this PR,
I left some comments in your PR with some questions.
Thanks

@@ -1,5 +1,5 @@
{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}} {
{{/objcCompatible}}{{#objcCompatible}}@objc {{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} class {{classname}}: NSObject, Codable {
{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}}, JSONEncodable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c to make the CI pass, could you please update this line to be

{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable, JSONEncodable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}}{

Because when using Vapor, we can't make the model conform to JSONEncodable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with aae8e6d

@0x0c
Copy link
Contributor Author

0x0c commented Jan 4, 2022

@4brunu Thank you for your review. I updated codes. Please check it again. Thanks.

@4brunu
Copy link
Contributor

4brunu commented Jan 4, 2022

@0x0c thanks for making the changes, let's wait for CI to see if passes.

@@ -206,3 +206,13 @@ extension Set: RequestDecodable where Element: Content {
extension Set: Content where Element: Content { }

extension AnyCodable: Content {}{{/useVapor}}

extension JSONEncodable where Self: Encodable {
Copy link
Contributor

@4brunu 4brunu Jan 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c could you please move this entire block of code to line 63, so that it's not generated when using vapor, for CI to pass?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with ebd7d01

{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}} {
{{/objcCompatible}}{{#objcCompatible}}@objc {{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} class {{classname}}: NSObject, Codable {
{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable, JSONEncodable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}}{
{{/objcCompatible}}{{#objcCompatible}}@objc {{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} class {{classname}}: NSObject, Codable, JSONEncodable {
{{/objcCompatible}}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please also add the JSONEncodable conformance after Codable in the following files, please?
modelEnum.mustache
modelInlineEnumDeclaration.mustache
modelOneOf.mustache

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with d855a09 and 4929d5b .

@0x0c
Copy link
Contributor Author

0x0c commented Jan 4, 2022

@4brunu I fixed issues which you mentioned. Let's wait the result of CI again.

Comment on lines 78 to 79
}
{{/useVapor}}{{#generateModelAdditionalProperties}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c just a small nit, could you please make this in one line to improve the generated code?

}{{/useVapor}}{{#generateModelAdditionalProperties}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with 8b1529f

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, could you please update the sample projects also?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last commit brought a lot of undesired changes, could you please revert it?
Maybe you need to first merge the master branch into yours and then run the following commands.

./mvnw clean package 
./bin/generate-samples.sh
./bin/utils/export_docs_generators.sh

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with 5a1661b and 51fe33f

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain why some enums are not JSONEncodable?
Is there any issue that you found?

Copy link
Contributor Author

@0x0c 0x0c Jan 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since enum value that confirms RawRepresentable a.k.a primitive value is already confirm JSONEncodable here and here, it seems to that modelEnum.mustache and modelInlineEnumDeclaration.mustache should not confirm JSONCodable. If confirm explicitly, Xcode could not build generated source code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, thanks 👍

@@ -1,4 +1,4 @@
{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} enum {{classname}}: {{dataType}}, {{#useVapor}}Content, Hashable{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}, CaseIterable{{#enumUnknownDefaultCase}}{{#isInteger}}, CaseIterableDefaultsLast{{/isInteger}}{{#isFloat}}, CaseIterableDefaultsLast{{/isFloat}}{{#isDouble}}, CaseIterableDefaultsLast{{/isDouble}}{{#isString}}, CaseIterableDefaultsLast{{/isString}}{{/enumUnknownDefaultCase}} {
{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} enum {{classname}}: {{dataType}}, {{#useVapor}}Content, Hashable{{/useVapor}}{{^useVapor}}Codable{{#isContainer}}{{#isString}}{{^isInteger}}{{^isFloat}}{{^isDouble}}, JSONEncodable{{/isDouble}}{{/isFloat}}{{/isInteger}}{{/isString}}{{/isContainer}}{{/useVapor}}, CaseIterable{{#enumUnknownDefaultCase}}{{#isInteger}}, CaseIterableDefaultsLast{{/isInteger}}{{#isFloat}}, CaseIterableDefaultsLast{{/isFloat}}{{#isDouble}}, CaseIterableDefaultsLast{{/isDouble}}{{#isString}}, CaseIterableDefaultsLast{{/isString}}{{/enumUnknownDefaultCase}} {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c Could you please explain this logic please?

{{#isContainer}}{{#isString}}{{^isInteger}}{{^isFloat}}{{^isDouble}}, JSONEncodable{{/isDouble}}{{/isFloat}}{{/isInteger}}{{/isString}}{{/isContainer}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reason of #11202 (comment).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So an Enum should conform to JSONEncodable if it's a string os is a container? Or if it's a string and is a container?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I copied and pasted modelInlineEnumDeclaration.mustache and it is not appropriate. modelInlineEnumDeclaration.mustache confirms String when it is a container. I fixed with 235b824

@@ -1,5 +1,5 @@
{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}} {
{{/objcCompatible}}{{#objcCompatible}}@objc {{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} class {{classname}}: NSObject, Codable {
{{^objcCompatible}}{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} {{#useClasses}}final class{{/useClasses}}{{^useClasses}}struct{{/useClasses}} {{{classname}}}: {{#useVapor}}Content{{/useVapor}}{{^useVapor}}Codable, JSONEncodable{{/useVapor}}{{#vendorExtensions.x-swift-hashable}}, Hashable{{/vendorExtensions.x-swift-hashable}}{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c another nit that I found is that at the end of the line, there is a space before the final { that was acidently removed and is creating some noise for example in the vapor client that shouldn't have any changes with this PR.
Could you please restore that space, to avoid noise in the generated code?
This command should generate the Swift sample projects.

./bin/generate-samples.sh bin/configs/swift5-*
`
``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with 672df35 and 8962b8b

@@ -1,4 +1,4 @@
{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} enum {{enumName}}: {{^isContainer}}{{dataType}}{{/isContainer}}{{#isContainer}}String{{/isContainer}}, {{#useVapor}}Content, Hashable{{/useVapor}}{{^useVapor}}Codable{{/useVapor}}, CaseIterable{{#enumUnknownDefaultCase}}{{#isInteger}}, CaseIterableDefaultsLast{{/isInteger}}{{#isFloat}}, CaseIterableDefaultsLast{{/isFloat}}{{#isDouble}}, CaseIterableDefaultsLast{{/isDouble}}{{#isString}}, CaseIterableDefaultsLast{{/isString}}{{#isContainer}}, CaseIterableDefaultsLast{{/isContainer}}{{/enumUnknownDefaultCase}} {
{{#nonPublicApi}}internal{{/nonPublicApi}}{{^nonPublicApi}}public{{/nonPublicApi}} enum {{enumName}}: {{^isContainer}}{{dataType}}{{/isContainer}}{{#isContainer}}String{{/isContainer}}, {{#useVapor}}Content, Hashable{{/useVapor}}{{^useVapor}}Codable{{^isContainer}}{{^isString}}{{^isInteger}}{{^isFloat}}{{^isDouble}}, JSONEncodable{{/isDouble}}{{/isFloat}}{{/isInteger}}{{/isString}}{{/isContainer}}{{/useVapor}}, CaseIterable{{#enumUnknownDefaultCase}}{{#isInteger}}, CaseIterableDefaultsLast{{/isInteger}}{{#isFloat}}, CaseIterableDefaultsLast{{/isFloat}}{{#isDouble}}, CaseIterableDefaultsLast{{/isDouble}}{{#isString}}, CaseIterableDefaultsLast{{/isString}}{{#isContainer}}, CaseIterableDefaultsLast{{/isContainer}}{{/enumUnknownDefaultCase}} {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0x0c is it necessary to check is it's not container?
This implementation is different from modelEnum.mustache, is it normal?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's related to #11202 (comment)?
If it's correct being different and you could give a bit of more context, I would appreciate it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If modelEnumDeclaration.mustache is container, the enum confirms String. Therefore, we have to check whether the enum is container or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this doesn't apply also to modelInlineEnumDeclaration.mustache?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry I missed type. In previous my post "If modelEnumDeclaration.mustache is container ..." is not correct.. "If modelInlineEnumDeclaration.mustache is container ..." is correct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good to me.
If you could explain why we need to check for container in one but not the other I would appreciate it, because this was something that was always a bit confusing to me

Copy link
Contributor Author

@0x0c 0x0c Jan 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you could explain why we need to check for container in one but not the other I would appreciate it, because this was something that was always a bit confusing to me

Sorry, I can't answer your question because a code for checking container is already implemented before I modify modelEnumDeclaration.mustache. I respect original mustache and don't change the mustache is better I think.

Originally, according to the first commit of these files, there is difference between modelEnum.mustache and modelInlineEnumDeclaration.mustache. One is checking for container and other is not.
See here and here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks for the explanation 👍

Copy link
Contributor

@4brunu 4brunu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

@0x0c
Copy link
Contributor Author

0x0c commented Jan 9, 2022

Thank you! Can't wait for merging this PR :)

@4brunu
Copy link
Contributor

4brunu commented Jan 9, 2022

Thank you for the patient, making the requested changes and answering my questions 👍

@wing328 wing328 added this to the 5.4.0 milestone Jan 9, 2022
@wing328 wing328 merged commit febf496 into OpenAPITools:master Jan 9, 2022
@honkmaster
Copy link
Contributor

@4brunu @0x0c Hello, I think I found an issue with this PR.

Within request, one may append query items, such as:

var localVariableUrlComponents = URLComponents(string: localVariableURLString)
localVariableUrlComponents?.queryItems = APIHelper.mapValuesToQueryItems([
 "_variable": variable?.encodeToJSON(),
])

If the variable is a string, the new encodeToJSON behaviour will break existing code. Previously, the variable was set as is. Now, the String is encoded as JSON... Is this really intended?

@4brunu
Copy link
Contributor

4brunu commented Feb 7, 2022

That's weird.

extension JSONEncodable {
func encodeToJSON() -> Any { self }
}

Looking at the code, it shouldn't be encoded to json.

@honkmaster
Copy link
Contributor

honkmaster commented Feb 7, 2022

But it does.... :/

If you look at the extension, both apply to a String (as it is Encodable).
I don't know how the compiler decides which extension to take.
From our point of view, we only upgraded from 5.3.1 to 5.4.0 and now all our tests fail.

extension JSONEncodable where Self: Encodable {
    func encodeToJSON() -> Any {
        let encoder = CodableHelper.jsonEncoder
        guard let data = try? encoder.encode(self) else {
            fatalError("Could not encode to json: \(self)")
        }
        return data.encodeToJSON()
    }
}
 extension JSONEncodable { 
     func encodeToJSON() -> Any { self } 
 }

@4brunu
Copy link
Contributor

4brunu commented Feb 7, 2022

@0x0c @honkmaster do you have a solution other than revert this PR? Thanks

@4brunu
Copy link
Contributor

4brunu commented Feb 7, 2022

I think the solution should be something along those lines.
4brunu@4647a79
Basically we want to push the Json encoding as far into the stack as possible.

But I'm getting the following error
protocol 'JSONEncodable' as a type cannot conform to 'Encodable'

4brunu@4647a79#r66188797

Any suggestions?

Or maybe restrict the extension extension JSONEncodable where Self: Encodable so that it does not apply to so many types?

@4brunu
Copy link
Contributor

4brunu commented Feb 7, 2022

I think I found a solution.
@honkmaster can you please test if this PR fixes the issue?
#11541

@rhys-rant
Copy link

But it does.... :/

If you look at the extension, both apply to a String (as it is Encodable). I don't know how the compiler decides which extension to take. From our point of view, we only upgraded from 5.3.1 to 5.4.0 and now all our tests fail.

extension JSONEncodable where Self: Encodable {
    func encodeToJSON() -> Any {
        let encoder = CodableHelper.jsonEncoder
        guard let data = try? encoder.encode(self) else {
            fatalError("Could not encode to json: \(self)")
        }
        return data.encodeToJSON()
    }
}
extension JSONEncodable { 
    func encodeToJSON() -> Any { self } 
}

I have a similar problem on 5.4.0 FWIW.
I'm now finding (after much debugging!) that uploading an image from a local file URL is unfortunately encoding the URL as a String instead of keeping it as a URL. When the FormDataEncoding.encode method is then run, the String representing the URL is then itself converted, rather than the file being read from the file URL it points to. As a result, the server I'm uploading images to is failing, throwing a 500 error, as it (understandably) cannot decode an image from a device-local file URL.

@4brunu
Copy link
Contributor

4brunu commented Mar 7, 2022

Could you please test if the version in master branch works for you?

@rhys-rant
Copy link

Could you please test if the version in master branch works for you?

Yep, will give it a try. Do you know of a quick way to get the master branch installed via Homebrew, or do I have to clone and build myself? 😄

@4brunu
Copy link
Contributor

4brunu commented Mar 7, 2022

You have to clone it and build it yourself.
Clone it, run ./mvnw package to build it and the output jar will be located at modules/openapi-generator-cli/target/openapi-generator-cli.jar.
Then you should copy the jar somewhere you want and the openapi through the jar file.

java -jar openapi-generator-cli.jar generate ...

@denizdogan
Copy link
Contributor

@4brunu For what it's worth, I have the exact same issue as @rhys-rantmedia in 5.4.0. I've tried debugging it, but I really can't figure out where in the whole process the URL gets turned into a local path string.

@4brunu
Copy link
Contributor

4brunu commented May 3, 2022

For now the only solution is to build from source or to use an older version.

@4brunu
Copy link
Contributor

4brunu commented May 3, 2022

@denizdogan could you please confirm if that works for you?

@denizdogan
Copy link
Contributor

@4brunu Yeah, I just resorted to using 5.3.1 again, no issues :) Thanks.

@4brunu
Copy link
Contributor

4brunu commented May 3, 2022

Sorry for the trouble, this will be fixed when the next version is out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[REQ] [Swift5] Encode a user-defined model to Json when using multipart/form-data
6 participants