Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GO] Problem with generating go client code, too many structs re-declared #741

Open
wari opened this issue Aug 6, 2018 · 21 comments
Open

Comments

@wari
Copy link

wari commented Aug 6, 2018

Description

I'm unable to run the following:

java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go
go get -v ./...                                                                                                                                                                                    
github.com/SolaceDev/semp-v2-go
# github.com/SolaceDev/semp-v2-go
./api_msg_vpn.go:149:6: CreateMsgVpnAclProfileOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:41:6
./api_msg_vpn.go:260:6: CreateMsgVpnAclProfileClientConnectExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:152:6
./api_msg_vpn.go:372:6: CreateMsgVpnAclProfilePublishExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:264:6
./api_msg_vpn.go:484:6: CreateMsgVpnAclProfileSubscribeExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:376:6
./api_msg_vpn.go:595:6: CreateMsgVpnAuthorizationGroupOpts redeclared in this block
        previous declaration at ./api_authorization_group.go:41:6
./api_msg_vpn.go:705:6: CreateMsgVpnBridgeOpts redeclared in this block
        previous declaration at ./api_bridge.go:41:6
./api_msg_vpn.go:817:6: CreateMsgVpnBridgeRemoteMsgVpnOpts redeclared in this block
        previous declaration at ./api_bridge.go:153:6
./api_msg_vpn.go:931:6: CreateMsgVpnBridgeRemoteSubscriptionOpts redeclared in this block
        previous declaration at ./api_bridge.go:267:6
./api_msg_vpn.go:1045:6: CreateMsgVpnBridgeTlsTrustedCommonNameOpts redeclared in this block
        previous declaration at ./api_bridge.go:381:6
./api_msg_vpn.go:1157:6: CreateMsgVpnClientProfileOpts redeclared in this block
        previous declaration at ./api_client_profile.go:41:6
./api_msg_vpn.go:1157:6: too many errors
openapi-generator version

openapi-generator-cli-4.0.0-20180728.030217-16.jar including earlier release versions (and swagger 2.4.0 as well)

OpenAPI declaration file content or url

https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON

Command line used for generation
java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go
Steps to reproduce
java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go
go get -v ./...
Related issues/PRs

Might be similar: #535

Suggest a fix/enhancement

None I can think of at the moment

@tomi77
Copy link
Contributor

tomi77 commented Aug 6, 2018

It's probably problem with tags. Creates the same code for acl_profile and ms_vpn tags.

@wari
Copy link
Author

wari commented Aug 17, 2018

A lot of the Opts struct are redeclared and it's everywhere, not just in acl_profile and msg_vpn:

$ grep -h "type .* struct" * | grep Opts  | sort | uniq -c | clipcopy
      2 type CreateMsgVpnAclProfileClientConnectExceptionOpts struct {
      2 type CreateMsgVpnAclProfileOpts struct {
      2 type CreateMsgVpnAclProfilePublishExceptionOpts struct {
      2 type CreateMsgVpnAclProfileSubscribeExceptionOpts struct {
      2 type CreateMsgVpnAuthorizationGroupOpts struct {
      2 type CreateMsgVpnBridgeOpts struct {
      2 type CreateMsgVpnBridgeRemoteMsgVpnOpts struct {
      2 type CreateMsgVpnBridgeRemoteSubscriptionOpts struct {
      2 type CreateMsgVpnBridgeTlsTrustedCommonNameOpts struct {
------ Lots of lines removed here ------
      2 type UpdateMsgVpnReplicatedTopicOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointQueueBindingOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointRestConsumerOpts struct {
      2 type UpdateMsgVpnTopicEndpointOpts struct {
$ grep -h "type .* struct" * | sort | uniq -c | grep " 2 " | wc -l        
106

The thought of manually removing those is a headache!

@tomi77
Copy link
Contributor

tomi77 commented Aug 17, 2018

Workaround is commenting / removing tags. But this is not a solution…

@wing328
Copy link
Member

wing328 commented Aug 19, 2018

What about an option to only generate code for the 1st tag (assuming there're multiple tags defined in the operation)?

@wing328
Copy link
Member

wing328 commented Aug 19, 2018

@wari can you please also try another client generator to see if you experience similar issues?

@wari
Copy link
Author

wari commented Aug 23, 2018

Hi @wing328 what is the option to generate the first tag? I've tested this with Swagger (released and unreleased) and I got the same problem as well. I've not reported to swagger yet though. Are there any other client generators I can try? I only know of openapi and swagger

@wing328
Copy link
Member

wing328 commented Aug 23, 2018

Sorry for client generator, I mean Python, Ruby, PHP client generators instead of the Go client generator.

If you manually clean up the tags to keep the first one only, does it solve your problem?

@wari
Copy link
Author

wari commented Aug 23, 2018

  • Python - Running python setup.py test has no failures
  • Ruby - rake spec 1056 examples, 0 failures
  • Bash - I've tested with this one before without failures.
  • In the Java generated code I can see duplicates, but since they are in their own class, this is not a problem, only that the IDE complains. Anyway, gradle test failed when I removed the @ignore lines on the test files. Not sure if that helps.

As for removing the duplicates manually, it should work, as the options contain about 1 or 4 options of similar types, for example:

grep type\ GetMsgVpnTopicEndpointsOpts\  struct * -A 5
api_msg_vpn.go:type GetMsgVpnTopicEndpointsOpts struct {
api_msg_vpn.go-    Count optional.Int32
api_msg_vpn.go-    Cursor optional.String
api_msg_vpn.go-    Where optional.Interface
api_msg_vpn.go-    Select_ optional.Interface
api_msg_vpn.go-}
--
api_topic_endpoint.go:type GetMsgVpnTopicEndpointsOpts struct {
api_topic_endpoint.go-    Count optional.Int32
api_topic_endpoint.go-    Cursor optional.String
api_topic_endpoint.go-    Where optional.Interface
api_topic_endpoint.go-    Select_ optional.Interface
api_topic_endpoint.go-}

@wari
Copy link
Author

wari commented Sep 18, 2018

Just wondering if this will get fixed or I will have to remove the duplicates manually whenever I generate code from this particular API.

@wing328
Copy link
Member

wing328 commented Sep 18, 2018

@wari I'll take another look later today

@ity42
Copy link

ity42 commented Oct 6, 2018

Same symptoms here with another 3.0 spec file.
Further example needed for analysis ?
(sorry, I am a beginner to openapi-generator only able to provide another example, not any further insights)

@wari
Copy link
Author

wari commented Oct 8, 2018

Thanks, @ity42 for reporting, I know I'm not the only here with this issue. Do you have the URL of the YAML or JSON file?

@tomi77
Copy link
Contributor

tomi77 commented Oct 8, 2018

Whats about this: Create *Opts structs in model_*_opts.go files. for example:

  • GetAboutUserOpts move to model_about_user_opts.go
  • GetAboutUserMsgVpnOpts move to model_about_user_msg_vpn_opts.go
  • GetAboutUserMsgVpnsOpts move to model_about_user_msg_vpns_opts.go
    and so on…

@ity42
Copy link

ity42 commented Oct 8, 2018

@wari: sorry, indirect link only; another example is:
psd2-api 1.2 Update 2018-08-17.yaml on https://www.berlin-group.org/nextgenpsd2-downloads
(I just tried it with openapi-generator-cli-3.3.1-20181008.101417-13.jar, still same symptoms)

@akutz
Copy link

akutz commented Dec 30, 2019

I'm still hitting this issue with the Golang generator for the HAProxy dataplane API. You can download their OpenAPI spec file from the top of the linked page (a direct link isn't working).

Total

$ grep -h "type .* struct" *.go | grep Opts | sort | uniq -c | awk '{s+=$1} END {print s}'
255

Details

$ grep -h "type .* struct" *.go | grep Opts | sort | uniq -c | grep -v '^[[:space:]]\{0,\}1'
      4 type CreateAclOpts struct {
      2 type CreateBackendOpts struct {
      3 type CreateBackendSwitchingRuleOpts struct {
      3 type CreateBindOpts struct {
      4 type CreateFilterOpts struct {
      2 type CreateFrontendOpts struct {
      4 type CreateHTTPRequestRuleOpts struct {
      4 type CreateHTTPResponseRuleOpts struct {
      4 type CreateLogTargetOpts struct {
      3 type CreateServerOpts struct {
      3 type CreateServerSwitchingRuleOpts struct {
      3 type CreateStickRuleOpts struct {
      4 type CreateTCPRequestRuleOpts struct {
      4 type CreateTCPResponseRuleOpts struct {
      4 type DeleteAclOpts struct {
      2 type DeleteBackendOpts struct {
      3 type DeleteBackendSwitchingRuleOpts struct {
      3 type DeleteBindOpts struct {
      4 type DeleteFilterOpts struct {
      2 type DeleteFrontendOpts struct {
      4 type DeleteHTTPRequestRuleOpts struct {
      4 type DeleteHTTPResponseRuleOpts struct {
      4 type DeleteLogTargetOpts struct {
      3 type DeleteServerOpts struct {
      3 type DeleteServerSwitchingRuleOpts struct {
      3 type DeleteStickRuleOpts struct {
      4 type DeleteTCPRequestRuleOpts struct {
      4 type DeleteTCPResponseRuleOpts struct {
      4 type GetAclOpts struct {
      4 type GetAclsOpts struct {
      2 type GetBackendOpts struct {
      3 type GetBackendSwitchingRuleOpts struct {
      3 type GetBackendSwitchingRulesOpts struct {
      2 type GetBackendsOpts struct {
      3 type GetBindOpts struct {
      3 type GetBindsOpts struct {
      2 type GetDefaultsOpts struct {
      4 type GetFilterOpts struct {
      4 type GetFiltersOpts struct {
      2 type GetFrontendOpts struct {
      2 type GetFrontendsOpts struct {
      2 type GetGlobalOpts struct {
      2 type GetHAProxyConfigurationOpts struct {
      4 type GetHTTPRequestRuleOpts struct {
      4 type GetHTTPRequestRulesOpts struct {
      4 type GetHTTPResponseRuleOpts struct {
      4 type GetHTTPResponseRulesOpts struct {
      4 type GetLogTargetOpts struct {
      4 type GetLogTargetsOpts struct {
      3 type GetServerOpts struct {
      3 type GetServerSwitchingRuleOpts struct {
      3 type GetServerSwitchingRulesOpts struct {
      3 type GetServersOpts struct {
      3 type GetStickRuleOpts struct {
      3 type GetStickRulesOpts struct {
      4 type GetTCPRequestRuleOpts struct {
      4 type GetTCPRequestRulesOpts struct {
      4 type GetTCPResponseRuleOpts struct {
      4 type GetTCPResponseRulesOpts struct {
      2 type PostHAProxyConfigurationOpts struct {
      4 type ReplaceAclOpts struct {
      2 type ReplaceBackendOpts struct {
      3 type ReplaceBackendSwitchingRuleOpts struct {
      3 type ReplaceBindOpts struct {
      2 type ReplaceDefaultsOpts struct {
      4 type ReplaceFilterOpts struct {
      2 type ReplaceFrontendOpts struct {
      2 type ReplaceGlobalOpts struct {
      4 type ReplaceHTTPRequestRuleOpts struct {
      4 type ReplaceHTTPResponseRuleOpts struct {
      4 type ReplaceLogTargetOpts struct {
      3 type ReplaceServerOpts struct {
      3 type ReplaceServerSwitchingRuleOpts struct {
      3 type ReplaceStickRuleOpts struct {
      4 type ReplaceTCPRequestRuleOpts struct {
      4 type ReplaceTCPResponseRuleOpts struct {

@akutz
Copy link

akutz commented Dec 30, 2019

FWIW, I wrote a quick Makefile target to remove the duplicate options and fix any vestigial imports:

remove-dupe-opts: ## Removes duplicate Opt structs from the generated code
	@for struct in $$(grep -h 'type .\{1,\} struct' $(OUTPUT_DIR)/*.go | grep Opts  | sort | uniq -c | grep -v '^      1' | awk '{print $$3}'); do \
	  for f in $$(/bin/ls $(OUTPUT_DIR)/*.go); do \
	    if grep -qF "type $${struct} struct" "$${f}"; then \
	      if eval "test -z \$${$${struct}}"; then \
	        echo "skipping first appearance of $${struct} in file $${f}"; \
	        eval "export $${struct}=1"; \
	      else \
	        echo "removing dupe $${struct} from file $${f}"; \
	        tr '\n' '\r' <"$${f}" | sed 's~// '"$${struct}"'.\{1,\}type '"$${struct}"' struct {[^}]\{1,\}}~~' | tr '\r' '\n' >"$${f}.tmp"; \
	        mv -f "$${f}.tmp" "$${f}"; \
	      fi; \
	    fi \
	  done \
	done
	@$(GOLANGCI_LINT) run -v --no-config --fast=false --fix --disable-all --enable goimports $(OUTPUT_DIR)

@displague
Copy link

@wing328 In my case, I deleted the extra tags and that fixed the problem:

equinix-labs/metal-go#31 (equinix-labs/metal-go@86e18b7)

@displague
Copy link

displague commented Jul 6, 2021

The following jq command will remove any extra tags from a Swagger document (keeping the first tag):

cat swagger-v2.json  | jq '. | select(((.paths[][].tags| type=="array"), length) > 1).paths[][].tags |= [.[0]]'

Ideally, I would like to preserve the original tag list as x-tags. Any jq help with that would be appreciated!

@displague
Copy link

displague commented Jul 6, 2021

What about an option to only generate code for the 1st tag (assuming there're multiple tags defined in the operation)?

@wing328 I think an option to enable this behavior would fix the problem. I don't know if this tool should be so opinionated that it always chooses the first tag. Perhaps an optional x-openapitools-tag could offer the preferred tag? If that is not set, then just use the first tag because a client that works is better than one that does not.

@iafilius
Copy link

iafilius commented Aug 2, 2021

Hi, a me too report and some additional info.

Using mvn version openapi-generator-cli-5.2.0.jar and latest build openapi-generator-cli-5.2.1-20210719.023544-9.jar and have same (and more ) issues.

4.2.2 works BUT to prevent duplicate structs need to use -p enumClassPrefix=true

5.2.0 or 20210719.023544-9 versions seem to ignore it or some other mechanism generates lots of duplicates.

Can't remember i've seen it with an early 5.0.1 version, but that was almost a year ago when tested.

[vendor provided swagger file used]
(https://vdc-download.vmware.com/vmwb-repository/dcr-public/d8e1559c-3fd9-4d48-a895-b1036479cea6/9e2aee74-1a4d-46ba-ac14-8694815b8ce2/swagger-4.4.0-dist.json)

java  -jar ../openapi-generator-cli-5.2.1-20210719.023544-9.jar generate --generate-alias-as-model -i ./swagger440_untagged.json -g go -o velocloudsdkgo440openapi521unt \
      --additional-properties=packageName="velocloudsdkgo440openapi521unt",packageVersion="0.0.0.1" \
      -p enumClassPrefix=true \
      --enable-post-process-file

To make it compilable without duplicates I need to modify the vendor provider with the suggested jq statement:

cat swagger-4.4.0-dist.json| jq '. | select(((.paths[][].tags| type=="array"), length) > 1).paths[][].tags |= [.[0]]' > swagger440_untagged.json

And with that it compiles.

Not sure if it affects anything else at this point.
update:"Functions are no longer available in respective api collection (defined by tags), which again could be workaround by hardcoding all tags to same value, instead of first tag found. I found another open issue (#9500) which states based on tags different packages should be created".

PS: i need the -p enumClassPrefix=true to hide the issue with generating ModMap's which are not defined anywhere (which happens about just 6 times or so in the swagger spec file), this is for version 4.2.2, 5.2.0 and 5.2.1-20210917

Regards

@gunivan
Copy link

gunivan commented Jan 17, 2022

Hi,
Confirmed that set structPrefix=true then the issue is resolved.
I have tested with OpenApiGenerator gradle plugin version 5.3.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants