feat: Render error message condition as json/structured message #3977

lburgazzoli · 2023-01-13T17:33:38Z

Release Note

NONE

lburgazzoli · 2023-01-14T16:22:39Z

@astefanutti I'm experiencing some issues as part of the ci build:

cd pkg/client/camel && go test ./...
# github.com/apache/camel-k/pkg/client/camel/applyconfiguration
Error: applyconfiguration/utils.go:82:19: undefined: v1.HealthCheckResponseApplyConfiguration
Error: applyconfiguration/utils.go:126:19: undefined: v1.PodConditionApplyConfiguration

What is the proper way to generate the client & apply config ?

astefanutti · 2023-01-16T09:08:55Z

@astefanutti I'm experiencing some issues as part of the ci build:
cd pkg/client/camel && go test ./...
# github.com/apache/camel-k/pkg/client/camel/applyconfiguration
Error: applyconfiguration/utils.go:82:19: undefined: v1.HealthCheckResponseApplyConfiguration
Error: applyconfiguration/utils.go:126:19: undefined: v1.PodConditionApplyConfiguration
What is the proper way to generate the client & apply config ?

Could you please try generating the client locally by replacing applyconfiguration-gen in the codegen-tools-install target of the Makefile with:

go install github.com/astefanutti/code-generator/cmd/applyconfiguration-gen@pr-144

That should generate the missing types.

This is a temporary "glitch" as a fix is required in code-generator, which will go away as soon as it lands upstream.

lburgazzoli · 2023-01-16T10:12:31Z

mh , looks like it did not help

astefanutti · 2023-01-16T10:23:10Z

mh , looks like it did not help

Hum, hasn't make generate created the new types in the pkg/client/camel/applyconfiguration Let me try on my side...

lburgazzoli · 2023-01-16T10:46:27Z

This is what I see as output for make generate

➜ make generate
Installing k8s.io/code-generator tools with version v0.25.2
#go install k8s.io/code-generator/cmd/applyconfiguration-gen@v0.25.2
go install github.com/astefanutti/code-generator/cmd/applyconfiguration-gen@pr-144
go: finding module for package k8s.io/code-generator/cmd/applyconfiguration-gen/generators
go: finding module for package k8s.io/code-generator/cmd/applyconfiguration-gen/args
go: found k8s.io/code-generator/cmd/applyconfiguration-gen/args in k8s.io/code-generator v0.26.0
go: found k8s.io/code-generator/cmd/applyconfiguration-gen/generators in k8s.io/code-generator v0.26.0
go install k8s.io/code-generator/cmd/client-gen@v0.25.2
go install k8s.io/code-generator/cmd/lister-gen@v0.25.2
go install k8s.io/code-generator/cmd/informer-gen@v0.25.2
./script/gen_client.sh
Generating Go client code...
F0116 11:45:08.040686   39309 main.go:51] Error: Failed executing generator: some packages had errors:
template: /Users/luca-rh/.cache/go/pkg/mod/k8s.io/code-generator@v0.26.0/cmd/applyconfiguration-gen/generators/applyconfiguration.go:220:1:28: executing "/Users/luca-rh/.cache/go/pkg/mod/k8s.io/code-generator@v0.26.0/cmd/applyconfiguration-gen/generators/applyconfiguration.go:220" at <raw>: error calling raw: runtime error: invalid memory address or nil pointer dereference
template: /Users/luca-rh/.cache/go/pkg/mod/k8s.io/code-generator@v0.26.0/cmd/applyconfiguration-gen/generators/applyconfiguration.go:220:1:28: executing "/Users/luca-rh/.cache/go/pkg/mod/k8s.io/code-generator@v0.26.0/cmd/applyconfiguration-gen/generators/applyconfiguration.go:220" at <raw>: error calling raw: runtime error: invalid memory address or nil pointer dereference
./script/gen_crd.sh
./script/gen_doc.sh
Generating API documentation...
Generating CRD API documentation...
I0116 11:45:21.152511   39774 main.go:136] parsing go packages in directory github.com/apache/camel-k/pkg/apis/camel/v1
I0116 11:45:24.586035   39774 main.go:240] using package=github.com/apache/camel-k/pkg/apis/camel/v1
I0116 11:45:24.586054   39774 main.go:240] using package=github.com/apache/camel-k/pkg/apis/camel/v1/trait
W0116 11:45:24.612611   39774 main.go:469] not found external link source for type encoding/xml.Name
I0116 11:45:24.616375   39774 main.go:176] written to ./script/gen_crd/../../docs/modules/ROOT/partials/apis/camel-k-crds.adoc
I0116 11:45:25.044803   39872 main.go:136] parsing go packages in directory github.com/apache/camel-k/pkg/apis/camel/v1alpha1
I0116 11:45:28.453325   39872 main.go:240] using package=github.com/apache/camel-k/pkg/apis/camel/v1alpha1
W0116 11:45:28.462888   39872 main.go:469] not found external link source for type encoding/json.Number
W0116 11:45:28.462939   39872 main.go:469] not found external link source for type encoding/json.Number
W0116 11:45:28.463025   39872 main.go:469] not found external link source for type encoding/json.Number
I0116 11:45:28.464208   39872 main.go:176] written to ./script/gen_crd/../../docs/modules/ROOT/partials/apis/kamelets-crds.adoc
Generating CRD API documentation... Done.
Generating API documentation... done!
Generating traits documentation...
Generating traits documentation... done!
cd pkg/apis/camel && /Users/luca-rh/.cache/go/bin/controller-gen paths="./..." object
cd addons/keda/duck && /Users/luca-rh/.cache/go/bin/controller-gen paths="./..." object
cd addons/strimzi/duck && /Users/luca-rh/.cache/go/bin/controller-gen paths="./..." object
./script/gen_client_strimzi.sh
Generating Go client code for Strimzi addon...

astefanutti · 2023-01-16T11:09:32Z

@lburgazzoli Could you please try with:

go install github.com/astefanutti/code-generator/cmd/applyconfiguration-gen@pr-144-install

I've done the required change in that new branch to overcome golang/go#50278, and tested it locally successfully.

lburgazzoli · 2023-01-16T11:18:50Z

Seems to break some other stuffs

go test ./...
# github.com/apache/camel-k/pkg/client/camel/applyconfiguration/camel/v1
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:45:11: cannot use &value (value of type *"k8s.io/api/core/v1".PodConditionType) as type "k8s.io/api/core/v1".PodConditionType in assignment
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:53:13: cannot use &value (value of type *"k8s.io/api/core/v1".ConditionStatus) as type "k8s.io/api/core/v1".ConditionStatus in assignment
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:61:20: cannot use &value (value of type *"k8s.io/apimachinery/pkg/apis/meta/v1".Time) as type "k8s.io/apimachinery/pkg/apis/meta/v1".Time in assignment
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:69:25: cannot use &value (value of type *"k8s.io/apimachinery/pkg/apis/meta/v1".Time) as type "k8s.io/apimachinery/pkg/apis/meta/v1".Time in assignment
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:77:13: cannot use &value (value of type *string) as type string in assignment
pkg/client/camel/applyconfiguration/camel/v1/podcondition.go:85:14: cannot use &value (value of type *string) as type string in assignment

And lints:

addons/master/master.go:98:65                                     typecheck      t.Client undefined (type *masterTrait has no field or method Client)
addons/master/master.go:232:45                                    typecheck      t.Client undefined (type *masterTrait has no field or method Client)
addons/resume/resume.go:88:75                                     typecheck      r.Client undefined (type *resumeTrait has no field or method Client)
pkg/controller/build/build_controller.go:165:5                    typecheck      a.InjectClient undefined (type Action has no field or method InjectClient)
pkg/controller/build/build_controller.go:167:5                    typecheck      a.InjectRecorder undefined (type Action has no field or method InjectRecorder)
pkg/controller/integrationkit/integrationkit_controller.go:263:5  typecheck      a.InjectClient undefined (type Action has no field or method InjectClient)
pkg/controller/kamelet/kamelet_controller.go:159:5                typecheck      a.InjectClient undefined (type Action has no field or method InjectClient)
addons/tracing/tracing.go:94:52                                   typecheck      t.Client undefined (type *tracingTrait has no field or method Client)
addons/tracing/tracing.go:99:8                                    typecheck      t.L undefined (type *tracingTrait has no field or method L)
pkg/cmd/root.go:231:24                                            typecheck      invalid operation: cannot compare command._client != nil (operator != not defined on untyped nil)
pkg/controller/integrationplatform/create_test.go:50:4            typecheck      h.InjectClient undefined (type Action has no field or method InjectClient)
pkg/controller/integrationplatform/initialize_test.go:50:4        typecheck      h.InjectClient undefined (type Action has no field or method InjectClient)
pkg/controller/integrationplatform/initialize_test.go:80:4        typecheck      h.InjectClient undefined (type Action has no field or method InjectClient)
pkg/metadata/metadata_capabilities_test.go:50:8                   typecheck      meta.Dependencies undefined (type IntegrationMetadata has no field or method Dependencies)
pkg/metadata/metadata_capabilities_test.go:57:8                   typecheck      meta.RequiredCapabilities undefined (type IntegrationMetadata has no field or method RequiredCapabilities)
pkg/metadata/metadata_dependencies_test.go:62:8                   typecheck      meta.Dependencies undefined (type IntegrationMetadata has no field or method Dependencies)
pkg/metadata/metadata_dependencies_test.go:95:8                   typecheck      meta.Dependencies undefined (type IntegrationMetadata has no field or method Dependencies)
pkg/metadata/metadata_dependencies_test.go:219:22                 typecheck      meta.RequiredCapabilities undefined (type IntegrationMetadata has no field or method RequiredCapabilities)
pkg/metadata/metadata_dependencies_test.go:376:22                 typecheck      meta.RequiredCapabilities undefined (type IntegrationMetadata has no field or method RequiredCapabilities)
pkg/metadata/metadata_http_test.go:48:22                          typecheck      meta.ExposesHTTPServices undefined (type IntegrationMetadata has no field or method ExposesHTTPServices)
pkg/metadata/metadata_http_test.go:49:23                          typecheck      meta.PassiveEndpoints undefined (type IntegrationMetadata has no field or method PassiveEndpoints)
pkg/metadata/metadata_http_test.go:73:22                          typecheck      meta.ExposesHTTPServices undefined (type IntegrationMetadata has no field or method ExposesHTTPServices)
pkg/metadata/metadata_http_test.go:74:22                          typecheck      meta.PassiveEndpoints undefined (type IntegrationMetadata has no field or method PassiveEndpoints)
pkg/metadata/metadata_http_test.go:95:22                          typecheck      meta.ExposesHTTPServices undefined (type IntegrationMetadata has no field or method ExposesHTTPServices)
pkg/metadata/metadata_http_test.go:96:22                          typecheck      meta.PassiveEndpoints undefined (type IntegrationMetadata has no field or method PassiveEndpoints)
pkg/metadata/metadata_uri_test.go:56:30                           typecheck      metadata.FromURIs undefined (type IntegrationMetadata has no field or method FromURIs)
pkg/metadata/metadata_uri_test.go:57:25                           typecheck      metadata.FromURIs undefined (type IntegrationMetadata has no field or method FromURIs)
pkg/metadata/metadata_uri_test.go:58:30                           typecheck      metadata.ToURIs undefined (type IntegrationMetadata has no field or method ToURIs)
pkg/metadata/metadata_uri_test.go:59:25                           typecheck      metadata.ToURIs undefined (type IntegrationMetadata has no field or method ToURIs)
pkg/metadata/metadata_uri_test.go:94:30                           typecheck      metadata.FromURIs undefined (type IntegrationMetadata has no field or method FromURIs)
pkg/metadata/metadata_uri_test.go:96:30                           typecheck      metadata.ToURIs undefined (type IntegrationMetadata has no field or method ToURIs)
pkg/trait/pull_secret.go:74:19                                    typecheck      invalid operation: cannot compare t.Client != nil (operator != not defined on untyped nil)
pkg/trait/trait_catalog.go:59:6                                   typecheck      t.InjectClient undefined (type Trait has no field or method InjectClient)

astefanutti · 2023-01-16T14:24:54Z

Could you add --input-dirs k8s.io/apimachinery/pkg/apis/meta/v1,k8s.io/api/core/v1 to the list of arguments passed to applyconfiguration-gen in gen_client.sh, as the changes to the Integration types import those packages, and regenerate the client packages?

lburgazzoli · 2023-01-20T13:56:15Z

@astefanutti @squakez @tadayosi @claudio4j it lacks a test for integration that are scaled > 1, let me know if the test should be part of this pr or can wait for a second iteration.

lburgazzoli · 2023-01-20T14:13:51Z

@astefanutti @squakez @tadayosi @claudio4j it lacks a test for integration that are scaled > 1, let me know if the test should be part of this pr or can wait for a second iteration.

Feel free to track it in a follow up issue and work it later.

#3997

squakez · 2023-01-20T15:29:03Z

pkg/client/camel/applyconfiguration/core/v1/affinity.go

+
+// AffinityApplyConfiguration represents an declarative configuration of the Affinity type for use
+// with apply.
+type AffinityApplyConfiguration struct {


I think this is some issue with the autogen. We should not have kubernetes resources here I guess. @astefanutti fyi

I agree it's not ideal to have all these types duplicated. I've giving this another cycle, and it seems there are issues with both embedding and cross-referencing.

@lburgazzoli I know I've been suggesting to embed corev1.PodCondition, but maybe it's not needed in the short term, so we can look into those issues. WDYT?

As an alternative I can replace the struct with an custom one with the same fields and explicit copy of fields.
Both are fine with me, @astefanutti @squakez cast your vote :)

If we don't have all those bunch of duplicated types it would be more maintainable, so, I'd prefer that option.

One alternative that seems to work is to not inline the corev1.PodCondition, i.e. adding a field, and remove metav1 and corev1 from the code-generator input dirs. That way the API still has the corresponding info, but the generated apply configurations do no contain any duplicate, and instead refer to the API rather than the apply configurations, which is acceptable IMO.

ok, will try that

done, please have a look at the latest commits (it is a little bit weird as there is a Pod nested field in the PodCondition (if you have better names, please suggest)

Maybe re-naming the field to Readiness or Ready would make it clear it's the ready condition from the Pod that's provided?

lburgazzoli · 2023-01-23T13:57:38Z

Note, a number of tests are failing because of:

  no matching manifest for linux/amd64 in the manifest list entries

astefanutti

Looks great!

script/gen_client_strimzi.sh

pkg/controller/integration/monitor.go

…e#3967

…yconfig) apache#3967

- support for MAVEN_CMD env var as camel-k does - run maven in batch mode to avoid including ASCII escaping chars in the error message which could cause the test to fail in some evironments

I've removed some expectation as based on timing an likely to randomly fail.

- remove goroutine for build related test as not needed - added waiting group to wait for async tasks completion

…e tests

tadayosi

LGMT. Thanks!

lburgazzoli · 2023-01-24T06:44:04Z

@squakez @astefanutti anything missing or ok to merge ?

astefanutti · 2023-01-24T08:13:06Z

@squakez @astefanutti anything missing or ok to merge ?

I'd be inclined to have the change to temurin as default base image (131915b) done in a separate PR to make it more visible in the release note and ease backport. Other than that, it's good to go 👍🏼.

lburgazzoli · 2023-01-24T08:16:45Z

let see if I can work on something

lburgazzoli · 2023-01-24T08:21:21Z

@astefanutti opened #4003 (it is actually made by 3 commits, I can squash them if needed)

astefanutti · 2023-01-24T08:23:54Z

@astefanutti opened #4003 (it is actually made by 3 commits, I can squash them if needed)

Awesome! Let's wait CI to pass for #4003, so we can merge it and have that one rebased on top.

lburgazzoli force-pushed the github-3967 branch 3 times, most recently from c2a384e to 2bc4463 Compare January 14, 2023 16:10

lburgazzoli force-pushed the github-3967 branch from 2bc4463 to 3694709 Compare January 16, 2023 09:44

lburgazzoli force-pushed the github-3967 branch from 3694709 to e00f247 Compare January 16, 2023 11:12

lburgazzoli force-pushed the github-3967 branch 9 times, most recently from 084147b to e80352b Compare January 20, 2023 11:47

lburgazzoli marked this pull request as ready for review January 20, 2023 11:49

lburgazzoli force-pushed the github-3967 branch 2 times, most recently from 39c55ae to 9d2fa69 Compare January 20, 2023 12:07

lburgazzoli requested review from astefanutti, squakez, tadayosi and claudio4j January 20, 2023 13:33

lburgazzoli mentioned this pull request Jan 20, 2023

Create a test for structured health check conditions when integrations are scaled > 1 #3997

Closed

squakez reviewed Jan 20, 2023

View reviewed changes

lburgazzoli force-pushed the github-3967 branch 3 times, most recently from 4181822 to 1923898 Compare January 23, 2023 13:29

astefanutti approved these changes Jan 23, 2023

View reviewed changes

script/gen_client_strimzi.sh Outdated Show resolved Hide resolved

pkg/controller/integration/monitor.go Outdated Show resolved Hide resolved

lburgazzoli force-pushed the github-3967 branch from 0a3a49d to 6efc791 Compare January 23, 2023 14:17

squakez approved these changes Jan 23, 2023

View reviewed changes

lburgazzoli added 4 commits January 23, 2023 15:22

feat: Render error message condition as json/structured message apach…

ea5fd24

…e#3967

feat: Render error message condition as json/structured message (appl…

11f066f

…yconfig) apache#3967

chore(test): improve maven log test

d5de6ef

- support for MAVEN_CMD env var as camel-k does - run maven in batch mode to avoid including ASCII escaping chars in the error message which could cause the test to fail in some evironments

chore(test): improve knative kamelet binding test

efb5e98

I've removed some expectation as based on timing an likely to randomly fail.

lburgazzoli force-pushed the github-3967 branch from 6efc791 to 379b7fb Compare January 23, 2023 14:22

lburgazzoli added 2 commits January 23, 2023 17:44

chore(test): improve locale e2e tests

517ff69

- remove goroutine for build related test as not needed - added waiting group to wait for async tasks completion

feat: use eclipse-temurin:11 as runtime base image

131915b

lburgazzoli force-pushed the github-3967 branch from 379b7fb to 131915b Compare January 23, 2023 16:44

lburgazzoli added 2 commits January 23, 2023 20:40

workaround: force eclipse-temurin:11 as runtime base image for upgrad…

9eff96c

…e tests

feat: use eclipse temurin as java distribution for github actions

668b08f

tadayosi approved these changes Jan 24, 2023

View reviewed changes

oscerd approved these changes Jan 24, 2023

View reviewed changes

squakez mentioned this pull request Jan 24, 2023

Release 1.12.0 #4002

Closed

lburgazzoli merged commit 30a82e5 into apache:main Jan 24, 2023

lburgazzoli deleted the github-3967 branch January 24, 2023 11:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Render error message condition as json/structured message #3977

feat: Render error message condition as json/structured message #3977

lburgazzoli commented Jan 13, 2023 •

edited

Loading

lburgazzoli commented Jan 14, 2023

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023 •

edited

Loading

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023

astefanutti commented Jan 16, 2023 •

edited

Loading

lburgazzoli commented Jan 20, 2023

lburgazzoli commented Jan 20, 2023

squakez Jan 20, 2023

astefanutti Jan 23, 2023 •

edited

Loading

lburgazzoli Jan 23, 2023

squakez Jan 23, 2023

astefanutti Jan 23, 2023

lburgazzoli Jan 23, 2023

lburgazzoli Jan 23, 2023

astefanutti Jan 23, 2023 •

edited

Loading

lburgazzoli Jan 23, 2023

lburgazzoli commented Jan 23, 2023

astefanutti left a comment

tadayosi left a comment

lburgazzoli commented Jan 24, 2023

astefanutti commented Jan 24, 2023

lburgazzoli commented Jan 24, 2023

lburgazzoli commented Jan 24, 2023

astefanutti commented Jan 24, 2023

feat: Render error message condition as json/structured message #3977

feat: Render error message condition as json/structured message #3977

Conversation

lburgazzoli commented Jan 13, 2023 • edited Loading

lburgazzoli commented Jan 14, 2023

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023 • edited Loading

astefanutti commented Jan 16, 2023

lburgazzoli commented Jan 16, 2023

astefanutti commented Jan 16, 2023 • edited Loading

lburgazzoli commented Jan 20, 2023

lburgazzoli commented Jan 20, 2023

squakez Jan 20, 2023

Choose a reason for hiding this comment

astefanutti Jan 23, 2023 • edited Loading

Choose a reason for hiding this comment

lburgazzoli Jan 23, 2023

Choose a reason for hiding this comment

squakez Jan 23, 2023

Choose a reason for hiding this comment

astefanutti Jan 23, 2023

Choose a reason for hiding this comment

lburgazzoli Jan 23, 2023

Choose a reason for hiding this comment

lburgazzoli Jan 23, 2023

Choose a reason for hiding this comment

astefanutti Jan 23, 2023 • edited Loading

Choose a reason for hiding this comment

lburgazzoli Jan 23, 2023

Choose a reason for hiding this comment

lburgazzoli commented Jan 23, 2023

astefanutti left a comment

Choose a reason for hiding this comment

tadayosi left a comment

Choose a reason for hiding this comment

lburgazzoli commented Jan 24, 2023

astefanutti commented Jan 24, 2023

lburgazzoli commented Jan 24, 2023

lburgazzoli commented Jan 24, 2023

astefanutti commented Jan 24, 2023

lburgazzoli commented Jan 13, 2023 •

edited

Loading

lburgazzoli commented Jan 16, 2023 •

edited

Loading

astefanutti commented Jan 16, 2023 •

edited

Loading

astefanutti Jan 23, 2023 •

edited

Loading

astefanutti Jan 23, 2023 •

edited

Loading