Skip to content

Conversation

@Dhruvit96
Copy link

Description

This unit test enhancement ensures that the configuration can be serialized correctly.

Link to tracking issue

Fixes #12906

@Dhruvit96 Dhruvit96 requested a review from a team as a code owner April 23, 2025 05:57
@Dhruvit96 Dhruvit96 requested a review from bogdandrutu April 23, 2025 05:57
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Apr 23, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: Dhruvit96 / name: Dhruvit Maniya (b12a992)

@Dhruvit96 Dhruvit96 force-pushed the config-serialization-test branch from b11c31d to e08944e Compare April 23, 2025 06:07
@dmathieu
Copy link
Member

Could you add tests? (and fix the failing CI checks)

@Dhruvit96 Dhruvit96 changed the title [mdatagen] Add validation for config serialization [chore][mdatagen] Add validation for config serialization Apr 23, 2025
@Dhruvit96
Copy link
Author

Could you add tests? (and fix the failing CI checks)

I don't think this change requires changelog.
For now I have added chore in PR title but I think it skipped few tests as well, could you please add skip changelog lable?

@dmathieu
Copy link
Member

I think it does require a changelog entry. This changes the behavior of a public method in a public package.

@Dhruvit96 Dhruvit96 changed the title [chore][mdatagen] Add validation for config serialization [mdatagen] Add validation for config serialization Apr 23, 2025
@codecov
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.63%. Comparing base (dc3f5d2) to head (dca103b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #12912      +/-   ##
==========================================
+ Coverage   91.61%   91.63%   +0.01%     
==========================================
  Files         505      505              
  Lines       28409    28429      +20     
==========================================
+ Hits        26027    26051      +24     
+ Misses       1873     1870       -3     
+ Partials      509      508       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Dhruvit96 Dhruvit96 force-pushed the config-serialization-test branch 3 times, most recently from d299e9b to 7efced9 Compare April 23, 2025 16:36
@Dhruvit96
Copy link
Author

I have completed all the changes from my end, and the implementation is now ready for review. It should pass all CI checks successfully.

Thank you for your patience.

@Dhruvit96 Dhruvit96 changed the title [mdatagen] Add validation for config serialization [componenttest] Add validation for config serialization Apr 29, 2025
@Dhruvit96 Dhruvit96 force-pushed the config-serialization-test branch from 7286ad6 to 51c85aa Compare April 29, 2025 06:45
@Dhruvit96 Dhruvit96 changed the title [componenttest] Add validation for config serialization [component][componenttest] Add validation for config serialization Apr 29, 2025
@Dhruvit96
Copy link
Author

Please let me know if there's anything I might have missed in this PR or anything I can do to help expedite the review process.

@Dhruvit96 Dhruvit96 force-pushed the config-serialization-test branch from c775b48 to 596a7d3 Compare May 6, 2025 06:13
@evan-bradley
Copy link
Contributor

@Dhruvit96 Sorry for the slow review, overall this looks good to me. Could you rebase?

@Dhruvit96
Copy link
Author

@evan-bradley I've rebased the branch, could you please have a look?

@evan-bradley
Copy link
Contributor

I like the idea of checking whether a config struct can be marshalled and I think CheckConfigStruct may be a good place to add it, but I'm not so sure about adding confmap as a dependency for most components, I think all the go.mod changes in this PR shows how widely impactful this will be.

I think ultimately we should be okay since virtually all Collector binaries will need confmap anyway, but I want to make sure we're deliberate before we grow the transitive dependency list like this. @open-telemetry/collector-approvers any thoughts?

@evan-bradley
Copy link
Contributor

From the 2025-05-21 SIG meeting: We think that since YAML is a superset of JSON, and confmap maintains close compatibility with YAML, we can get by with just testing JSON now to reduce the number of added dependencies.

@Dhruvit96 Could you revise this PR so it only checks the output of json.Marshal? We think this will likely be sufficient for now and we can address any discrepancies between JSON and confmap or YAML in a separate PR.

@Dhruvit96
Copy link
Author

Hi @evan-bradley,

I've updated the PR to validate only JSON marshalling. However, this change did not reduce dependencies in any package other than the scrapertest.

I agree with your point that YAML is a superset of JSON, and we might be able to rely on JSON validation alone. That said, since the dependency impact is similar, validating both JSON and YAML would make the testing more comprehensive.

Could you please share your thoughts?

Comment on lines +52 to +59
conf := confmap.New()
if err := conf.Marshal(config); err != nil {
return err
}

configMap := conf.ToStringMap()

_, err := json.Marshal(configMap)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
conf := confmap.New()
if err := conf.Marshal(config); err != nil {
return err
}
configMap := conf.ToStringMap()
_, err := json.Marshal(configMap)
_, err := json.Marshal(config)

If we remove marshaling with confmap, we should be able to only test JSON marshaling here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a generic safeguard to prevent panics during the marshalling of configuration maps. We observed that the OpenTelemetry Collector encountered panics when used in conjunction with the OpAMP extension. As identified in this comment, the root cause was a failure to marshal a configuration represented as a map[string]any.

Validating only the marshallability of structs does not address such scenarios, as marshalling errors in structs can often be resolved using the json:"-" tag. However, in the case of maps, resolving the issue requires the use of the mapstructure:"-" tag.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evan-bradley Based on the information provided above, could you please let me know the next steps?

@evan-bradley
Copy link
Contributor

Thanks @Dhruvit96. The dependency impact is coming from the use of confmap, we can remove that as well. I left a suggestion that shows how I think it should look.

@Dhruvit96 Dhruvit96 requested a review from evan-bradley May 30, 2025 20:05
@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jun 14, 2025
@Dhruvit96 Dhruvit96 force-pushed the config-serialization-test branch from 77ea88e to b12a992 Compare June 14, 2025 08:47
@github-actions github-actions bot removed the Stale label Jun 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2025

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jul 1, 2025
@github-actions
Copy link
Contributor

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Jul 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[mdatagen] - Add auto-generated tests to verify that the config structure is marshallable

3 participants