Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTel Collector Builder: Go "monorepo" support #9252

Open
jmacd opened this issue Jan 9, 2024 · 0 comments
Open

OTel Collector Builder: Go "monorepo" support #9252

jmacd opened this issue Jan 9, 2024 · 0 comments

Comments

@jmacd
Copy link
Contributor

jmacd commented Jan 9, 2024

Is your feature request related to a problem? Please describe.
The only supported way to build an OTel Collector requires generating a "fresh" go.mod file for the main package. This creates a challenging workflow for users of a "monorepo", which is to describe a single go.mod file used to build a large code base. This can also create a security concern; users who are familiar with Go module tooling may expect to update go.mod files using the Go toolchain. However, when they do, the next invocation of the builder will wipe out any changes that were made to the go.mod files.

Describe the solution you'd like
The proposed solution is to offer a mode in which a new go.mod file is not created. Instead, when this mode is selected, each of the modules that would have been required will be individually updated. There is a concept of the "enclosing module" which is determined by the Go toolchain. When the builder is invoked in this new mode, it will go get each module using the version in the build configuration, only when the configured module is newer than the one in the enclosing Go module, with emphasis on use of semantic version comparison to control whether an update is performed.

We can imagine two steps, the first already described: with a --skip-new-go-module flag the builder will go get each component instead of generating a new go.mod file. The second, nice-to-have but not essential feature called for, for users of a monorepo, is to use the configuration of the enclosing module to update the build configuration by editing the yaml and writing a new configuration. This builder sync command, named for its simlarity to the go work sync command, would rewrite the build configuration using the go module version that is found in the enclosing module for every component. That would allow the build configuration to be distributed and re-built outside of the monorepo, using the intended "current" versions that were in effect inside the monorepo.

Describe alternatives you've considered
In the case where as user has an OTel Collector build and wants to update one of the indirect dependencies, instead of a simple go get command, they can continue with the status quo. They will add a replace statement to the configuration and re-invoke the builder. This is error-prone for users who are not familiar with the collector build process.

mx-psi added a commit that referenced this issue May 3, 2024
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
When running tests, waiting for `downloadModules()` to fail 3 times when
that's expected adds time to the test run. This updates tests to only
attempt downloading once. Note: if there's a network failure that could
cause `downloadModules()` to fail when it should normally succeed. Also
the wording here is `retries` when in actuality it's the number of
attempts. I didn't change this to keep the log wording the same, but I
can change the wording if that's preferable.

<!-- Issue number if applicable -->
#### Link to tracking issue
this will help for adding tests for
#9252
and
#9896

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Tests ran

---------

Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
andrzej-stencel pushed a commit to andrzej-stencel/opentelemetry-collector that referenced this issue May 27, 2024
…10035)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
When running tests, waiting for `downloadModules()` to fail 3 times when
that's expected adds time to the test run. This updates tests to only
attempt downloading once. Note: if there's a network failure that could
cause `downloadModules()` to fail when it should normally succeed. Also
the wording here is `retries` when in actuality it's the number of
attempts. I didn't change this to keep the log wording the same, but I
can change the wording if that's preferable.

<!-- Issue number if applicable -->
#### Link to tracking issue
this will help for adding tests for
open-telemetry#9252
and
open-telemetry#9896

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Tests ran

---------

Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant