Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Receive all logs if no attributes are specified #37721

Merged
merged 8 commits into from
Mar 3, 2025

Conversation

petern48
Copy link
Contributor

@petern48 petern48 commented Feb 5, 2025

Description

I'm proposing a new convention where if the attributes field is empty, then it indicates we want to ingest everything that is sent. This helps users avoid having to modify which fields they want in two places. Happy to submit a PR myself. Please check out the linked issue and let me know if you approve or want any changes. Then I will finish implementing whatever else is necessary (e.g docs, tests).

Link to tracking issue

Fixes #37720

Testing

I added a TestEmptyAttributes() test to verify the behavior in logs_test.go

Documentation

I documented the new behavior in the README.md and added a separate config example of how it can be used

Copy link
Contributor

@dehaansa dehaansa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice overall, should have a doc update as well, and a test.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Feb 25, 2025
@petern48
Copy link
Contributor Author

Sorry, haven't gotten around to writing the tests and doc changes. Hope to do so soon.

@github-actions github-actions bot removed the Stale label Feb 26, 2025
@petern48 petern48 requested a review from dehaansa March 1, 2025 06:52
@petern48
Copy link
Contributor Author

petern48 commented Mar 1, 2025

@dehaansa Should be ready for full test and review. It's my first time contributing code to this repo, who's approval do I need to get CI running? (I see another reviewer as well as an assignee).

@dehaansa
Copy link
Contributor

dehaansa commented Mar 2, 2025

@dehaansa Should be ready for full test and review. It's my first time contributing code to this repo, who's approval do I need to get CI running? (I see another reviewer as well as an assignee).

Any approver or maintainer can approve CI when approval is required. I've approved them - looks like you've got some linter issues to resolve before this is ready to merge, otherwise looking good!

@petern48 petern48 force-pushed the receive_all_cloudflare branch from 5a9c302 to 4a281f7 Compare March 2, 2025 06:24
@petern48
Copy link
Contributor Author

petern48 commented Mar 3, 2025

I noticed CI didn't run for the most recent commit. Was that because I force-pushed, or does approval only last for a few commits before needing it again?

@dehaansa
Copy link
Contributor

dehaansa commented Mar 3, 2025

I noticed CI didn't run for the most recent commit. Was that because I force-pushed, or does approval only last for a few commits before needing it again?

Sadly it only lasts per commit, I was manually responding to the others. I believe CI runs much friendleir once you've got a contribution or three under your belt and can apply for membership.

@petern48
Copy link
Contributor Author

petern48 commented Mar 3, 2025

Relevant parts of the failed test (everything else was essentially repeats of these). Seems pretty unrelated to me. Guessing it just needs a re-run?

DONE 27 tests, 1 failure in 308.177s

✖  . (5m0.044s)

=== Failed
=== FAIL: . TestIntegration (300.00s)
XXX: too many requests: &{AuthConfigs:map[[https://index.docker.io/v1/:{Username:](https://index.docker.io/v1/:%7BUsername:) Password: Auth:Z2l0aHViYWN0aW9uczozZDY0NzJiOS0zZDQ5LTRkMTctOWZjOS05MGQyNDI1ODA0M2I= Email: ServerAddress: IdentityToken: RegistryToken:}] HTTPHeaders:map[] PsFormat: ImagesFormat: NetworksFormat: PluginsFormat: VolumesFormat: StatsFormat: DetachKeys: CredentialsStore: CredentialHelpers:map[] Filename: ServiceInspectFormat: ServicesFormat: TasksFormat: SecretFormat: ConfigFormat: NodesFormat: PruneFilters:[] Proxies:map[] Experimental: StackOrchestrator: Kubernetes:<nil> CurrentContext: CLIPluginsExtraDirs:[] Aliases:map[]}XXX: too many requests: &
(... repeated a ton of times)
make: *** [Makefile:174: gointegration-test] Error 2
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/coreinternal/scraperinttest/scraperint.go:146
        	            				/opt/hostedtoolcache/go/1.23.6/x64/src/runtime/asm_amd64.s:1700
        	Error:      	Condition never satisfied
        	Test:       	TestIntegration
        	Messages:   	create container timeout: <nil>
    scraperint.go:273: 
        	Error Trace:	/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/coreinternal/scraperinttest/scraperint.go:273
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/coreinternal/scraperinttest/scraperint.go:255
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/coreinternal/scraperinttest/scraperint.go:251
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/haproxyreceiver/integration_test.go:42
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/coreinternal/scraperinttest/scraperint.go:72
        	            				/home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/haproxyreceiver/integration_test.go:52
        	Error:      	Should NOT be empty, but was map[]
        	Test:       	TestIntegration
        	Messages:   	no containers in use

Copy link
Contributor

@dehaansa dehaansa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, integration test flake seems unrelated. Thanks for the contribution!

@dehaansa dehaansa added the ready to merge Code review completed; ready to merge by maintainers label Mar 3, 2025
@songy23 songy23 merged commit ac72fb4 into open-telemetry:main Mar 3, 2025
167 checks passed
@github-actions github-actions bot added this to the next release milestone Mar 3, 2025
@petern48 petern48 deleted the receive_all_cloudflare branch March 3, 2025 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready to merge Code review completed; ready to merge by maintainers receiver/cloudflare
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[cloudflarereceiver] Allow receiving all log attributes without manually specifying fields
4 participants