Collector logging exporter #373
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Start work on logs collector * New commit to poke Github * make gotidy Co-authored-by: Punya Biswal <punya@google.com>
* collector: added log conversion to entries Added functionality to convert pdata Logs to Cloud Logging entries. Also added necessary configuration and self observability. * collector: log mapper, tests, resolving comments Switched to the logMapper pattern, mimicking what the MetricsExporter does. This separation allowed for unit testing the pure functions on the logMapper. Added the first set of unit tests which cover lines but probably don't exhaust edge cases. Should be expanded upon. Removed some fields that we've decided are too nuanced to be part of this PR. * integrationtest: ran make gotidy Some changes were made after running make gotidy. * collector: added required header, fixed linting Forgot to add the Google header to logs_test.go `golint` wants http to be all capitals. `govet` warned about a shadowed declaration of `err` and even though it wasn't a problem figured I'd make it go away anyway. * collector: resetting go.mod Resetting go.mod to the versions in the target branch since there is another PR to update dependencies that we'll rebase on. * collector: parse HTTPRequest from proper place The HTTPRequest data is actually going to come from an attribute in the log, which actually enables this code to be far simpler; originally I wrote it as if the httpRequest was going to be a key in the body that needed to be extracted. With this not being the case, got to remove a ton of json marshalling code and even removed the need to configure parsing HTTP request, since the operation to look it up is constant and we don't lose anything by just checking every time. * collector: lint fix HTTP caps * integrationtest: revert go.mod changes Going back to the previous version here too to make the feature branch rebase easier. * collector: added todo comment Unmarshalling JSON can be slow when done incessantly, left a TODO to investigate an alternative for serializing the com.google.httpRequest data.
damemi
force-pushed
the
collector-logging-feature
branch
from
301fefd
to
b4906a6
Compare
…m#355) * Parse instrumentation scope labels * Parse timestamp * Parse sourceLocation * Parse trace and span ID * Pass Body() to Payload and add checks for instrumentation labels * Change tests and log body parsing for different types * Add tests for trace/span/sourceLocation * go lint
* Add DefaultLogName config option * Switch to generated client and parse log name * Add test cases
…rm#363) * Add config options and implement request batching * Refactor to send requests in chunks
* Rename TestCases to MetricsTestCases * Add logs integration test * Logs integration test (copying metrics framework) * Update HTTPRequest parsing for different value types * Fix up logs.go * Unify MetricsTestCase and LogsTestCase structs * Update recordfixtures command for logs * gofmt * Set project from ADC * Use development logger * Record logs and metrics
damemi
force-pushed
the
collector-logging-feature
branch
from
b4906a6
to
e828553
Compare
dashpole
approved these changes
Apr 29, 2022
This was referenced May 12, 2022
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This picks the logic commits from https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/tree/collector_logging into a new branch based off
main