Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

processor_content_modifier: add support for OTel Logs Resource and Scopes #8989

Merged
merged 1 commit into from
Jun 24, 2024

Conversation

edsiper
Copy link
Member

@edsiper edsiper commented Jun 20, 2024

The following patch extends the processor to allow to modify the resources and scopes of Logs generated by an OpenTelemetry source.

The following new contexts are supported:

  • otel_resource_attributes: alter resource attributes
  • otel_scope_name: manipulate the scope name
  • otel_scope_version: manipulate the scope version
  • otel_scope_attributes: alter the scope attributes

example: fluent-bit.yaml

service:
  flush: 1
  log_level: info
  http_server: on
  http_port: ${FLUENT_BIT_HTTP_MONITORING_PORT}

  pipeline:
    inputs:
      - name: opentelemetry
        port: ${FLUENT_BIT_TEST_LISTENER_PORT}
        processors:
          logs:
            - name: content_modifier
              context: otel_resource_attributes
              action: upsert
              key: "new_attr"
              value: "my_val"

            - name: content_modifier
              context: otel_resource_attributes
              action: delete
              key: "service.name"

            - name: content_modifier
              context: otel_scope_attributes
              action: upsert
              key: "my_new_scope_attr"
              value: "123"

            - name: content_modifier
              context: otel_scope_name
              action: upsert
              value: "new scope name"

            - name: content_modifier
              context: otel_scope_version
              action: upsert
              value: "3.1.0"

  outputs:
    - name: stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: 127.0.0.1
      port: ${TEST_SUITE_HTTP_PORT}

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

…opes

The following patch extends the processor to allow to modify the resources and scopes
of Logs generated by an OpenTelemetry source.

The following new contexts are supported:

 - otel_resource_attributes: alter resource attributes
 - otel_scope_name: manipulate the scope name
 - otel_scope_version: manipulate the scope version
 - otel_scope_attributes: alter the scope attributes

example:

----- fluent-bit.yaml -----

pipeline:
  inputs:
    - name: opentelemetry
      port: ${FLUENT_BIT_TEST_LISTENER_PORT}
      processors:
        logs:
          - name: content_modifier
            context: otel_resource_attributes
            action: upsert
            key: "new_attr"
            value: "my_val"

          - name: content_modifier
            context: otel_resource_attributes
            action: delete
            key: "service.name"

          - name: content_modifier
            context: otel_scope_attributes
            action: upsert
            key: "my_new_scope_attr"
            value: "123"

          - name: content_modifier
            context: otel_scope_name
            action: upsert
            value: "new scope name"

          - name: content_modifier
            context: otel_scope_version
            action: upsert
            value: "3.1.0"

  outputs:
    - name: stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: 127.0.0.1
      port: ${TEST_SUITE_HTTP_PORT}

----- end of file -----

Signed-off-by: Eduardo Silva <eduardo@calyptia.com>
@edsiper edsiper changed the title procesor_content_modifier: add support for OTel Logs Resource and Scopes processor_content_modifier: add support for OTel Logs Resource and Scopes Jun 20, 2024
@edsiper edsiper added this to the Fluent Bit v3.1.0 milestone Jun 21, 2024
@edsiper edsiper merged commit 7060979 into master Jun 24, 2024
47 checks passed
@edsiper edsiper deleted the otel-content-modifier branch June 24, 2024 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant