[processor/transform] not enough preconditions to guard against warnings

[ajsaclayan]

Component(s)

processor/transform

Is your feature request related to a problem? Please describe.

I have a transform processor configured to truncate log attributes and the body field:

  transform/newrelic:
    log_statements:
      - context: resource
        statements:
          - truncate_all(attributes, 4090)
      - context: log
        statements:
          - truncate_all(attributes, 4090)
          - set(body, Substring(body.string, 0, 4090))

This works, but the problem is when we send a body whose length is less than 4090 this warning message occurs:
invalid range for substring function, 4090 cannot be greater than the length of target string

Describe the solution you'd like

It would be nice to be able to have a precondition like WHERE body.Length() > 4090 to be able to guard against this warning.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[jack-berg]

Another alternative would be to adjust allow truncate to be called on on body when its an AnyValue string. Right now, calling truncate_all(body, 4090) errors with:

failed to execute statement: truncate_all(body, 4095), expected pcommon.Map but got string"}

Definitely useful to have some way to truncate string log bodies in the transform processor.

[TylerHelmuth]

Both suggestions sound valuable.

It would be good to have a way to check the lengths of things (string, maps, slices, etc). It would also be good to have a function that can truncate strings.

In the meantime, this will work and will not cause a warning when len(body) is less than 4090:

transform/newrelic:
    log_statements:
      - context: resource
        statements:
          - truncate_all(attributes, 4090)
      - context: log
        statements:
          - truncate_all(attributes, 4090)
          - set(cache["body"], body)
          - truncate_all(cache, 4090)
          - set(body, cache["body"])

[TylerHelmuth]

@ajsaclayan @jack-berg @danelson is this something you'd like to work on?

[TylerHelmuth]

Technically more efficient solution:

transform/newrelic:
    log_statements:
      - context: resource
        statements:
          - truncate_all(attributes, 4090)
      - context: log
        statements:
          - set(attributes["truncate-body"], body)
          - truncate_all(attributes, 4090)
          - set(body, attributes["truncate-body"])
          - delete_key(attributes, "truncate-body")

But won't be viable if truncate-body already exists in attributes as you'd be overwriting the value.