Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with $lookup function in Telegraf json_transformation derived from jsonata-go #23

Open
iserranoe opened this issue Oct 16, 2023 · 1 comment

Comments

@iserranoe
Copy link

iserranoe commented Oct 16, 2023

I recently had a problem with a json transformation using jsonata in Telegraf. The transformation is the following:

$each($merge([metrics.fields]), function($value, $key) {{
    "key": $key,
    "value": $value
  }}) ~> $reduce(function($acc, $entry) {(
    $contains($entry.key, "__")
      ? (
        $outerLevelKey := $split($entry.key, "__")[0];
        $innerLevelKey := $split($entry.key, "__")[1];        
        $merge([$acc, { 
          $outerLevelKey: $merge([
            $lookup($acc, $outerLevelKey), 
            { $innerLevelKey:$entry.value } 
          ])
        }])
      )
      : $merge([$acc, { $entry.key: $entry.value }])
  )}, {})

That works perfectly in the simulator https://try.jsonata.org/5aN2vrMY3, but when used in Telegraf it doesn't work and I found the problematic function is $lookup($acc, $outerLevelKey), since using $acc.PressSPM instead doesn't cause any problem.

I first asked in Telegraf but they say the issue seems to be in the upstream library they are using, i.e. jsonata-go.

influxdata/telegraf#14069

Also, if anyone knows an alternative way to do the transformation, I'd really appreciate it.

Thanks a lot

@tbal999
Copy link

tbal999 commented Oct 27, 2023

$each($merge([metrics.fields]), function($value, $key) {{ "key": $key, "value": $value }}) ~> $reduce(function($acc, $entry) {( $contains($entry.key, "__") ? ( $outerLevelKey := $split($entry.key, "__")[0]; $innerLevelKey := $split($entry.key, "__")[1]; $merge([$acc, { $outerLevelKey: $merge([ {"key": $lookup($acc, $outerLevelKey)}.key, { $innerLevelKey: $entry.value } ]) }]) ) : $merge([$acc, { $entry.key: $entry.value }]) )}, {})

the above works for your requirement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants