You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Kestra relies on JSON structured data (e.g., Amazon ION) for exchanging outputs among flow tasks. Frequently, data processing entails applying transformations to JSON records (e.g., formatting, selection, aggregation).
Presently, achieving these transformations entails using the FileTransform task, which necessitates scripting in Javascript, Python, or Groovy. However, this approach isn't optimal for basic transformations and is not very user-friendly
Proposition
Introduce a dedicated task in Kestra for JSON manipulation, leveraging solutions like JSONata. This would empower users to perform data transformations efficiently using a powerful querying language, enhancing workflow simplicity and flexibility.
Flow Example
For example, using the following Flow:
id: examplenamespace: exampletasks:
- id: transformJsontask: io.kestra.plugin.task.JSONataTransform# can be either a Kestra URI or a STRINGfrom: {{ previousTask.outputs.uri }}expr: | { "order_id": order_id, "customer_name": customer_name, "total_price": $sum(items.(quantity * price_per_unit)) }
Feature description
Motivation
Kestra relies on JSON structured data (e.g., Amazon ION) for exchanging outputs among flow tasks. Frequently, data processing entails applying transformations to JSON records (e.g., formatting, selection, aggregation).
Presently, achieving these transformations entails using the FileTransform task, which necessitates scripting in Javascript, Python, or Groovy. However, this approach isn't optimal for basic transformations and is not very user-friendly
Proposition
Introduce a dedicated task in Kestra for JSON manipulation, leveraging solutions like JSONata. This would empower users to perform data transformations efficiently using a powerful querying language, enhancing workflow simplicity and flexibility.
Flow Example
For example, using the following Flow:
with the given JSON input:
will output:
The text was updated successfully, but these errors were encountered: