Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(templating)!: return json, not go structs #12909

Merged
merged 13 commits into from
Jun 13, 2024
Merged
22 changes: 22 additions & 0 deletions docs/fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ Workflow is the definition of a workflow resource

- [`exit-handlers.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handlers.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -565,6 +567,8 @@ WorkflowSpec is the specification of a Workflow.

- [`exit-handlers.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handlers.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -1004,6 +1008,8 @@ CronWorkflowSpec is the specification of a CronWorkflow

- [`exit-handlers.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handlers.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -1333,6 +1339,8 @@ Arguments to a template

- [`exit-handler-with-param.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handler-with-param.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -2131,6 +2139,8 @@ Parameter indicate a passed string parameter to a service template with an optio

- [`exit-handler-with-param.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handler-with-param.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -2677,6 +2687,8 @@ Inputs are the mechanism for passing parameters, artifacts, volumes from one tem

- [`exit-handler-with-param.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handler-with-param.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -2878,6 +2890,8 @@ ScriptTemplate is a template subtype to enable scripting through code steps

- [`exit-handler-with-param.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handler-with-param.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -3909,6 +3923,8 @@ DataSource sources external data into a data template

- [`exit-handler-with-param.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handler-with-param.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -4659,6 +4675,8 @@ ObjectMeta is metadata that all persisted resources must have, which includes al

- [`exit-handlers.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handlers.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -5572,6 +5590,8 @@ EnvVar represents an environment variable present in a Container.

- [`colored-logs.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/colored-logs.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down Expand Up @@ -5990,6 +6010,8 @@ PersistentVolumeClaimSpec describes the common attributes of storage devices and

- [`exit-handlers.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/exit-handlers.yaml)

- [`expression-destructure-json-complex.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json-complex.yaml)

- [`expression-destructure-json.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-destructure-json.yaml)

- [`expression-reusing-verbose-snippets.yaml`](https://github.com/argoproj/argo-workflows/blob/main/examples/expression-reusing-verbose-snippets.yaml)
Expand Down
36 changes: 36 additions & 0 deletions examples/expression-destructure-json-complex.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: expression-destructure-json-complex-
spec:
arguments:
parameters:
- name: config
value: '{"employees": [{"name": "Baris", "age":43},{"name": "Mo", "age": 42}, {"name": "Jai", "age" :44}]}'
entrypoint: main
templates:
- name: main
inputs:
parameters:
- name: a
value: "{{=jsonpath(workflow.parameters.config, '$.employees[?(@.name==\"Baris\")].age')}}"
- name: b
value: "{{=jsonpath(workflow.parameters.config, '$.employees[?(@.age>42 && @.age<44)].age')}}"
- name: c
value: "{{=jsonpath(workflow.parameters.config, '$.employees[0:1]')}}"
- name: d
value: "{{=jsonpath(workflow.parameters.config, '$.employees[*].name')}}"
script:
env:
- name: A
value: "{{inputs.parameters.a}}"
- name: B
value: "{{inputs.parameters.b}}"
- name: C
value: "{{inputs.parameters.c}}"
- name: D
value: "{{inputs.parameters.d}}"
image: debian:9.4
command: [bash]
source: |
echo "$A$B$C$D"
13 changes: 10 additions & 3 deletions util/template/expression_template.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"fmt"
"io"
"os"
"strconv"
"strings"

"github.com/doublerebel/bellows"
Expand Down Expand Up @@ -69,7 +70,7 @@ func expressionReplace(w io.Writer, expression string, env map[string]interface{
if result == nil {
return 0, fmt.Errorf("failed to evaluate expression %q", expression)
}
resultMarshaled, err := json.Marshal(fmt.Sprintf("%v", result))
resultMarshaled, err := json.Marshal(result)
if (err != nil || resultMarshaled == nil) && allowUnresolved {
log.WithError(err).Debug("resultMarshaled is nil and unresolved is allowed ")
return w.Write([]byte(fmt.Sprintf("{{%s%s}}", kindExpression, expression)))
Expand All @@ -80,9 +81,15 @@ func expressionReplace(w io.Writer, expression string, env map[string]interface{
if resultMarshaled == nil {
return 0, fmt.Errorf("failed to marshal evaluated marshaled expression %q", expression)
}
// Trim leading and trailing quotes. The value is being inserted into something that's already a string.
marshaledLength := len(resultMarshaled)
return w.Write(resultMarshaled[1 : marshaledLength-1])

// Trim leading and trailing quotes. The value is being inserted into something that's already a string.
if len(resultMarshaled) > 1 && resultMarshaled[0] == '"' && resultMarshaled[marshaledLength-1] == '"' {
return w.Write(resultMarshaled[1 : marshaledLength-1])
}

resultQuoted := []byte(strconv.Quote(string(resultMarshaled)))
return w.Write(resultQuoted[1 : len(resultQuoted)-1])
}

func EnvMap(replaceMap map[string]string) map[string]interface{} {
Expand Down
21 changes: 21 additions & 0 deletions util/template/template_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,4 +65,25 @@ func Test_Template_Replace(t *testing.T) {
})
}
})

t.Run("ExpressionWithJsonPath", func(t *testing.T) {
testCases := map[string]struct {
input, want string
}{
"ExprNumArrayOutput": {input: `{{=jsonpath('{"employees": [{"name": "Baris", "age":43, "friends": ["Mo", "Jai"]}, {"name": "Mo", "age": 42, "friends": ["Baris", "Jai"]}, {"name": "Jai", "age" :44, "friends": ["Baris", "Mo"]}]}', '$.employees[*].name')}}`, want: "[\"Baris\",\"Mo\",\"Jai\"]"},
"ExprStringArrayOutput": {input: `{{=jsonpath('{"employees": [{"name": "Baris", "age":43, "friends": ["Mo", "Jai"]}, {"name": "Mo", "age": 42, "friends": ["Baris", "Jai"]}, {"name": "Jai", "age" :44, "friends": ["Baris", "Mo"]}]}', '$.employees[0].friends')}}`, want: "[\"Mo\",\"Jai\"]"},
"ExprSimpleObjectOutput": {input: `{{=jsonpath('{"employees": [{"name": "Baris", "age":43},{"name": "Mo", "age": 42}, {"name": "Jai", "age" :44}]}', '$.employees[0]')}}`, want: "{\"age\":43,\"name\":\"Baris\"}"},
"ExprObjectArrayOutput": {input: `{{=jsonpath('{"employees": [{"name": "Baris", "age":43},{"name": "Mo", "age": 42}, {"name": "Jai", "age" :44}]}', '$')}}`, want: "{\"employees\":[{\"age\":43,\"name\":\"Baris\"},{\"age\":42,\"name\":\"Mo\"},{\"age\":44,\"name\":\"Jai\"}]}"},
"ExprArrayInObjectOutput": {input: `{{=jsonpath('{"employees": [{"name": "Baris", "age":43, "friends": ["Mo", "Jai"]}, {"name": "Mo", "age": 42, "friends": ["Baris", "Jai"]}, {"name": "Jai", "age" :44, "friends": ["Baris", "Mo"]}]}', '$.employees[0]')}}`, want: "{\"age\":43,\"friends\":[\"Mo\",\"Jai\"],\"name\":\"Baris\"}"},
}

for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
tmpl := SimpleValue{Value: tc.input}
newTmpl := processTemplate(t, tmpl, map[string]string{})
assert.Equal(t, tc.want, newTmpl.Value)
})
}
})

}
Loading