Changing value of a json object in a yaml file

[nicodmt]

Hi,

I have a config.yaml file with a json object within that I need to change a value during YQ task run in pipeline.
I want the key "currentVersion" to get it's value updated but for some reason I just can't access the "config.json" object and the task creates a new field at the end (see config.yaml example below, the last three rows).

In my pipeline.yaml I left some of the expressions I've tried to work with.
ChatGPT was unfortunately not of any help and the documentation on here didn't help.

Here follows some simplified code samples.

config.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: ...
  name: app-config
data:
  config.json: |-
    {
      ... ,
      "currentVersion": "value",
     ...
    }
  config:
    json:
      currentVersion: test15

pipeline.yaml:

    - name: update-version-tag
      params:
        - name: files
          value:
          - ./kustomize/overlays/$(params.project)/app-config.yaml
        - name: expression
          value: '.data."config.json".currentVersion = \"$(params.tag)"\'
          #value: '.data."config.json" |= . + {"currentVersion": "$(params.tag)"}'
          #value: '.data."config.json" |= (. | split("\n")[:-1] + ["  \"currentVersion\": \"$(params.tag)\",", "  }"] | join("\n"))'
          #value: '.data."config.json" |= (input | .currentVersion = "$(params.tag)"; .)'
          #value: '.data."config.json" |= sub("(.*\"currentVersion\": \")[^\"]+(\".*)"; "\\1$(params.tag)\\2")'
          #value: '.data."config.json" |= sub("(.*\"currentVersion\": \")[^\"]+(\".*)"; "\\1" + "$(params.tag)" + "\\2")'
          #value: 'select(.data == "want-to-update").foo |= ( fromjson | .a = "updated a" | tojson )' sample.yaml 
          #value: 'select (.data."config.json") |= ( fromjson | .currentVersion = $(params.tag) | tojson)'
      taskRef:
        name: yq
        kind: Task
      workspaces:
        - name: ...

and finally the YQ task itself:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: "yq"
  labels:
    app.kubernetes.io/version: "0.3"
  annotations:
    tekton.dev/pipelines.minVersion: "0.12.1"
    tekton.dev/categories: "Developer Tools"
    tekton.dev/tags: "yq"
    tekton.dev/displayName: "YQ replace"
    tekton.dev/platforms: "linux/amd64"
spec:
  description: >-
    This task can be used to replace fields in YAML files. For example for altering helm charts on GitOps repos.
  workspaces:
    - name: source
      description: A workspace that contains the file which needs to be altered.
  params:
    - name: files
      type: array
      description: A list of files to execute the expression on. Needs to be relative to the source workspace.
    - name: expression
      type: string
      description: The yq expression to apply. Can be used to replace yaml fields.
    - name: image
      type: string
      description: The yq image to use.
      default: docker.io/mikefarah/yq:4.16.2@some_sha_numbers
  steps:
    - name: replace-in-yaml
      image: $(params.image)
      workingDir: $(workspaces.source.path)
      args: ["$(params.files[*])"]
      script: |
        set -e
        for var in "$@"
        do
            /usr/bin/yq eval -i "$(params.expression)" "$var"
        done
        echo "$(params.expression)"
      securityContext:
        runAsUser: 0

[mikefarah]

So the expression you need is something like this:

yq '.data["config.json"] |= (@jsond | .currentVersion = "cat" | @json)' file.yaml

Explanation:

• From the yaml document perspective, you want to update the .data["config.json"] field. Note that because the key "config.json" has a dot in it, we have to use the bracket notation.
• We want to update the string encoded JSON data. So first we decode it has JSON @jsond, then do the updates required, and finally convert it back to json.

Hope that's clear!

[mikefarah]

See https://mikefarah.gitbook.io/yq/operators/encode-decode as well

[toopsy]

Thanks for the discussion here. I have been trying to accomplish something similar. In the following yaml I would like to change the value for priority in the json.

nrf_client:
  product_client:
    site: abc
  nrfs:
  - name: west_nrf
    geo_region: WEST_NRF
    endpoints:
    - http://1.2.3.4:80
    - http://1.2.3.5:80
    - http://1.2.3.6:80
    profiles:
    - udr_abc_west_nrf.json
  profiles:
  - name: udr_abc_west_nrf
    json: |
      {
        "fqdn": "abc.udr.5gc.mncX.mccY.3gppnetwork.org",
        "ipv4Addresses": [
          "1.4.3.2"
        ],
        "nfInstanceId": "5e094b8a-3df6-4ece-8796-7502f5f9db94",
        "locality": "abc",
        "nfServices": [
          {
            "fqdn": "abc.udr.5gc.mncX.mccY.3gppnetwork.org",
            "ipEndPoints": [
              {
                "ipv4Address": "1.4.3.2",
                "port": 8001
              }
            ],
            "nfServiceStatus": "REGISTERED",
            "scheme": "http",
            "serviceInstanceId": "5e094b8a-3df6-4ece-8796-7502f5f9db94",
            "serviceName": "nudr-dr",
            "versions": [
              {
                "apiFullVersion": "2.0.0.alpha-1",
                "apiVersionInUri": "v2"
              }
            ]
          }
        ],
        "nfStatus": "REGISTERED",
        "nfType": "UDR",
        "priority": 42,
        "udrInfo": {
          "supiRanges": [
            {
              "pattern": "^imsi-\\d{15}$"
            }
          ]
        }
      }

Based on the available documentation I was able to print the value

yq '.nrf_client.profiles[] | select (.name == "udr_abc_west_nrf").json | fromjson | .priority' prod_short_values-nrf.yaml
42

Updating this value without changing the overall yaml has proven more difficult. I tried to apply several different approaches that I have found here, but none have worked out for me. Probably overlooking something, so I would appreciate any pointers.

Going with the approach shown here, I ended up with this:

yq '.nrf_client.profiles[]|select(.name == "udr_abc_west_nrf").json |= (@jsond |.priority = 33 | @json)' prod_short_values-nrf.yaml
name: udr_abc_west_nrf
json: |-
  {"fqdn":"abc.udr.5gc.mncX.mccY.3gppnetwork.org","ipv4Addresses":["1.4.3.2"],"nfInstanceId":"5e094b8a-3df6-4ece-8796-7502f5f9db94","locality":"abc","nfServices":[{"fqdn":"abc.udr.5gc.mncX.mccY.3gppnetwork.org","ipEndPoints":[{"ipv4Address":"1.4.3.2","port":8001}],"nfServiceStatus":"REGISTERED","scheme":"http","serviceInstanceId":"5e094b8a-3df6-4ece-8796-7502f5f9db94","serviceName":"nudr-dr","versions":[{"apiFullVersion":"2.0.0.alpha-1","apiVersionInUri":"v2"}]}],"nfStatus":"REGISTERED","nfType":"UDR","priority":33,"udrInfo":{"supiRanges":[{"pattern":"^imsi-\\d{15}$"}]}}

Which seems fine on the surface, but... it did not actually change the value in the yaml:

cat prod_short_values-nrf.yaml | grep priority
        "priority": 42,

Tried to replicate the approach taken in issue #181 , but that was not successful, presumably because that issue was from back in 2018 :)
Tried a variety of approaches that involved "-i", no luck there either (yq -i '.nrf_client.profiles[]|select(.name == "udr_abc_west_nrf").json |= (@jsond |.priority = 33 | @json)' prod_short_values-nrf.yaml overwrites the entire yaml file with just the json - with the desired priority, but still missing the mark :)

Long story short: any help would be much appreciated!