base64 encode/decode of opaque binary data

[tis-rpage]

Please add support for base64 encoding/decoding of opaque binary data as documented in the YAML draft standard:
https://yaml.org/type/binary.html

yq already supports these conversion encodings:
Format | Decode (from string) | Encode (to string)
Yaml | from_yaml | to_yaml(i)/@yaml
JSON | from_json | to_json(i)/@JSON
Properties | | to_props/@props
CSV | | to_csv/@csv
TSV | | to_tsv/@TSV
XML | from_xml | to_xml(i)/@xml

Recomendation:
Base64 | from_base64 | to_base64(i)/@base64

[mikefarah]

Just thinking how this would work, I can imagine encoding doing something like:

yq '.image_data = load_as_base64("myimage.png")

Which would load the image file and encode it into the yaml as base64 string into the image_data node.

For decoding - it gets a little stranger as it doesn't really make sense (as far as I can tell) to have decoded binary data in an expression - that is, you can't really pipe it into another operator and make sense of it.

Not sure how that would work. Maybe its an output format option like:

yq '.image_data' -o=decode_base64 > image.png

thoughts? What are you trying to do with the binary data?

[tis-rpage]

For most use-cases, I expect base64 decode to be more valuable as it enables decoding any ASCII armored text for reuse in assigning to other fields in a YAML document. Base64 encode with kubernetes is valuable because all of the configmap and secret values are supposed to be base64 encoded, which is a natural fit for encoding secrets/credentials that might have binary content. Some folks may just want base64 encode as a light-weight obfuscation mechanism.

Two real world examples that would benefit are my use of kubernetes to decode configmap and secret values and with hashicorp vault for encoding data in friendly yaml to output to json for vault to ingest.

Example usage of jq in-place of yq due to this missing feature:
kubectl get secret credential -o json | jq -r '.data[] | @base64d'
so I'd want to be able to do this:
kubectl get secret credential -o yaml | yq e '.data[] | @base64d' -

Dealing with x.509 certificates is another common use case where PEM content needs to be base64 encoded, and if there's raw DER somewhere, it'd be nice to be able to generate the PEM from a raw DER without shelling to openssl.

So perhaps the following might become a thing with using hashicorp vault for storing secrets:
yq '.x509.cert = pem_header(load_as_base64("mycert.der"))' -o json | vault kv write secret/x509/mycert.pem @/dev/stdin
where pem_header() would represent some external mechanism for generating the PEM header/footer blocks and is outside the scope of this request for base64 encode/decode support.

[baurmatt]

Also interested in this functionality. Mostly interested to work with Kubernetes Secrets.

[mikefarah]

Cheers - I'll work on this in the next release.

[mikefarah]

Released in 4.21.1

[mbigras]

Thanks for implementing! This @base64d Base64 decode operator helped me decode Kubernetes secrets like the following example illustrates—also see https://mikefarah.gitbook.io/yq/operators/encode-decode.

$ kubectl get secret mysecret -o yaml
apiVersion: v1
data:
  PASSWORD: MWYyZDFlMmU2N2Rm
  USER_NAME: YWRtaW4=
kind: Secret
# ...
$ kubectl get secret mysecret -o yaml | yq '.data | map_values(. | @base64d)'
PASSWORD: 1f2d1e2e67df
USER_NAME: admin

This is more ergonomic than the copy–paste suggestion in the Decode the Secret Kubernetes documentation and also avoids the gotcha where you paste your secret in your shell history.