yaml merge Examples

1. Introduction
2. Rudimentary Example of Core Concepts
   1. Whole File Merge
   2. Document Fragment Merge
   3. CLI-Supplied Data
3. Setting Complex Values
   1. Add an Element to the End of an Array
   2. Add Multiple Array Elements
   3. Add a New Record to an Array-of-Hashes

This document is part of the body of knowledge about yaml-merge, one of the reference command-line tools provided by the YAML Path project.

Introduction

This page explores various real-world use-cases for the yaml-merge command-line tool.

Rudimentary Example of Core Concepts

Document: LHS.yaml

---
deep:
  hash:
    structure:
      key: value
      leave: intact

There are a few ways to change value to New Value. You could just use the yaml-set command rather than yaml-merge, like so: yaml-set --change=/deep/hash/structure/key --value='New Value' LHS.yaml

But what if you don't want LHS.yaml to be modified, or you want the changed document on STDOUT, say to feed into a subsequent piped command? This is where yaml-merge can help and there are multiple ways to solve these questions. These include:

1. Whole file merge where the RHS document contains the minimum data structure necessary to indicate where with the LHS document its data is to be merged.
2. File fragment merge where only the desired RHS data is written and a --mergeat (-m) indicates where within the LHS document this RHS data is to be merged.
3. CLI-supplied merge data where only the desired RHS data is fed to yaml-merge via STDIN and a --mergeat (-m) indicates where within the LHS document this RHS data is to be merged. The RHS data can be a proper YAML document, a proper JSON string, or a Python JSON-like string (without all the quoting of proper JSON).

The following sections demonstrate all three approaches.

Whole File Merge

You could create a complete RHS document as an entirely new YAML file, like this:

Document: RHS-FULL.yaml

---
deep:
  hash:
    structure:
      key: New Value

To merge these two documents together, you'd simply execute: yaml-merge LHS.yaml RHS-FULL.yaml

While this would generate the desired outcome, far more complex LHS structures could be not only tedious, but also error-prone to build up a sufficient RHS document to affect the desired alteration.

Document Fragment Merge

You could supply a document fragment for the RHS, like so:

Document: RHS-FRAGMENT.yaml

---
key: New Value

To merge the fragment into the whole, execute: yaml-merge --mergeat=/deep/hash/structure LHS.yaml RHS-FRAGMENT.yaml

CLI-Supplied Data

You could spare all versions of the RHS file and instead use STDIN to emulate a yaml-set call via yaml-merge to get the same outcome: echo 'New Value' | yaml-merge --mergeat=/deep/hash/structure/key LHS.yaml -

Notice that only the actual Scalar data was provided in this case because for this example, we were only trying to change a single String value. You could provide the key-value pair of the destination and shorten the merge target path, like so: echo '{key: New Value}' | yaml-merge --mergeat=/deep/hash/structure LHS.yaml -

And naturally, any amount of the RHS can be provided as long as the supplied YAML Path places it precisely where you want it to be merged. For example:

echo '{structure: {key: New Value}}' | \
  yaml-merge --mergeat=/deep/hash LHS.yaml -`

And so on.

Setting Complex Values

The yaml-set command-line tool is limited to writing a single Scalar value to one or more destinations in a data file. While its core purpose is to change existing data elements, it is capable of adding limited structure to the document in order to receive the new value whenever the specified YAML Path points to a nonexistent structure (assuming you've not set --mustexist). Whenever you need to set a complex Array or Hash data structure, use yaml-merge, instead. This is because writing complex data is, in fact, a merge of two documents; one of them happens to be a document fragment that is supplied via the command-line or STDIN.

The following are some examples of using yaml-merge instead of yaml-set for setting complex data in documents. Remember that you have the full capabilities of the yaml-merge command for all of these, granting users fine-grained control over the behavior of the operation.

Add an Element to the End of an Array

The yaml-set command can change the value of an element in an Array, but it cannot add a new element to one unless you explicitly specify the new element's index in your YAML Path. For convenience, use yaml-merge in such cases: echo "New Element" | yaml-merge --overwrite=new-element.yaml new-element.yaml

Before:

---
an_array:
  - element 0
  - element 1
  - element 2

After:

---
an_array:
  - element 0
  - element 1
  - element 2
  - New Element

You could accomplish the same using yaml-set but only if you already knew there were 3 elements at an_array: yaml-set --change=an_array[4] --value="New Element" new-element.yaml would produce exactly the same "After" document.

Add Multiple Array Elements

When you need to add more than one element to an Array in the same, single operation, you need only to pass a JSON, Python-JSON, or YAML document into yaml-merge along with the target document and its --mergeat destination. Consider: echo "[new 1, new 2, new 3]" | yaml-merge --mergeat=/an_array new-element.yaml

Before:

an_array:
  - element 0
  - element 1
  - element 2

After:

---
an_array:
  - element 0
  - element 1
  - element 2
  - new 1
  - new 2
  - new 3

Note that in this example, the "After" document was printed to STDOUT. In order to emulate the behavior of yaml-set -- actually change the source document -- simply instruct yaml-merge to --overwrite the input document, like so: echo "[new 1, new 2, new 3]" | yaml-merge --mergeat=/an_array --overwrite=new-element.yaml new-element.yaml

Add a New Record to an Array-of-Hashes

It is possible to add an entire record -- one key-value pair at a time -- to an Array-of-Hashes by using yaml-set. Doing so is however, tedious and human error-prone. When you need to add a whole record at once, use yaml-merge, instead.

In this case, we'll add a new record to the following data:

---
# Source: https://www.qmul.ac.uk/sbcs/iupac/AtWt/
atomic_weights:
  - atomic_number: 1
    symbol: H
    name: hydrogen
    weights:
      minimum: 1.00784
      maximum: 1.00811
  - atomic_number: 3
    symbol: Li
    name: lithium
    weights:
      minimum: 6.938
      maximum: 6.997
  - atomic_number: 5
    symbol: B
    name: boron
    weights:
      minimum: 10.806
      maximum: 10.821

Per the cited table, the next record is for carbon. We could add it with a single operation by using Python-JSON format via, echo "{atomic_number: 6, symbol: C, name: carbon, weights: {minimum: 12.0096, maximum: 12.0116}}" | yaml-merge --mergeat=atomic_weights atomic_weights.yaml. We could use regular JSON or YAML as well, but Python-JSON is simpler to work with on the command-line. This command produces:

---
atomic_weights:
  - atomic_number: 1
    symbol: H
    name: hydrogen
    weights:
      minimum: 1.00784
      maximum: 1.00811
  - atomic_number: 3
    symbol: Li
    name: lithium
    weights:
      minimum: 6.938
      maximum: 6.997
  - atomic_number: 5
    symbol: B
    name: boron
    weights:
      minimum: 10.806
      maximum: 10.821
  - atomic_number: 6
    symbol: C
    name: carbon
    weights:
      minimum: 12.0096
      maximum: 12.0116

As you can see, the entire new record was appended exactly where expected. We could apply an update to one of the existing records simply by adding --aoh=deep to the argument list and ensuring that our command-line-supplied record had an atomic_number matching the record-to-be-updated.

You may have noticed that the comment at the top of the file was stripped out. This is a side-effect of merging complex documents together wherein comments are treated as arbitrary text with no trivial means of combining. If you really must preserve the comments in the final document, use yaml-set, instead. However, adding the same record via yaml-set would require 6 operations, one per record field.