unknown anchor error when merging multiple files

[chr1shung]

Hi, my use case is merging 2 (or more) docker compose files and retrieve only the services list.
Consider the following simplified examples:

• basic.yml

x-common-env-variables: &common-variables
  BASIC_KEY: value

services:
  consul:
    image: consul:1.13.2
    container_name: consul
    environment:
      <<: *common-variables

• advance.yml

x-advance-env-variables: &advance-variables
  ADVANCE_KEY: value

services:
  consul:
    environment:
      <<: *advance-variables
  nginx:
    image: nginx:stable-alpine
    container_name: nginx
    environment:
      <<: *advance-variables

Here's what I've tried:

$ yq ea '. as $item ireduce ({}; . * $item )' basic.yml advance.yml | yq '.services | keys | ... comments=""'
Error: bad file '-': yaml: unknown anchor 'advance-variables' referenced

Any chance I can make this work ?
Thanks for the help

[mikefarah]

The issue is that the merge result has x-advance-env-variables at the bottom of the yaml file, which isn't valid (need to define anchors before you use them). This is because yq isn't smart enough to pull it up to the top - when merging files, it will take the 2nd file and merge into the 1st. New map keys get added to the bottom.

There are two options to fix it:

a) add an empty x-advance-env-variables to the first file (or, better, generic variables map that all variables get merged into). Nice thing about a 'variables' map in the first file is you don't need knowledge of 'x-advance-env-variables' outside of advance.yml.

or b) manually re-order the map after the merge like so:

yq ea '. as $item ireduce ({}; . * $item ) | pick(["x-advance-env-variables", keys | .[]] | unique)' data1.yaml data2.yaml | ./yq '.services | keys | ... comments=""'

fixed: clarified first option.

[chr1shung]

Thanks for the elaboration and solutions!