New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I want to filter instances by matching a substring of a tag value #2206

Closed
DrStrangepork opened this Issue Sep 29, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@DrStrangepork

DrStrangepork commented Sep 29, 2016

I'm about ripping my hair out over this. All of my instances have a tag named "Data", and the value of "Data" one of the following: "foo=yes,bar=yes", "foo=yes,bar=no", "foo=no,bar=yes", "foo=no,bar=no", I want to run a --query which returns the InstanceId for any instance that has a tag "Data" value that contains "foo=yes". So something like

aws --region us-east-1 ec2 describe-instances --query 'Reservations[].Instances[].[?contains(Tags[?Key==`Data`].Value, `foo=yes` == `true`)].InstanceId'

But that doesn't work.

@JordonPhillips

This comment has been minimized.

Show comment
Hide comment
@JordonPhillips

JordonPhillips Sep 29, 2016

Member

That certainly is a tricky one! I managed it with the following expression:

"Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]][].InstanceId"

That's a long one, so I'm going to break it up, piece by piece. For starters, lets use this sample data:

{
    "Reservations": [{"Instances": [
        {
            "InstanceId": "id-target1",
            "Tags": [
                {"Key": "Data", "Value": "foo=yes,bar=no"},
                {"Key": "Name", "Value": "target"}
            ]
        },
        {
            "InstanceId": "id-target2",
            "Tags": [
                {"Key": "Data", "Value": "foo=yes,bar=yes"},
                {"Key": "Name", "Value": "target"}
            ]
        },
        {
            "InstanceId": "id-invalid1",
            "Tags": [
                {"Key": "Data", "Value": "foo=no,bar=no"},
                {"Key": "Name", "Value": "invalid"}
            ]
        },
        {
            "InstanceId": "id-invalid2",
            "Tags": [
                {"Key": "Data", "Value": "foo=no,bar=yes"},
                {"Key": "Name", "Value": "invalid"}
            ]
        }
    ]}]
}

First, we want to get the instances list, so we start with Reservations[].Instances[]:

[
  {
    "Tags": [
      {
        "Value": "foo=yes,bar=no",
        "Key": "Data"
      },
      {
        "Value": "target",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-target1"
  },
  {
    "Tags": [
      {
        "Value": "foo=yes,bar=yes",
        "Key": "Data"
      },
      {
        "Value": "target",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-target2"
  },
  {
    "Tags": [
      {
        "Value": "foo=no,bar=no",
        "Key": "Data"
      },
      {
        "Value": "invalid",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-invalid1"
  },
  {
    "Tags": [
      {
        "Value": "foo=no,bar=yes",
        "Key": "Data"
      },
      {
        "Value": "invalid",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-invalid2"
  }
]

Now we want the tags which have a Key with the value "Data": .Tags[?Key == 'Data'] for the full expression Reservations[].Instances[].Tags[?Key == 'Data'] and the result:

[
  [
    {
      "Value": "foo=yes,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=yes,bar=yes",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=no,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=no,bar=yes",
      "Key": "Data"
    }
  ]
]

Now we also want the value of that tag to contain "foo=yes" so we add the condition && contains(Value, 'foo=yes') for the full expression Reservations[].Instances[].Tags[?Key == 'Data' && contains(Value, 'foo=yes')] and the result:

[
  [
    {
      "Value": "foo=yes,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=yes,bar=yes",
      "Key": "Data"
    }
  ],
  [],
  []
]

Alright, now lets flatten that list with [] for the full expression Reservations[].Instances[].Tags[?Key == 'Data' && contains(Value, 'foo=yes')][] and the result:

[
  {
    "Value": "foo=yes,bar=no",
    "Key": "Data"
  },
  {
    "Value": "foo=yes,bar=yes",
    "Key": "Data"
  }
]

Alright, so now we have the tags we want, but what we want the full instance data. So lets move that tag filter back as a filter on instances. So the full expression is now Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]] with the result:

[
  [
    {
      "Tags": [
        {
          "Value": "foo=yes,bar=no",
          "Key": "Data"
        },
        {
          "Value": "target",
          "Key": "Name"
        }
      ],
      "InstanceId": "id-target1"
    },
    {
      "Tags": [
        {
          "Value": "foo=yes,bar=yes",
          "Key": "Data"
        },
        {
          "Value": "target",
          "Key": "Name"
        }
      ],
      "InstanceId": "id-target2"
    }
  ]
]

To explain a bit more, ? is checking for a truthy value. For those instances that don't have the proper tag, ?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][] will be return null, a falsy value.

Now all that's left to do is flatten that list and select the instance ids with [].InstanceId for the full expression "Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]][].InstanceId" and the result:

[
  "id-target1",
  "id-target2"
]

And we're done!

Member

JordonPhillips commented Sep 29, 2016

That certainly is a tricky one! I managed it with the following expression:

"Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]][].InstanceId"

That's a long one, so I'm going to break it up, piece by piece. For starters, lets use this sample data:

{
    "Reservations": [{"Instances": [
        {
            "InstanceId": "id-target1",
            "Tags": [
                {"Key": "Data", "Value": "foo=yes,bar=no"},
                {"Key": "Name", "Value": "target"}
            ]
        },
        {
            "InstanceId": "id-target2",
            "Tags": [
                {"Key": "Data", "Value": "foo=yes,bar=yes"},
                {"Key": "Name", "Value": "target"}
            ]
        },
        {
            "InstanceId": "id-invalid1",
            "Tags": [
                {"Key": "Data", "Value": "foo=no,bar=no"},
                {"Key": "Name", "Value": "invalid"}
            ]
        },
        {
            "InstanceId": "id-invalid2",
            "Tags": [
                {"Key": "Data", "Value": "foo=no,bar=yes"},
                {"Key": "Name", "Value": "invalid"}
            ]
        }
    ]}]
}

First, we want to get the instances list, so we start with Reservations[].Instances[]:

[
  {
    "Tags": [
      {
        "Value": "foo=yes,bar=no",
        "Key": "Data"
      },
      {
        "Value": "target",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-target1"
  },
  {
    "Tags": [
      {
        "Value": "foo=yes,bar=yes",
        "Key": "Data"
      },
      {
        "Value": "target",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-target2"
  },
  {
    "Tags": [
      {
        "Value": "foo=no,bar=no",
        "Key": "Data"
      },
      {
        "Value": "invalid",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-invalid1"
  },
  {
    "Tags": [
      {
        "Value": "foo=no,bar=yes",
        "Key": "Data"
      },
      {
        "Value": "invalid",
        "Key": "Name"
      }
    ],
    "InstanceId": "id-invalid2"
  }
]

Now we want the tags which have a Key with the value "Data": .Tags[?Key == 'Data'] for the full expression Reservations[].Instances[].Tags[?Key == 'Data'] and the result:

[
  [
    {
      "Value": "foo=yes,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=yes,bar=yes",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=no,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=no,bar=yes",
      "Key": "Data"
    }
  ]
]

Now we also want the value of that tag to contain "foo=yes" so we add the condition && contains(Value, 'foo=yes') for the full expression Reservations[].Instances[].Tags[?Key == 'Data' && contains(Value, 'foo=yes')] and the result:

[
  [
    {
      "Value": "foo=yes,bar=no",
      "Key": "Data"
    }
  ],
  [
    {
      "Value": "foo=yes,bar=yes",
      "Key": "Data"
    }
  ],
  [],
  []
]

Alright, now lets flatten that list with [] for the full expression Reservations[].Instances[].Tags[?Key == 'Data' && contains(Value, 'foo=yes')][] and the result:

[
  {
    "Value": "foo=yes,bar=no",
    "Key": "Data"
  },
  {
    "Value": "foo=yes,bar=yes",
    "Key": "Data"
  }
]

Alright, so now we have the tags we want, but what we want the full instance data. So lets move that tag filter back as a filter on instances. So the full expression is now Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]] with the result:

[
  [
    {
      "Tags": [
        {
          "Value": "foo=yes,bar=no",
          "Key": "Data"
        },
        {
          "Value": "target",
          "Key": "Name"
        }
      ],
      "InstanceId": "id-target1"
    },
    {
      "Tags": [
        {
          "Value": "foo=yes,bar=yes",
          "Key": "Data"
        },
        {
          "Value": "target",
          "Key": "Name"
        }
      ],
      "InstanceId": "id-target2"
    }
  ]
]

To explain a bit more, ? is checking for a truthy value. For those instances that don't have the proper tag, ?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][] will be return null, a falsy value.

Now all that's left to do is flatten that list and select the instance ids with [].InstanceId for the full expression "Reservations[].Instances[?Tags[?Key == 'Data' && contains(Value, 'foo=yes')][]][].InstanceId" and the result:

[
  "id-target1",
  "id-target2"
]

And we're done!

@DrStrangepork

This comment has been minimized.

Show comment
Hide comment
@DrStrangepork

DrStrangepork Sep 29, 2016

Wow, awesome! The Tags[?Key == 'Data' && contains(Value, 'foo=yes')] step is where I derailed. Thank you!

DrStrangepork commented Sep 29, 2016

Wow, awesome! The Tags[?Key == 'Data' && contains(Value, 'foo=yes')] step is where I derailed. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment