Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to describe instances from all regions? #1777

Closed
ApproximateIdentity opened this issue Feb 8, 2016 · 18 comments
Closed

How to describe instances from all regions? #1777

ApproximateIdentity opened this issue Feb 8, 2016 · 18 comments

Comments

@ApproximateIdentity
Copy link

@ApproximateIdentity ApproximateIdentity commented Feb 8, 2016

I'm trying to switch my workflow over to the cli from the aws control panel, but I've run into a very odd issue. I know how to describe instances within a region, but how do I describe all of my instances irrespective of regions? Or at least get an output of all ec2 regions so I can do this programmatically one by one (i.e. something like aws ec2 list-regions).

It's very odd because the cli seems to use different names for regions than the control panel. For one, apparently all of my instances are being started in region "us-west-2" (at least so far), but I cannot find that string anywhere in the control panel. In fact, I can't seem to find anywhere listing my default region at all.

Is there a command inside of aws that does this?

@JordonPhillips
Copy link
Member

@JordonPhillips JordonPhillips commented Feb 8, 2016

The console doesn't allow you to list all instances irrespective of region, but it does hide the region identifiers with more descriptive text. us-west-2 is the datacenter located in Oregon, for instance. There's a drop-down in the top right of the console page that lets you select a region (pictured below).

For most services, there is not currently a programmatic way of discovering which regions are available, and so you will need to refer to this list. EC2 is one of the exceptions, however, as it provides the describe-regions function. Since the CLI is a wrapper over service APIs, we can't do multi-region searches unless the service supports it (S3, for instance, is mostly region-agnostic). You will have to provide the region with the --region argument to query in a different region.

Your default region is located in your configuration file (~/.aws/config on UNIX systems), and can be both shown and set with the aws configure command.

regions

@nonbeing
Copy link

@nonbeing nonbeing commented Sep 9, 2016

For people arriving here from Google, here's one way to list all your instances across all regions with the AWS CLI (based on the discussion above):

for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region
done
@Fitblip
Copy link

@Fitblip Fitblip commented Mar 5, 2017

For those of you who also don't want to parse JSON with your eyeballs, here's an appended version that uses jq to give you a bit more-to-the-point summary.

for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region | jq '.Reservations[] | ( .Instances[] | {state: .State.Name, name: .KeyName, type: .InstanceType, key: .KeyName})'
done

Example output:

Listing Instances in region:'us-west-1'...
{
  "state": "stopped",
  "name": "scanner-1",
  "type": "t2.micro",
  "key": "scan1"
}
{
  "state": "stopped",
  "name": "cors",
  "type": "t2.micro",
  "key": "cors"
}
{
  "state": "stopped",
  "name": "scanner-2",
  "type": "t2.medium",
  "key": "scan2"
}
@JordonPhillips
Copy link
Member

@JordonPhillips JordonPhillips commented Mar 6, 2017

@nonbeing Nice! If you use that often, you should consider making it an alias.

@pcgeek86
Copy link

@pcgeek86 pcgeek86 commented Jun 16, 2017

Another option would be to use the AWS PowerShell .NET Core module, which runs natively on Linux, Mac, and Windows.

foreach ($Region in (Get-AWSRegion)) {
  Get-EC2Instance -Region $Region.Region
}

Results

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

GroupNames    : {}
Groups        : {}
Instances     : {asdf}
OwnerId       : 123
RequesterId   : 123
ReservationId : r-asdf

Output using Format-Table

By default, in the current version of the AWS PowerShell module, the EC2 instances are returned in the PowerShell "list" format. However, this is easily changed to tabular formatting, using the Format-Table command.

$InstanceList = @()
foreach ($Region in (Get-AWSRegion)) {
  $InstanceList += Get-EC2Instance -Region $Region.Region -ProfileName CIDBTest
}
$InstanceList | Format-Table

Results

GroupNames Groups Instances   OwnerId      RequesterId  ReservationId       RunningInstance
---------- ------ ---------   -------      -----------  -------------       ---------------
{}         {}     {asdf222}   123          123                  r-123               {asdf222}
{}         {}     {asdf222}   123          123                  r-123               {asdf222}
{}         {}     {asdf222}   123          123                  r-123               {asdf222}
@jimdigriz
Copy link

@jimdigriz jimdigriz commented Jun 25, 2017

You could use jq, if you like adding more dependencies, or just use the built in --query in AWS CLI:

aws --profile whatever ec2 describe-instances --query 'Reservations[].Instances[].[State.Name,InstanceType]'
[
    [
        "running", 
        "t2.small"
    ], 
    [
        "running", 
        "t2.small"
    ], 
    [
        "running", 
        "m3.medium"
    ]
]
@ygeo
Copy link

@ygeo ygeo commented Aug 8, 2017

while read -r r ; echo start region : $r ; do while read -r i ; do echo instance-id: $i 2>&1 ; done < <(aws ec2 describe-instances --query "Reservations[].Instances[].InstanceId" --region $r --profile prd|perl -nl -e 's/\s+/\n/g;print'); echo "stop region: $r" ; done < <(aws ec2 describe-regions --output text --profile prd| cut -f 3)

@Selimcse98
Copy link

@Selimcse98 Selimcse98 commented Dec 7, 2017

Include tags value to better understand your EC2 instances


for region in `aws ec2 describe-regions --output text | cut -f3`
do
     echo -e "\nListing Instances in region:'$region'..."
     aws ec2 describe-instances --region $region | jq '.Reservations[] | ( .Instances[] | {state: .State.Name, name: .KeyName, Tag_Name: .Tags[].Value,type: .InstanceType, key: .KeyName})'
done

@Ahmed-AG
Copy link

@Ahmed-AG Ahmed-AG commented Feb 13, 2018

Just another solution I just wrote!
This small script can execute commands across all regions
Usage examples:
sudo ./aws-x.sh ec2 describe-instances
sudo ./aws-x.sh ec2 describe-instances | grep PrivateDnsName

#!/bin/bash

echo "Getting AWS Avaiable Regions:"
sudo aws ec2 describe-regions --region eu-west-2 |grep RegionName |cut -d '"' -f 4 > /tmp/regions.txt
cat /tmp/regions.txt

#Start executing
while read region; do
echo "Executing " $@ " on " $region ":"
aws --region $region $@
done</tmp/regions.txt

#cleanup
rm /tmp/regions.txt

@nlsko
Copy link

@nlsko nlsko commented Aug 19, 2018

#!/bin/bash
echo Grabbing instances in all regions, please wait..
for region in aws ec2 describe-regions --output text | cut -f3
do
aws ec2 describe-instances --region $region --query 'Reservations[].Instances[].[State.Name,InstanceType,PublicIpAddress,Placement.AvailabilityZone]' --output text
done

#------output-------

#./get-instances.sh
#Grabbing instances in all regions, please wait..
#running t2.micro 54.93.120.174 eu-central-1b
#running t2.micro 54.86.91.202 us-east-1d

@obijan42
Copy link

@obijan42 obijan42 commented Dec 6, 2018

For those of you who also don't want to parse JSON with your eyeballs, here's an appended version that uses jq to give you a bit more-to-the-point summary.

aws cli supports the "query" parameter, which does this stuff as a build-in.

@Ahmed-AG
Copy link

@Ahmed-AG Ahmed-AG commented Dec 6, 2018

Thanks obijan! I I started using jq few months back and it changed my life :)

@ajorpheus
Copy link

@ajorpheus ajorpheus commented Jun 3, 2019

Note that the AWS Cli Docs do still recommend JQ for more advanced usage:

If you need more advanced features that might not be possible with --query, you can check out jq, a command line JSON processor. You can download it and find the official tutorial at http://stedolan.github.io/jq/

IMHO, --query is still worth looking if it works for your query use case since it is natively supported by AWS CLI. This option was added in aws cli v. 1.2.0 which was released on 2013/10/18, so you should have definitely have this option unless you have an ancient version of the CLI.

If you prefer to use the --query param, note that it uses JMESPath as its query language (the home page has a live JMESPath tester). The tutorial is worth a gander as well.


Sample Comparison of a similar query

The following queries list the RDS instances in the default region using the aws cli, parsing the json output from it using JQ and JMESPath

For example, a similar query seems to be far more readable OOTB in JMESPath:

# Using Jquery
aws  rds describe-db-instances --output json | jq -r '.DBInstances[] | "\(.DBInstanceIdentifier) -> \(.DBInstanceStatus) ( \(.DBInstanceClass), \(.Endpoint.Address):\(.Endpoint.Port)  ) "  '

## Outputs
someinstancename -> available ( db.t2.small, xxxxx.rds.amazonaws.com:1234 )
## Show a list of properties for DBInstances which have a status containing the word 'avail' 
aws rds describe-db-instances --output json --query "DBInstances[?contains(DBInstanceStatus, 'avail')].[DBInstanceIdentifier, DBInstanceClass, DBInstanceStatus, Endpoint.[Address, Port]]"

## Outputs
[
    [
        "someinstancename",
        "db.t2.small",
        "available",
        [
            "xxxxx.rds.amazonaws.com",
            1234
        ]
    ]
]
@PatMyron
Copy link

@PatMyron PatMyron commented Aug 9, 2019

If opt-in-not-required is the 3rd column of output when you run aws ec2 describe-regions --output text, you can change cut -f3 to cut -f4 like so:

for region in `aws ec2 describe-regions --output text | cut -f4`
do

done
@coltoncat
Copy link

@coltoncat coltoncat commented Aug 14, 2019

Probably a more robust way is to use a JMESPath query as follows:

for region in $(aws ec2 describe-regions --query "Regions[*].RegionName" --output text)
do

done
@genyrosk
Copy link

@genyrosk genyrosk commented Oct 11, 2019

You could also do some further parsing to filter to the regions that you want with grep.
For instance, after installing awscli-aliases, add the following to ~/.aws/cli/alias to list your instances:

list-instances =
  !f() {
    echo "Filtering by \`$1\`"
    for region in $(aws ec2 describe-regions --query "Regions[*].RegionName" --output text | tr "\t" "\n" | grep "$1")
      do
        echo "\nListing Instances in region:'$region'..."
        aws ec2 describe-instances --region $region \
          --output table \
          --query "Reservations[*].Instances[*].{
                    Instance:InstanceId,
                    Type:InstanceType,
                    AZ:Placement.AvailabilityZone,
                    KeyName:KeyName,
                    Name:Tags[?Key==\`Name\`]|[0].Value,
                    Project:Tags[?Key==\`project\`]|[0].Value,
                    IP:PublicIpAddress,
                    State:State.Name,
                    CPUcores:CpuOptions.CoreCount,
                    CPUThreads:CpuOptions.ThreadsPerCore
                }"
      done
  }; f

Usage:

$ aws list-instances

Apply a filter to the region code, e.g. us for US regions:

$ aws list-instances 'us'
@Johnjosh11
Copy link

@Johnjosh11 Johnjosh11 commented May 20, 2020

for region in aws ec2 describe-regions --output text | cut -f3
do
echo -e "\nListing Instances in region:'$region'..."
aws ec2 describe-instances --region $region | grep running > file.txt
cat file.txt | wc -l
done

image

@laidig
Copy link

@laidig laidig commented Jan 13, 2021

The output format from aws ec2 describe-regions has changed slightly. If it doesn't work for you, try cut -f4.

for region in `aws ec2 describe-regions --output text | cut -f4`;
  do 
     echo -e "\nListing Instances in region:'$region'...";
      aws ec2 describe-instances --region $region | jq '.Reservations[] | ( .Instances[] | {state: .State.Name, name: .KeyName, type: .InstanceType, key: .KeyName})' > all_instances ;
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet