AWS CLI

Purpose

To manage AWS resources remotely with scripts.

Initial use case is to create and manage snapshots of EC2 instances. I want to create new snapshots then delete old ones. This could be managed internally with AWS, but it looks like that would cost more and I don't mind learning something new.

Process

Listing EC2 Instances

I need to list the instances and parse that list so I know what I am working with. Though not needed for creating snapshots, this was helpful in learning how aws-cli functions. I started with a serverfault post and broke down the steps contained within. The example provided was

aws ec2 describe-instances --filters Name=vpc-id,Values=vpc-e2f17e8b --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]'

But the query statement didn't make much sense to me. At this point I started to replicate this in my lab. The key here is that you want to look at the output of the standard describe-instances command

{
    "Reservations": [
        {
            "Instances": [
                {
                    "Monitoring": {
                        "State": "disabled"
                    }, 
                    "PublicDnsName": "ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com", 
                    "StateReason": {
                        "Message": "Client.UserInitiatedShutdown: User initiated shutdown", 
                        "Code": "Client.UserInitiatedShutdown"
                    }, 
                    "State": {
                        "Code": 80, 
                        "Name": "stopped"
...
                    "InstanceId": "i-xxxxxxxxxxad6183c",
...

                    "Tags": [
                        {
                            "Value": "My-VM", 
                            "Key": "Name"
                        }
                    ], 

Since the information I need is nested, I will need to drill down. Starting with Reservations, then Instances, I can then select the information I need.
NOTE: This is case sensitive

[root@aws-cli ~]# aws ec2 describe-instances --query 'Reservations[].Instances[].{Instance_name:Tags[?Key==`Name`].Value,ID:InstanceId,State:State.Name,Volume:BlockDeviceMappings[].Ebs.VolumeId}'
[
    {
        "Instance_name": [
            "My-VM"
        ],
        "Volume": [
            "vol-xxxxxxxxxxxxxx142"
        ], 
        "State": "stopped", 
        "ID": "i-xxxxxxxxxxad6183c"
    }
] 

To break this down:

• "Reservations[]" This will query ALL reservations.
• "Instances[]" This will query ALL instances
• "{}" This is creating an array, since we want multiple values found inside of Instances.
• "Instance_name" is an arbitrary name, you can put anything you want here without spaces. There might be a way to use spaces, but you shouldn't use them anyway.
• ":Tags[?Key==`Name`].Value" I do not fully understand this as of yet. However I needed it to parse the human readable name I gave the instance.
• "ID" is an arbitrary name.
• ":InstanceId" will pull the instance ID.
• "State" is an arbitrary name.
• ":State.Name" will pull the human readable state of the instance. In this case "Stopped".
• "Volume" is an arbitrary name.
• ":BlockDeviceMappings[].Ebs.VolumeId" Will grab the VolumeIDs that we will need later.

To only list Instance IDs for processing

root@aws-cli ~]# aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId' | sed -e '/\[\|]/d; s/^\s\+//g' | awk 'BEGIN {FS="\""} {print $2}'
i-xxxxxxxxxxad6183c
i-xxxxxxxxxxad6345c

To grab the associated volume IDs

[root@aws-cli ~]# aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId' | sed -e '/\[\|]/d; s/^\s\+//g' | awk 'BEGIN {FS="\""} {print $2}' | while read line; do aws ec2 describe-instances --instance-ids "$line" --query 'Reservations[].Instances[].BlockDeviceMappings[].Ebs.VolumeId' | sed -e '/\[\|]/d; s/^\s\+//g' | awk 'BEGIN {FS="\""} {print $2}'; done
vol-xxxxxxxxxxxxxx142
vol-xxxxxxxxxxxxxxc0f