AWS CLI - Revision history

Michael.mast: /* Launch with startup bash script */

2018-07-10T19:16:59Z

Launch with startup bash script

Michael.mast: /* Launch with startup bash script */

2018-07-10T19:15:25Z

Launch with startup bash script

Michael.mast: /* Listing EC2 Instances */

2018-07-10T19:13:57Z

Listing EC2 Instances

Michael.mast at 18:59, 10 July 2018

2018-07-10T18:59:44Z

Michael.mast at 18:58, 10 July 2018

2018-07-10T18:58:47Z

Michael.mast: /* Restoring Snapshots */

2018-07-03T16:38:13Z

Restoring Snapshots

Michael.mast: /* Process */

2018-07-02T16:34:38Z

Process

Michael.mast: /* Creating Snapshots */

2018-05-02T20:24:42Z

Creating Snapshots

Michael.mast: /* Deleting Snapshots */

2018-05-02T20:23:34Z

Deleting Snapshots

Michael.mast: /* Process */

2018-05-02T20:18:36Z

Process

@@ Line 78: / Line 78: @@
 ==Launching Instances==
 ===Launch with startup bash script===
-*Create the shell script.
+*Create the shell script.<ref>https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-shell-scripts</ref>
 <pre>
 #!/bin/bash

@@ Line 78: / Line 78: @@
 ==Launching Instances==
 ===Launch with startup bash script===
-*Create the shell script. Then run the following as you normally would
+*Create the shell script.
 <pre>
 #!/bin/bash
@@ Line 120: / Line 120: @@
 </pre>
 <pre>
 [root@aws-cli]# aws ec2 run-instances --image-id ami-8c122be9 --count 1 --instance-type m5.large --key-name <your key name> --subnet-id <your subnet id> --security-group-ids <your security group> --instance-initiated-shutdown-behavior stop --user-data file://web.txt

@@ Line 75: / Line 75: @@
 vol-xxxxxxxxxxxxxx142
 vol-xxxxxxxxxxxxxxc0f
 </pre>

@@ Line 77: / Line 77: @@
 </pre>
-==Creating Snapshots==
+==Snapshots==
 According to Amazon<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/create-snapshot.html</ref>, you want to stop the instance before taking a snapshot to ensure the state is clean. We will see about that. If you have time for that go for it! Otherwise we will look at making snapshots of live root volumes.
 <br>
@@ Line 97: / Line 98: @@
 </pre>
-==Deleting Snapshots==
+===Deleting Snapshots===
 Since you have to pay for the storage of these snapshots, you probably want to delete old ones.<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/delete-snapshot.html</ref> We want to search through the snapshots using tags, then delete ones older than a set time. In my case I want to delete any older than two weeks. All my time stamps are in seconds making math easy.
 <pre>
@@ Line 112: / Line 113: @@
 done;
 </pre>
-==Restoring Snapshots==
+===Restoring Snapshots===
 The process for restoring a snapshot for a live system is
 *Create a volume from the snapshot.<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-snapshots.html</ref>

@@ Line 1: / Line 1: @@
-==Purpose==
+==Listing EC2 Instances==
 I need to list the instances and parse that list so I know what I am working with. Though not needed for creating snapshots, this was helpful in learning how aws-cli functions. I started with a serverfault post<ref>https://serverfault.com/questions/578921/how-would-you-go-about-listing-instances-using-aws-cli-in-certain-vpc-with-the-t</ref> and broke down the steps contained within. The example provided was
 <pre>aws ec2 describe-instances --filters Name=vpc-id,Values=vpc-e2f17e8b --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value[]'</pre>
@@ Line 83: / Line 77: @@
 </pre>
-===Creating Snapshots===
+==Creating Snapshots==
 According to Amazon<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/create-snapshot.html</ref>, you want to stop the instance before taking a snapshot to ensure the state is clean. We will see about that. If you have time for that go for it! Otherwise we will look at making snapshots of live root volumes.
 <br>
@@ Line 103: / Line 97: @@
 </pre>
-===Deleting Snapshots===
+==Deleting Snapshots==
 Since you have to pay for the storage of these snapshots, you probably want to delete old ones.<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/delete-snapshot.html</ref> We want to search through the snapshots using tags, then delete ones older than a set time. In my case I want to delete any older than two weeks. All my time stamps are in seconds making math easy.
 <pre>
@@ Line 118: / Line 112: @@
 done;
 </pre>
-===Restoring Snapshots===
+==Restoring Snapshots==
 The process for restoring a snapshot for a live system is
 *Create a volume from the snapshot.<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-snapshots.html</ref>

@@ Line 142: / Line 142: @@
 	echo "$1 not found. Check for typos"
 else
-	values=$(aws ec2 describe-instances --filter "Name=tag-value,Values=$1" --query 'Reservations[].Instances[].{IP:PublicIpAddress,VPC:VpcId,ID:InstanceId,Type:InstanceType,AZ:Placement.AvailabilityZone,Group:SecurityGroups[].GroupName,VOLID:BlockDeviceMappings[].Ebs.{ID:VolumeId}}')
+	echo "Gathering data"
 	ipaddress=$(grep -oE "\"IP\":\ \"[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\"" <<<$values | awk 'BEGIN {FS="\""}{print $4}')
 	type=$(grep -Eo "\"Type\": \"[a-z][0-9].[a-z]+\"" <<<$values | awk 'BEGIN {FS="\""}{print $4}')
-	az=$(grep -Eo "\"AZ\": \"[a-z0-9-]+\"" <<<$values| awk 'BEGIN {FS="\""}{print $4}')
+	az=$(grep -Eo "\"AZ\": \"us-[a-z0-9-]+\"" <<<$values| grep -Eo us-[a-z0-9-]+ | head -1)
-	security=$(grep -Eo "\"Group\":\ \[\ \"[A-Za-z]+\"\ \]" <<<$values | awk 'BEGIN {FS="\""}{print $4}')
+	security=$(grep -Eo sg-[0-9a-z]+ <<<$values)
 	id=$(grep -Eo "\"ID\":\ \"i-[0-9a-z]+\"" <<<$values | awk 'BEGIN {FS="\""}{print $4}')
 	volid=$(grep -Eo "\"VOLID\":\ \[\ \{\ \"ID\":\ \"vol-[0-9a-z]+\"" <<<$values | awk 'BEGIN {FS="\""}{print $6}')
+	imageid=$(grep -Eo ami-[0-9]+ <<<$values)
 #Search for last snapshot created for this site.
 	dayago=$(date +%Y-%m-%d --date=yesterday)
 	volvalues=$(aws ec2 describe-snapshots --filter "Name=volume-id,Values=$volid" "Name=start-time,Values=$dayago*" --query 'Snapshots[].{State:State,CreationDate:StartTime,SnapID:SnapshotId}')
-	snapid=$(grep -Eo "\"SnapID\":\ \"snap-[0-9a-z]+" <<<$volvalues | awk 'BEGIN {FS="\""}{print $4}')
+	snapid=$(grep -Eo "\"SnapID\":\ \"snap-[0-9a-z]+" <<<$volvalues | grep -Eo snap-[0-9a-z]+)
 #Create volume from snapshot
-	volvalues=$(aws ec2 create-volume --availability-zone $az --snapshot-id $snapid --volume-type gp2 --tag-specifications "ResourceType=volume,Tags=[{Key=Name,Value=$1}]")
+	echo "Creating volume from snapshot..."
 	volidnew=$(grep -Eo "\"VolumeId\":\ \"vol-[0-9a-z]+\"" <<<$volvalues | awk 'BEGIN {FS="\""}{print $4}')
@@ Line 164: / Line 170: @@
 	if [ -z "$volstatus" ]; then
 		echo "Can not determine volume creation status. Stopping."
 	else
 		until [ "$volstatus" = "available" ]; do
@@ Line 171: / Line 178: @@
 				exit 1
 			fi
 			sleep=3
 		done
 	fi
+#Re-lable / tag / name volumes and instances
 #Create instance for the new volume to use
 #Clear variables just to make sure we don't break something.
 unset ipaddress
 unset type
@@ Line 190: / Line 266: @@
 unset volidnew
 unset volstatus
 fi
 </pre>

@@ Line 117: / Line 117: @@
 fi;
 done;
 </pre>

@@ Line 93: / Line 93: @@
 #!/bin/bash
 now=$(date +%s)
-aws ec2 describe-instances --filter "Name=tag-value,Values=My_Tag" --query 'Reservations[].Instances[].{ID:InstanceId}' --output text | while read line;
+tag="My_Tag"
 do id=$line;
 vol=$(aws ec2 describe-instances --instance-ids "$id" --query 'Reservations[].Instances[].{Volume:BlockDeviceMappings[].Ebs.VolumeId}' --output text | awk '{print $2}');
 name=$(aws ec2 describe-instances --instance-ids "$id" --query 'Reservations[].Instances[].{Instance_name:Tags[?Key==`Name`].Value}' --output text | awk '{print $2}');
 snapid=$(aws ec2 create-snapshot --description "$name $id $now" --volume-id $vol | grep -oE snap-[0-9a-z]+)
-aws ec2 create-tags --resources $snapid --tags Key=function,Value=web Key=source,Value=$id Key=creation_date,Value=$now;
+aws ec2 create-tags --resources $snapid --tags Key=function,Value=$tag Key=source,Value=$id Key=creation_date,Value=$now;
 done
 </pre>
 ===Deleting Snapshots===
 Since you have to pay for the storage of these snapshots, you probably want to delete old ones.<ref>https://docs.aws.amazon.com/cli/latest/reference/ec2/delete-snapshot.html</ref> We want to search through the snapshots using tags, then delete ones older than a set time. In my case I want to delete any older than two weeks. All my time stamps are in seconds making math easy.