This playbook contains example of working with AWS. There are commands for
starting up infrakit (which assumes you have used AWS CLI on your local computer and
the .aws/credentials file exists) and commands for spinning up an on-demand or
spot instance configured with Docker, Git and Go compiler.
Adding files locally:
infrakit playbook add aws file://$(pwd)/index.yml
Adding the playbook from Github:
infrakit playbook add aws https://raw.githubusercontent.com/docker/infrakit/master/examples/playbooks/aws/index.yml
It's now possible to build a whole VPC using infrakit's resource controller. Warning: this is not production ready. You can't delete the resources with infrakit yet.
- Start up infrakit
$ infrakit use aws start
- In another terminal, let's watch some events
$ infrakit local resource tail / --view 'str://{{.Type}} - {{.ID}} - {{.Message}}'
- In another terminal, commit the spec to monitor resources as they are created:
$infrakit use aws inventory --plugin mystack/inventory
Before any resources are created, we expect to see no metadata:
$ infrakit local inventory/myproject keys -al
total 0:
- Set up project specific variables: this example uses the
metadataplugin to set some global variables. Run this to set them
$ infrakit use aws vars
Project? [myproject]:
CIDR block? [10.0.0.0/16]:
CIDR block? [10.0.100.0/24]:
CIDR block? [10.0.200.0/24]:
Availability Zone? [eu-central-1a]:
Availability Zone? [eu-central-1b]:
Proposing 0 changes, hash=b3e009daf23a4c248eb7a7003e778c78
{
"cidr": "10.0.0.0/16",
"project": "myproject",
"subnet1": {
"az": "eu-central-1a",
"cidr": "10.0.100.0/24"
},
"subnet2": {
"az": "eu-central-1b",
"cidr": "10.0.200.0/24"
}
}
Project is myproject
Proposing 0 changes, hash=b3e009daf23a4c248eb7a7003e778c78
{
"cidr": "10.0.0.0/16",
"project": "myproject",
"subnet1": {
"az": "eu-central-1a",
"cidr": "10.0.100.0/24"
},
"subnet2": {
"az": "eu-central-1b",
"cidr": "10.0.200.0/24"
}
}
- Commit the
mystack.ymlplaybook to the resource controller. This file has specs of all the resources and their dependencies in one place. The playbook also contains other commands to provision the resources individually (eg.infrakit use aws vpcwill provision just a vpc).
Once committed, The controller will try to reconcile and begin to provision the resources in the VPC. In this case it will provision these resources:
- The VPC (equivalent to running
infrakit use aws vpc) - Internet Gateway, with one route table and route to the internet through the gateway.
The standalone equivalent:
infrakit use aws gateway(provision-gateway.yml) followed byinfrakit use aws routetable(seeprovision-routetable.yml`). - Two subnets (see
provision-subnet.yml). - One security group (see
provision-securitygroup.yml)
Commit the file:
$ infrakit use aws vpc --plugin mystack/resource
Please enter your user name: [davidchung]:
Project? [myproject]:
CIDR block? [10.0.0.0/16]:
CIDR block? [10.0.100.0/24]:
CIDR block? [10.0.200.0/24]:
Availability Zone? [eu-central-1a]:
Availability Zone? [eu-central-1b]:
kind: resource
metadata:
id: myproject
name: myproject
tags: null
options:
ObserveInterval: 5s
properties:
igw:
Properties:
AttachInternetGatewayInput:
VpcId: '@depend(''vpc/Properties/VpcId'')@'
select:
Name: myproject-igw
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-internetgateway
rtb:
Properties:
CreateRouteInputs:
- DestinationCidrBlock: 0.0.0.0/0
GatewayId: '@depend(''igw/Properties/InternetGatewayId'')@'
CreateRouteTableInput:
VpcId: '@depend(''vpc/Properties/VpcId'')@'
select:
Name: myproject-rtb
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-routetable
sg1:
Properties:
AuthorizeSecurityGroupIngressInput:
- CidrIp: 0.0.0.0/0
FromPort: 22
IpProtocol: tcp
ToPort: 22
- CidrIp: 0.0.0.0/0
FromPort: 24864
IpProtocol: tcp
ToPort: 24864
CreateSecurityGroupInput:
Description: basic-sg
GroupName: myproject-sg1
VpcId: '@depend(''vpc/Properties/VpcId'')@'
select:
Name: myproject-sg1
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-securitygroup
subnet1:
Properties:
CreateSubnetInput:
AvailabilityZone: eu-central-1a
CidrBlock: 10.0.100.0/24
VpcId: '@depend(''vpc/Properties/VpcId'')@'
RouteTableAssociation:
RouteTableId: '@depend(''rtb/Properties/RouteTableId'')@'
select:
Name: myproject-subnet1
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-subnet
subnet2:
Properties:
CreateSubnetInput:
AvailabilityZone: eu-central-1b
CidrBlock: 10.0.200.0/24
VpcId: '@depend(''vpc/Properties/VpcId'')@'
RouteTableAssociation:
RouteTableId: '@depend(''rtb/Properties/RouteTableId'')@'
select:
Name: myproject-subnet2
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-subnet
vpc:
Properties:
CreateVpcInput:
CidrBlock: 10.0.0.0/16
ModifyVpcAttributeInputs:
- EnableDnsSupport:
Value: true
- EnableDnsHostnames:
Value: true
select:
Name: myproject-vpc
infrakit_created: 2018-03-18
infrakit_scope: myproject
infrakit_user: davidchung
plugin: aws/ec2-vpc
state:
- Key: igw
State: REQUESTED
- Key: rtb
State: REQUESTED
- Key: sg1
State: REQUESTED
- Key: subnet1
State: REQUESTED
- Key: subnet2
State: REQUESTED
- Key: vpc
State: REQUESTED
version: ""
In the terminal where you are watching events, you should see:
CollectionUpdate - myproject/sg1 - update collection
CollectionUpdate - myproject/subnet1 - update collection
CollectionUpdate - myproject/subnet2 - update collection
CollectionUpdate - myproject/igw - update collection
CollectionUpdate - myproject/rtb - update collection
Pending - myproject/sg1 - resource blocked waiting on dependencies
Provision - myproject/vpc - provisioning resource
Pending - myproject/rtb - resource blocked waiting on dependencies
Pending - myproject/igw - resource blocked waiting on dependencies
Pending - myproject/subnet2 - resource blocked waiting on dependencies
Pending - myproject/subnet1 - resource blocked waiting on dependencies
MetadataUpdate - myproject/vpc - update metadata
Ready - myproject/vpc - resource ready
Provision - myproject/igw - provisioning resource
Provision - myproject/sg1 - provisioning resource
MetadataUpdate - myproject/igw - update metadata
Ready - myproject/igw - resource ready
Provision - myproject/rtb - provisioning resource
MetadataUpdate - myproject/sg1 - update metadata
Ready - myproject/sg1 - resource ready
Ready - myproject/rtb - resource ready
MetadataUpdate - myproject/rtb - update metadata
Provision - myproject/subnet2 - provisioning resource
Provision - myproject/subnet1 - provisioning resource
MetadataUpdate - myproject/subnet2 - update metadata
Ready - myproject/subnet2 - resource ready
MetadataUpdate - myproject/subnet2 - update metadata
MetadataUpdate - myproject/rtb - update metadata
Ready - myproject/subnet1 - resource ready
MetadataUpdate - myproject/subnet1 - update metadata
MetadataUpdate - myproject/subnet1 - update metadata
MetadataUpdate - myproject/rtb - update metadata
After the events stopped, you can query the inventory controller for the known resources that's been created:
$ infrakit local inventory/myproject keys -al
total 289:
networking/aws/ec2-internetgateway/myproject-igw/ID
networking/aws/ec2-internetgateway/myproject-igw/LogicalID
networking/aws/ec2-internetgateway/myproject-igw/Properties/Attachments/[0]/State
networking/aws/ec2-internetgateway/myproject-igw/Properties/Attachments/[0]/VpcId
networking/aws/ec2-internetgateway/myproject-igw/Properties/InternetGatewayId
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[0]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[0]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[1]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[1]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[2]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[2]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[3]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[3]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[4]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[4]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[5]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[5]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[6]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[6]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[7]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[7]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[8]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[8]/Value
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[9]/Key
networking/aws/ec2-internetgateway/myproject-igw/Properties/Tags/[9]/Value
networking/aws/ec2-internetgateway/myproject-igw/Tags/Name
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_created
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_namespace
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_scope
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_user
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_collection
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_instance
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_link
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_link_context
networking/aws/ec2-internetgateway/myproject-igw/Tags/infrakit_link_created
networking/aws/ec2-routetable/myproject-rtb/ID
networking/aws/ec2-routetable/myproject-rtb/LogicalID
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[0]/Main
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[0]/RouteTableAssociationId
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[0]/RouteTableId
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[0]/SubnetId
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[1]/Main
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[1]/RouteTableAssociationId
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[1]/RouteTableId
networking/aws/ec2-routetable/myproject-rtb/Properties/Associations/[1]/SubnetId
networking/aws/ec2-routetable/myproject-rtb/Properties/PropagatingVgws
networking/aws/ec2-routetable/myproject-rtb/Properties/RouteTableId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/DestinationCidrBlock
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/DestinationIpv6CidrBlock
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/DestinationPrefixListId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/EgressOnlyInternetGatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/GatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/InstanceId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/InstanceOwnerId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/NatGatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/NetworkInterfaceId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/Origin
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/State
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[0]/VpcPeeringConnectionId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/DestinationCidrBlock
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/DestinationIpv6CidrBlock
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/DestinationPrefixListId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/EgressOnlyInternetGatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/GatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/InstanceId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/InstanceOwnerId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/NatGatewayId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/NetworkInterfaceId
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/Origin
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/State
networking/aws/ec2-routetable/myproject-rtb/Properties/Routes/[1]/VpcPeeringConnectionId
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[0]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[0]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[1]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[1]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[2]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[2]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[3]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[3]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[4]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[4]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[5]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[5]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[6]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[6]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[7]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[7]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[8]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[8]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[9]/Key
networking/aws/ec2-routetable/myproject-rtb/Properties/Tags/[9]/Value
networking/aws/ec2-routetable/myproject-rtb/Properties/VpcId
networking/aws/ec2-routetable/myproject-rtb/Tags/Name
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_created
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_namespace
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_scope
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_user
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_collection
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_instance
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_link
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_link_context
networking/aws/ec2-routetable/myproject-rtb/Tags/infrakit_link_created
networking/aws/ec2-securitygroup/myproject-sg1/ID
networking/aws/ec2-securitygroup/myproject-sg1/LogicalID
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Description
networking/aws/ec2-securitygroup/myproject-sg1/Properties/GroupId
networking/aws/ec2-securitygroup/myproject-sg1/Properties/GroupName
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/FromPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/IpProtocol
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/IpRanges/[0]/CidrIp
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/Ipv6Ranges
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/PrefixListIds
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/ToPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[0]/UserIdGroupPairs
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/FromPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/IpProtocol
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/IpRanges/[0]/CidrIp
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/Ipv6Ranges
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/PrefixListIds
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/ToPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissions/[1]/UserIdGroupPairs
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/FromPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/IpProtocol
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/IpRanges/[0]/CidrIp
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/Ipv6Ranges
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/PrefixListIds
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/ToPort
networking/aws/ec2-securitygroup/myproject-sg1/Properties/IpPermissionsEgress/[0]/UserIdGroupPairs
networking/aws/ec2-securitygroup/myproject-sg1/Properties/OwnerId
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[0]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[0]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[1]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[1]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[2]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[2]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[3]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[3]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[4]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[4]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[5]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[5]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[6]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[6]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[7]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[7]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[8]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[8]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[9]/Key
networking/aws/ec2-securitygroup/myproject-sg1/Properties/Tags/[9]/Value
networking/aws/ec2-securitygroup/myproject-sg1/Properties/VpcId
networking/aws/ec2-securitygroup/myproject-sg1/Tags/Name
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_created
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_namespace
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_scope
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_user
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_collection
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_instance
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_link
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_link_context
networking/aws/ec2-securitygroup/myproject-sg1/Tags/infrakit_link_created
networking/aws/ec2-subnet/myproject-subnet1/ID
networking/aws/ec2-subnet/myproject-subnet1/LogicalID
networking/aws/ec2-subnet/myproject-subnet1/Properties/AssignIpv6AddressOnCreation
networking/aws/ec2-subnet/myproject-subnet1/Properties/AvailabilityZone
networking/aws/ec2-subnet/myproject-subnet1/Properties/AvailableIpAddressCount
networking/aws/ec2-subnet/myproject-subnet1/Properties/CidrBlock
networking/aws/ec2-subnet/myproject-subnet1/Properties/DefaultForAz
networking/aws/ec2-subnet/myproject-subnet1/Properties/Ipv6CidrBlockAssociationSet
networking/aws/ec2-subnet/myproject-subnet1/Properties/MapPublicIpOnLaunch
networking/aws/ec2-subnet/myproject-subnet1/Properties/State
networking/aws/ec2-subnet/myproject-subnet1/Properties/SubnetId
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[0]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[0]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[10]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[10]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[1]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[1]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[2]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[2]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[3]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[3]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[4]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[4]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[5]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[5]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[6]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[6]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[7]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[7]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[8]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[8]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[9]/Key
networking/aws/ec2-subnet/myproject-subnet1/Properties/Tags/[9]/Value
networking/aws/ec2-subnet/myproject-subnet1/Properties/VpcId
networking/aws/ec2-subnet/myproject-subnet1/Tags/Name
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_created
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_namespace
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_scope
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_user
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_collection
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_instance
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_link
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_link_context
networking/aws/ec2-subnet/myproject-subnet1/Tags/infrakit_link_created
networking/aws/ec2-subnet/myproject-subnet1/Tags/routeTableAssociation
networking/aws/ec2-subnet/myproject-subnet2/ID
networking/aws/ec2-subnet/myproject-subnet2/LogicalID
networking/aws/ec2-subnet/myproject-subnet2/Properties/AssignIpv6AddressOnCreation
networking/aws/ec2-subnet/myproject-subnet2/Properties/AvailabilityZone
networking/aws/ec2-subnet/myproject-subnet2/Properties/AvailableIpAddressCount
networking/aws/ec2-subnet/myproject-subnet2/Properties/CidrBlock
networking/aws/ec2-subnet/myproject-subnet2/Properties/DefaultForAz
networking/aws/ec2-subnet/myproject-subnet2/Properties/Ipv6CidrBlockAssociationSet
networking/aws/ec2-subnet/myproject-subnet2/Properties/MapPublicIpOnLaunch
networking/aws/ec2-subnet/myproject-subnet2/Properties/State
networking/aws/ec2-subnet/myproject-subnet2/Properties/SubnetId
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[0]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[0]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[10]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[10]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[1]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[1]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[2]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[2]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[3]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[3]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[4]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[4]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[5]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[5]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[6]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[6]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[7]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[7]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[8]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[8]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[9]/Key
networking/aws/ec2-subnet/myproject-subnet2/Properties/Tags/[9]/Value
networking/aws/ec2-subnet/myproject-subnet2/Properties/VpcId
networking/aws/ec2-subnet/myproject-subnet2/Tags/Name
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_created
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_namespace
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_scope
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_user
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_collection
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_instance
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_link
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_link_context
networking/aws/ec2-subnet/myproject-subnet2/Tags/infrakit_link_created
networking/aws/ec2-subnet/myproject-subnet2/Tags/routeTableAssociation
networking/aws/ec2-vpc/myproject-vpc/ID
networking/aws/ec2-vpc/myproject-vpc/LogicalID
networking/aws/ec2-vpc/myproject-vpc/Properties/CidrBlock
networking/aws/ec2-vpc/myproject-vpc/Properties/DhcpOptionsId
networking/aws/ec2-vpc/myproject-vpc/Properties/InstanceTenancy
networking/aws/ec2-vpc/myproject-vpc/Properties/Ipv6CidrBlockAssociationSet
networking/aws/ec2-vpc/myproject-vpc/Properties/IsDefault
networking/aws/ec2-vpc/myproject-vpc/Properties/State
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[0]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[0]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[1]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[1]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[2]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[2]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[3]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[3]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[4]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[4]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[5]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[5]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[6]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[6]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[7]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[7]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[8]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[8]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[9]/Key
networking/aws/ec2-vpc/myproject-vpc/Properties/Tags/[9]/Value
networking/aws/ec2-vpc/myproject-vpc/Properties/VpcId
networking/aws/ec2-vpc/myproject-vpc/Tags/Name
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_created
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_namespace
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_scope
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_user
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_collection
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_instance
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_link
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_link_context
networking/aws/ec2-vpc/myproject-vpc/Tags/infrakit_link_created
- Provision a spot instance in your new VPC
There's an playbook command called spot which will guide you through
provisioning a single spot instance in one of the subnets.
When answering the questions you will be asked to provide vpc and subnet
ids. To get those values, do
$ infrakit local mystack/resource describe -o
COLLECTION KEY STATE DATA
myproject igw READY igw-08e6cb5872f1b83cb
myproject rtb READY rtb-0c3ff9daa89a5e06c
myproject sg1 READY sg-0ea554fba9c28b3bf
myproject subnet1 READY subnet-01b5e450f49749676
myproject subnet2 READY subnet-09c5c220a71268f7a
myproject vpc READY vpc-07003eb7f376414b1
The resource's infrastructure resource ids will be listed as DATA along side the logical
ids (e.g sg1) you have given them in the mystack.yml. Use these values in the steps that follow
to provision a single spot instance or a pool of spot instances.
$ infrakit use aws spot
Please enter your user name: [davidchung]:
Project? [myproject]:
AMI? [ami-df8406b0]:
Instance type? [t2.micro]:
Host name? [myproject-Zm6UfgDt]:
Spot price? [0.03]:
SSH key? [infrakit]:
Subnet? [subnet2]:
Private IP address? [10.0.200.0/24]: 10.0.200.100
Security group ID? [sg-2e3f8143]:
This command can sometimes timeout because it takes a while to provision a spot
instance. You can set INFRAKIT_CLIENT_TIMEOUT=10s as environment variable prior
to running infrakit use aws spot. In case the client times out, you can see if
the instance has been created:
$ infrakit local aws/ec2-spot-instance describe
ID LOGICAL TAGS
sir-m81rhchp 10.0.200.100 Name=myproject-Zm6UfgDt,infrakit_created=2018-03-18,infrakit_namespace=davidchung,infrakit_scope=myproject,infrakit_user=davidchung
or via the inventory controller. We query for entries under the compute category
(see inventory.yml where we defined the compute category) and under the
aws/ec2-spot-instance (the plugin name as sub namespace):
$ infrakit local inventory/myproject keys compute/aws/ec2-spot-instance
myproject-YjM9d5Vs
infrakit local inventory/myproject keys -al compute/aws/ec2-spot-instance/myproject-YjM9d5Vs
total 132:
ID
LogicalID
Properties/Instance/AmiLaunchIndex
Properties/Instance/Architecture
Properties/Instance/BlockDeviceMappings/[0]/DeviceName
Properties/Instance/BlockDeviceMappings/[0]/Ebs/AttachTime
Properties/Instance/BlockDeviceMappings/[0]/Ebs/DeleteOnTermination
Properties/Instance/BlockDeviceMappings/[0]/Ebs/Status
Properties/Instance/BlockDeviceMappings/[0]/Ebs/VolumeId
Properties/Instance/ClientToken
Properties/Instance/EbsOptimized
Properties/Instance/EnaSupport
Properties/Instance/Hypervisor
Properties/Instance/IamInstanceProfile
Properties/Instance/ImageId
Properties/Instance/InstanceId
Properties/Instance/InstanceLifecycle
Properties/Instance/InstanceType
Properties/Instance/KernelId
Properties/Instance/KeyName
Properties/Instance/LaunchTime
Properties/Instance/Monitoring/State
Properties/Instance/NetworkInterfaces/[0]/Association/IpOwnerId
# more fields...
$ infrakit local inventory/myproject cat compute/aws/ec2-spot-instance/myproject-YjM9d5Vs/Properties/Instance/PublicIpAddress
18.184.52.135
Let's ssh in:
~$ eval `ssh-agent -s`
Agent pid 35295
~$ ssh-add ~/.ssh/infrakit
Identity added: /Users/davidchung/.ssh/infrakit (/Users/davidchung/.ssh/infrakit)
~$ ssh ubuntu@$(infrakit local inventory/myproject cat compute/aws/ec2-spot-instance/myproject-YjM9d5Vs/Properties/Instance/PublicIpAddress)
The authenticity of host '18.184.52.135 (18.184.52.135)' can't be established.
ECDSA key fingerprint is SHA256:EnKhV+8cgUjQzL1Wvh2nwS+T5Meoxn6K/diAJtM+o9Y.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '18.184.52.135' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.4 LTS (GNU/Linux 4.4.0-1052-aws x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
58 packages can be updated.
17 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
- Provision a pool of nodes: You can use the
poolcontroller to provision a pool of spot instances:
$ infrakit use aws nodes --plugin mystack/pool --az eu-central-1a --subnet-id subnet-01b5e450f49749676 --security-group-id sg-0ea554fba9c28b3bf --accept-defaults
This will provision 5 spot instances. You can watch the progress via
$ watch -d infrakit local mystack/pool describe -o
Every 2.0s: infrakit local mystack/pool describe -o
COLLECTION KEY STATE DATA
myproject-nodes myproject-nodes_0000 READY sir-46yrgvnn
myproject-nodes myproject-nodes_0001 READY sir-dxt8hk4m
myproject-nodes myproject-nodes_0002 READY sir-4z6rh6sq
myproject-nodes myproject-nodes_0003 READY sir-46tihzkq
myproject-nodes myproject-nodes_0004 READY sir-15frhx2p
You can scale down this pool of nodes by adding a --count flag:
infrakit use aws nodes --plugin mystack/pool --az eu-central-1a --subnet-id subnet-01b5e450f49749676 --security-group-id sg-0ea554fba9c28b3bf --count 1 --accept-defaults
You will see that some nodes become UNMATCHED:
Every 2.0s: infrakit local mystack/pool describe -o
COLLECTION KEY STATE DATA
myproject-nodes myproject-nodes_0000 READY sir-46yrgvnn
myproject-nodes myproject-nodes_0001 UNMATCHED sir-dxt8hk4m
myproject-nodes myproject-nodes_0002 UNMATCHED sir-4z6rh6sq
myproject-nodes myproject-nodes_0003 UNMATCHED sir-46tihzkq
myproject-nodes myproject-nodes_0004 UNMATCHED sir-15frhx2p
Slowly you will see the unmatched nodes be terminated and removed:
Every 2.0s: infrakit local mystack/pool describe -o
COLLECTION KEY STATE DATA
myproject-nodes myproject-nodes_0000 READY sir-46yrgvnn
myproject-nodes myproject-nodes_0003 TERMINATING sir-46tihzkq
myproject-nodes myproject-nodes_0004 TERMINATING sir-15frhx2p
Currently we do not support termination of resources. So you must do this manually.
The ordering to destroy:
- Destroy the instances first
$ infrakit local aws/ec2-spot-instance destroy ...
- Destroy the security groups:
$ infrakit local aws/ec2-securitygroup destroy ...
- Destroy the subnets
$ infrakit local aws/ec2-subnet destroy ...
- Destroy the route tables
$ infrakit local aws/ec2-routetable destroy ...
- Destroy the gateway
$ infrakit local aws/ec2-internetgateway destroy ...
- Destroy the VPC
$ infrakit local aws/ec2-vpc destroy ...