Switch branches/tags
Nothing to show
Find file History
jsleeio and k8s-ci-robot [imagebuilder] new flag for injecting extra tags (#708)
* [imagebuilder] new flag for injecting extra tags

Currently only implemented for AWS, as I don't have a GCE environment to
test in and didn't want to submit a change that would risk breaking
other people's images.

Add a flag `-addtags` to allow specifying one or more additional tags to
be injected into the `imagebuilder` configuration. This has two
purposes:

1. users operating in the same AWS account can use tags to select images

2. it also allows injecting extra runtime config into the `bootstrap-vz`
   templates. Possibly best explained with an example from our internal
   CI job:

```
---
{{- define "gitref" -}}
{{- $ref := or .Tags.GitRef "" -}}
{{- if ne $ref "master" -}}
{{- printf "%s" $ref -}}
{{- end -}}
{{- end -}}
{{ if eq .Cloud "aws" }}
name: sm-k8s-1.8-debian-{system.release}-{system.architecture}-{provider.virtualization}-ebs-{%Y}-{%m}-{%d}{{ template "gitref" . }}
{{ else }}
name: k8s-1.8-debian-{system.release}-{system.architecture}-{%Y}-{%m}-{%d}{{- template "gitref" . }}
{{ end }}
```

* [imagebuilder] propagate tags between Config/provider

Don't want to pass the whole Config object through to the imagebuilder
process as it contains a private key. Instead, just propagate tags, so
that they can be used in both the template and in taggin the resulting
AMI.

This could probably use some refactoring.
Latest commit d550583 Aug 17, 2018
Permalink
..
Failed to load latest commit information.
hack fix directory path and ensure region is set Jan 31, 2017
pkg/imagebuilder [imagebuilder] new flag for injecting extra tags (#708) Aug 16, 2018
templates 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
vendor imagebuilder: add glide dependencies and vendor Jul 4, 2016
.gitignore 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
Makefile Add gofmt target to imagebuilder makefile Sep 16, 2016
README.md adding setup-aws.sh script Nov 16, 2016
aws-1.10-jessie.yaml 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
aws-1.10-stretch.yaml 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
aws-1.11-jessie.yaml 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
aws-1.11-stretch.yaml 1.10 and 1.11 k8s-optimized images (#699) Aug 16, 2018
aws-1.3.yaml imagebuilder: create an image for k8s 1.4 Oct 5, 2016
aws-1.4.yaml 1.5 image: bump to docker 1.12.3 Dec 19, 2016
aws-1.5.yaml Lock to specific branches of bootstrap-vz Nov 27, 2017
aws-1.6.yaml Lock to specific branches of bootstrap-vz Nov 27, 2017
aws-1.7.yaml Create 1.8 AMI Dec 1, 2017
aws-1.8-jessie.yaml Separate 1.8 AMIs for stretch & jessie Dec 1, 2017
aws-1.8-stretch.yaml Separate 1.8 AMIs for stretch & jessie Dec 1, 2017
aws-1.9-jessie.yaml AWS 1.9 image Mar 11, 2018
aws-1.9-stretch.yaml AWS 1.9 image Mar 11, 2018
gce.yaml Create 1.3 optimized image for kube-up / upup / anyone Jun 20, 2016
glide.lock imagebuilder: add glide dependencies and vendor Jul 4, 2016
glide.yaml imagebuilder: add glide dependencies and vendor Jul 4, 2016
main.go [imagebuilder] new flag for injecting extra tags (#708) Aug 16, 2018

README.md

ImageBuilder

ImageBuilder is a tool for building an optimized k8s images, currently only supporting AWS.

Please also see the README in templates for documentation as to the motivation for building a custom image.

It is a wrapper around bootstrap-vz (the tool used to build official Debian cloud images).
It adds functionality to spin up an instance for building the image, and publishing the image to all regions.

Imagebuilder create an instance to build the image, builds the image as specified by TemplatePath, makes the image public and copies it to all accessible regions (on AWS), and then shuts down the builder instance. Each of these stages can be controlled through flags (for example, you might not want use --publish=false for an internal image.)

AWS

  • export AWS_PROFILE=... if you are not using the default profile. (or generate a new account & use export AWS_ACCESS_KEY_ID and export AWS_SECRET_ACCESS_KEY)
  • Create a VPC (with a subnet) and tag the subnet with k8s.io/role/imagebuilder=1
  • Create a security group in the VPC, allowing port 22, and tag with k8s.io/role/imagebuilder=1
  • The following commands are scripted in hack/setup-aws.sh
VPC_ID=`aws ec2 create-vpc --cidr-block 172.20.0.0/16 --query Vpc.VpcId --output text`
aws ec2 create-tags --resources ${VPC_ID} --tags Key=k8s.io/role/imagebuilder,Value=1

SUBNET_ID=`aws ec2 create-subnet --cidr-block 172.20.1.0/24 --vpc-id ${VPC_ID} --query Subnet.SubnetId --output text`
aws ec2 create-tags --resources ${SUBNET_ID} --tags Key=k8s.io/role/imagebuilder,Value=1


IGW_ID=`aws ec2 create-internet-gateway --query InternetGateway.InternetGatewayId --output text`
aws ec2 create-tags --resources ${IGW_ID} --tags Key=k8s.io/role/imagebuilder,Value=1

aws ec2 attach-internet-gateway --internet-gateway-id ${IGW_ID} --vpc-id ${VPC_ID}

RT_ID=`aws ec2 describe-route-tables --filters Name=vpc-id,Values=${VPC_ID} --query RouteTables[].RouteTableId --output text`

SG_ID=`aws ec2 create-security-group --vpc-id ${VPC_ID} --group-name imagebuilder --description "imagebuilder security group" --query GroupId --output text`
aws ec2 create-tags --resources ${SG_ID} --tags Key=k8s.io/role/imagebuilder,Value=1

aws ec2 associate-route-table --route-table-id ${RT_ID} --subnet-id ${SUBNET_ID}

aws ec2 create-route --route-table-id ${RT_ID} --destination-cidr-block 0.0.0.0/0 --gateway-id ${IGW_ID}

aws ec2 authorize-security-group-ingress  --group-id ${SG_ID} --protocol tcp --port 22 --cidr 0.0.0.0/0

Then:

go get k8s.io/kube-deploy/imagebuilder

Build code: make

Run the image builder:

cd ${GOPATH}/src/k8s.io/kube-deploy/imagebuilder`
make
${GOPATH}/bin/imagebuilder --config aws.yaml --v=8

It will print the IDs of the image in each region, but it will also tag the image with a Name as specified in the template) and this is the easier way to retrieve the image.

GCE

  • Edit gce.yaml, at least to specify the Project and GCSDestination to use
  • Create the GCS bucket in GCSDestination (if it does not exist) gsutil mb gs://<bucketname>/

Then:

go get k8s.io/kube-deploy/imagebuilder

Run the image builder:

cd ${GOPATH}/src/k8s.io/kube-deploy/imagebuilder`
make
${GOPATH}/bin/imagebuilder --config gce.yaml --v=8 --publish=false

Note that because GCE does not currently support publishing images, you must pass --publish=false. Also, images on GCE are global, so replicate does not actually need to do anything.

Advanced options

Check out --help, but these options control which operations we perform, and may be useful for debugging or publishing a lot of images:

  • --up=true/false, --down=true/false control whether we try to create and terminate an instance to do the building

  • --publish=true/false controls whether we make the image public

  • --replicate=true/false controls whether we copy the image to all regions

  • --config=<configpath> lets you configure most options