Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS support #1918

Closed
justincormack opened this issue May 27, 2017 · 17 comments

Comments

@justincormack
Copy link
Collaborator

commented May 27, 2017

We need support for AWS. Includes:

  • push to AMI
  • run an AMI

Plus CI.

@justincormack justincormack added this to the 0.1 release milestone May 27, 2017

@allingeek

This comment has been minimized.

Copy link

commented May 31, 2017

And Metadata support.

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented May 31, 2017

@justincormack how can I help with this?

@justincormack

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 1, 2017

@kencochrane I am just cleaning up the code that can generate raw disk images (currently we have img-gz which is a compressed raw image). These should work fine on AWS. @dave-tucker was looking at the VM import code to see if we could use that, but it seems buggy, so we probably need to do the disk snapshotting method we were using before, which should be fine with the current images. We basically want a linuxkit push and linuxkit run for AWS so we have easy ability to test in CI, see eg the gcp code in https://github.com/linuxkit/linuxkit/tree/master/src/cmd/linuxkit for example.

@dave-tucker

This comment has been minimized.

Copy link
Collaborator

commented Jun 1, 2017

I don't think the VM import method will work without some support from AWS.

See here for their docs and prerequisites. I have no idea why AWS can only support import of some OSs.

Following the example in their docs with a raw LinuxKit image gives you ClientError: Unknown OS / Missing OS files.

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

@dave-tucker that stinks. Looking at this page http://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html it looks like they have only a small list of Linux flavors that they support. We would either need to have them support a LinuxKit version, or we need a different way to do it.

Linux/Unix (64-bit)

  • Ubuntu 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 16.04, 16.10
  • Red Hat Enterprise Linux (RHEL) 5.1-5.11, 6.1-6.6, 7.0-7.1 (6.0 lacks required drivers)
  • SUSE Linux Enterprise Server 11 with Service Pack 1 and kernel 2.6.32.12-0.7
  • SUSE Linux Enterprise Server 11 with Service Pack 2 and kernel 3.0.13-0.27
  • SUSE Linux Enterprise Server 11 with Service Pack 3 and kernel 3.0.76-0.11, 3.0.101-0.8, or 3.0.101-0.15
  • SUSE Linux Enterprise Server 11 with Service Pack 4 and kernel 3.0.101-63
  • SUSE Linux Enterprise Server 12 with kernel 3.12.28-4
  • SUSE Linux Enterprise Server 12 with Service Pack 1 and kernel 3.12.49-11
  • CentOS 5.1-5.11, 6.1-6.6, 7.0-7.2 (6.0 lacks required drivers)
  • Debian 6.0.0-6.0.8, 7.0.0-7.8.0, 8.0.0
  • Oracle Linux 6.1-6.6, 7.0-7.1
  • Fedora Server 19-21
@justincormack

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 1, 2017

Reading that guide suggests it does weird things like install drivers, so I think its not going to work. We should probably stick with the disk snapshot method that we use for docker 4 aws now

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

@dave-tucker

This comment has been minimized.

Copy link
Collaborator

commented Jun 1, 2017

Re-posting the link I pasted in Slack yesterday:
https://blog.zhaw.ch/icclab/walk-through-importing-virtual-machine-images-into-ec2/

Quoting the author, emphasis mine:

The derived recommendation for VM images is therefore to build them with the Grub boot loader and its configuration, using a partitioned disk with at least one partition, and an appropriate 64-bit Xen-enabled kernel booting with initrd. Yet, when building a minimal image from scratch using this combination of configuration choices, it still results in the ominous Unknown OS / Missing OS files error message, which is not very helpful without further details

I agree that the aforementioned docker 4 aws method would be easier

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

@allingeek for metadata support do you mean adding a provider here, like we have for GCP? https://github.com/linuxkit/linuxkit/tree/master/pkg/metadata

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

@justincormack @dave-tucker if no one is working on adding the aws provider to the metadata package I can work on that if you would like?

@justincormack

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 1, 2017

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

Cool, I'll get started. thanks

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 1, 2017

Submitted my PR to add an AWS provider to the metadata package. Let me know if you need help with anything else. #1954

@dave-tucker

This comment has been minimized.

Copy link
Collaborator

commented Jun 2, 2017

Success with uploading an LinuxKit raw image as an AMI

$ aws s3 cp disk.raw s3://<your-bucket-name>
$ cat container.json
{
    "Description": "LinuxKit",
    "Format": "raw",
    "UserBucket": {
        "S3Bucket": "<your-bucket-name>",
        "S3Key": "disk.raw"
    }
}
$ aws ec2 import-snapshot --description "LinuxKit Test" --disk-container file://containers.json
$ aws ec2 describe-import-snapshot-tasks --import-task-ids <task-id-from-above-command>
$ # You should have the snapshot ID when this finishes
$ aws ec2 register-image --name dt-LinuxKit --architecture x86_64 --root-device-name /dev/sda1 --virtualization-type hvm --block-device-mappings DeviceName="/dev/sda1",Ebs={SnapshotId=snap-0cae423393883e067}

I started the AMI from the console and it worked - booted, outputted stuff to serial console and then shut down!

I'll get a PR ready for this today!

@kencochrane

This comment has been minimized.

Copy link
Contributor

commented Jun 2, 2017

@dave-tucker awesome!

@DieterReuter

This comment has been minimized.

Copy link
Contributor

commented Jun 2, 2017

I just created a PR #1958 with an AWS example, which is basically a clone of the updated GCP example.

So with this example it's easy to create a first AWS LinuxKit VM with ssh support, where the ssh keys will be provided through the newly added AWS support (see #1954).

@dave-tucker

This comment has been minimized.

Copy link
Collaborator

commented Jun 2, 2017

Progress is in a branch here
I'm behind where I wanted to be due to issues with vendoring 🦀

I'm currently testing this now but it should manage to get you an AMI but YMMV.

Then I just need to plumb in the DescribeImages and RunInstances API calls...
Displaying output from run will be more complicated as the GetConsoleOutput is bufffered and just appears after a given period afaict.

Testing with raw images as follows:

$ moby -v build -output img -size 200M linuxkit.yml
$  ./bin/linuxkit -v push aws -bucket dave-tucker-linuxkit linuxkit.img

This expects you to have AWS_REGION exported in your environment as well as the standard AWS variables and/or a ~/.aws/credentials file via aws config

Docs will be included with the final PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.