New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS support #1918

Closed
justincormack opened this Issue May 27, 2017 · 17 comments

Comments

Projects
None yet
5 participants
@justincormack
Collaborator

justincormack commented May 27, 2017

We need support for AWS. Includes:

  • push to AMI
  • run an AMI

Plus CI.

@justincormack justincormack added this to the 0.1 release milestone May 27, 2017

@allingeek

This comment has been minimized.

Show comment
Hide comment
@allingeek

allingeek May 31, 2017

And Metadata support.

And Metadata support.

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane May 31, 2017

Contributor

@justincormack how can I help with this?

Contributor

kencochrane commented May 31, 2017

@justincormack how can I help with this?

@justincormack

This comment has been minimized.

Show comment
Hide comment
@justincormack

justincormack Jun 1, 2017

Collaborator

@kencochrane I am just cleaning up the code that can generate raw disk images (currently we have img-gz which is a compressed raw image). These should work fine on AWS. @dave-tucker was looking at the VM import code to see if we could use that, but it seems buggy, so we probably need to do the disk snapshotting method we were using before, which should be fine with the current images. We basically want a linuxkit push and linuxkit run for AWS so we have easy ability to test in CI, see eg the gcp code in https://github.com/linuxkit/linuxkit/tree/master/src/cmd/linuxkit for example.

Collaborator

justincormack commented Jun 1, 2017

@kencochrane I am just cleaning up the code that can generate raw disk images (currently we have img-gz which is a compressed raw image). These should work fine on AWS. @dave-tucker was looking at the VM import code to see if we could use that, but it seems buggy, so we probably need to do the disk snapshotting method we were using before, which should be fine with the current images. We basically want a linuxkit push and linuxkit run for AWS so we have easy ability to test in CI, see eg the gcp code in https://github.com/linuxkit/linuxkit/tree/master/src/cmd/linuxkit for example.

@dave-tucker

This comment has been minimized.

Show comment
Hide comment
@dave-tucker

dave-tucker Jun 1, 2017

Collaborator

I don't think the VM import method will work without some support from AWS.

See here for their docs and prerequisites. I have no idea why AWS can only support import of some OSs.

Following the example in their docs with a raw LinuxKit image gives you ClientError: Unknown OS / Missing OS files.

Collaborator

dave-tucker commented Jun 1, 2017

I don't think the VM import method will work without some support from AWS.

See here for their docs and prerequisites. I have no idea why AWS can only support import of some OSs.

Following the example in their docs with a raw LinuxKit image gives you ClientError: Unknown OS / Missing OS files.

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 1, 2017

Contributor

@dave-tucker that stinks. Looking at this page http://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html it looks like they have only a small list of Linux flavors that they support. We would either need to have them support a LinuxKit version, or we need a different way to do it.

Linux/Unix (64-bit)

  • Ubuntu 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 16.04, 16.10
  • Red Hat Enterprise Linux (RHEL) 5.1-5.11, 6.1-6.6, 7.0-7.1 (6.0 lacks required drivers)
  • SUSE Linux Enterprise Server 11 with Service Pack 1 and kernel 2.6.32.12-0.7
  • SUSE Linux Enterprise Server 11 with Service Pack 2 and kernel 3.0.13-0.27
  • SUSE Linux Enterprise Server 11 with Service Pack 3 and kernel 3.0.76-0.11, 3.0.101-0.8, or 3.0.101-0.15
  • SUSE Linux Enterprise Server 11 with Service Pack 4 and kernel 3.0.101-63
  • SUSE Linux Enterprise Server 12 with kernel 3.12.28-4
  • SUSE Linux Enterprise Server 12 with Service Pack 1 and kernel 3.12.49-11
  • CentOS 5.1-5.11, 6.1-6.6, 7.0-7.2 (6.0 lacks required drivers)
  • Debian 6.0.0-6.0.8, 7.0.0-7.8.0, 8.0.0
  • Oracle Linux 6.1-6.6, 7.0-7.1
  • Fedora Server 19-21
Contributor

kencochrane commented Jun 1, 2017

@dave-tucker that stinks. Looking at this page http://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html it looks like they have only a small list of Linux flavors that they support. We would either need to have them support a LinuxKit version, or we need a different way to do it.

Linux/Unix (64-bit)

  • Ubuntu 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 16.04, 16.10
  • Red Hat Enterprise Linux (RHEL) 5.1-5.11, 6.1-6.6, 7.0-7.1 (6.0 lacks required drivers)
  • SUSE Linux Enterprise Server 11 with Service Pack 1 and kernel 2.6.32.12-0.7
  • SUSE Linux Enterprise Server 11 with Service Pack 2 and kernel 3.0.13-0.27
  • SUSE Linux Enterprise Server 11 with Service Pack 3 and kernel 3.0.76-0.11, 3.0.101-0.8, or 3.0.101-0.15
  • SUSE Linux Enterprise Server 11 with Service Pack 4 and kernel 3.0.101-63
  • SUSE Linux Enterprise Server 12 with kernel 3.12.28-4
  • SUSE Linux Enterprise Server 12 with Service Pack 1 and kernel 3.12.49-11
  • CentOS 5.1-5.11, 6.1-6.6, 7.0-7.2 (6.0 lacks required drivers)
  • Debian 6.0.0-6.0.8, 7.0.0-7.8.0, 8.0.0
  • Oracle Linux 6.1-6.6, 7.0-7.1
  • Fedora Server 19-21
@justincormack

This comment has been minimized.

Show comment
Hide comment
@justincormack

justincormack Jun 1, 2017

Collaborator

Reading that guide suggests it does weird things like install drivers, so I think its not going to work. We should probably stick with the disk snapshot method that we use for docker 4 aws now

Collaborator

justincormack commented Jun 1, 2017

Reading that guide suggests it does weird things like install drivers, so I think its not going to work. We should probably stick with the disk snapshot method that we use for docker 4 aws now

@kencochrane

This comment has been minimized.

Show comment
Hide comment
Contributor

kencochrane commented Jun 1, 2017

@dave-tucker

This comment has been minimized.

Show comment
Hide comment
@dave-tucker

dave-tucker Jun 1, 2017

Collaborator

Re-posting the link I pasted in Slack yesterday:
https://blog.zhaw.ch/icclab/walk-through-importing-virtual-machine-images-into-ec2/

Quoting the author, emphasis mine:

The derived recommendation for VM images is therefore to build them with the Grub boot loader and its configuration, using a partitioned disk with at least one partition, and an appropriate 64-bit Xen-enabled kernel booting with initrd. Yet, when building a minimal image from scratch using this combination of configuration choices, it still results in the ominous Unknown OS / Missing OS files error message, which is not very helpful without further details

I agree that the aforementioned docker 4 aws method would be easier

Collaborator

dave-tucker commented Jun 1, 2017

Re-posting the link I pasted in Slack yesterday:
https://blog.zhaw.ch/icclab/walk-through-importing-virtual-machine-images-into-ec2/

Quoting the author, emphasis mine:

The derived recommendation for VM images is therefore to build them with the Grub boot loader and its configuration, using a partitioned disk with at least one partition, and an appropriate 64-bit Xen-enabled kernel booting with initrd. Yet, when building a minimal image from scratch using this combination of configuration choices, it still results in the ominous Unknown OS / Missing OS files error message, which is not very helpful without further details

I agree that the aforementioned docker 4 aws method would be easier

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 1, 2017

Contributor

@allingeek for metadata support do you mean adding a provider here, like we have for GCP? https://github.com/linuxkit/linuxkit/tree/master/pkg/metadata

Contributor

kencochrane commented Jun 1, 2017

@allingeek for metadata support do you mean adding a provider here, like we have for GCP? https://github.com/linuxkit/linuxkit/tree/master/pkg/metadata

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 1, 2017

Contributor

@justincormack @dave-tucker if no one is working on adding the aws provider to the metadata package I can work on that if you would like?

Contributor

kencochrane commented Jun 1, 2017

@justincormack @dave-tucker if no one is working on adding the aws provider to the metadata package I can work on that if you would like?

@justincormack

This comment has been minimized.

Show comment
Hide comment
@justincormack

justincormack Jun 1, 2017

Collaborator
Collaborator

justincormack commented Jun 1, 2017

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 1, 2017

Contributor

Cool, I'll get started. thanks

Contributor

kencochrane commented Jun 1, 2017

Cool, I'll get started. thanks

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 1, 2017

Contributor

Submitted my PR to add an AWS provider to the metadata package. Let me know if you need help with anything else. #1954

Contributor

kencochrane commented Jun 1, 2017

Submitted my PR to add an AWS provider to the metadata package. Let me know if you need help with anything else. #1954

@dave-tucker

This comment has been minimized.

Show comment
Hide comment
@dave-tucker

dave-tucker Jun 2, 2017

Collaborator

Success with uploading an LinuxKit raw image as an AMI

$ aws s3 cp disk.raw s3://<your-bucket-name>
$ cat container.json
{
    "Description": "LinuxKit",
    "Format": "raw",
    "UserBucket": {
        "S3Bucket": "<your-bucket-name>",
        "S3Key": "disk.raw"
    }
}
$ aws ec2 import-snapshot --description "LinuxKit Test" --disk-container file://containers.json
$ aws ec2 describe-import-snapshot-tasks --import-task-ids <task-id-from-above-command>
$ # You should have the snapshot ID when this finishes
$ aws ec2 register-image --name dt-LinuxKit --architecture x86_64 --root-device-name /dev/sda1 --virtualization-type hvm --block-device-mappings DeviceName="/dev/sda1",Ebs={SnapshotId=snap-0cae423393883e067}

I started the AMI from the console and it worked - booted, outputted stuff to serial console and then shut down!

I'll get a PR ready for this today!

Collaborator

dave-tucker commented Jun 2, 2017

Success with uploading an LinuxKit raw image as an AMI

$ aws s3 cp disk.raw s3://<your-bucket-name>
$ cat container.json
{
    "Description": "LinuxKit",
    "Format": "raw",
    "UserBucket": {
        "S3Bucket": "<your-bucket-name>",
        "S3Key": "disk.raw"
    }
}
$ aws ec2 import-snapshot --description "LinuxKit Test" --disk-container file://containers.json
$ aws ec2 describe-import-snapshot-tasks --import-task-ids <task-id-from-above-command>
$ # You should have the snapshot ID when this finishes
$ aws ec2 register-image --name dt-LinuxKit --architecture x86_64 --root-device-name /dev/sda1 --virtualization-type hvm --block-device-mappings DeviceName="/dev/sda1",Ebs={SnapshotId=snap-0cae423393883e067}

I started the AMI from the console and it worked - booted, outputted stuff to serial console and then shut down!

I'll get a PR ready for this today!

@kencochrane

This comment has been minimized.

Show comment
Hide comment
@kencochrane

kencochrane Jun 2, 2017

Contributor

@dave-tucker awesome!

Contributor

kencochrane commented Jun 2, 2017

@dave-tucker awesome!

@DieterReuter

This comment has been minimized.

Show comment
Hide comment
@DieterReuter

DieterReuter Jun 2, 2017

Contributor

I just created a PR #1958 with an AWS example, which is basically a clone of the updated GCP example.

So with this example it's easy to create a first AWS LinuxKit VM with ssh support, where the ssh keys will be provided through the newly added AWS support (see #1954).

Contributor

DieterReuter commented Jun 2, 2017

I just created a PR #1958 with an AWS example, which is basically a clone of the updated GCP example.

So with this example it's easy to create a first AWS LinuxKit VM with ssh support, where the ssh keys will be provided through the newly added AWS support (see #1954).

@dave-tucker

This comment has been minimized.

Show comment
Hide comment
@dave-tucker

dave-tucker Jun 2, 2017

Collaborator

Progress is in a branch here
I'm behind where I wanted to be due to issues with vendoring 🦀

I'm currently testing this now but it should manage to get you an AMI but YMMV.

Then I just need to plumb in the DescribeImages and RunInstances API calls...
Displaying output from run will be more complicated as the GetConsoleOutput is bufffered and just appears after a given period afaict.

Testing with raw images as follows:

$ moby -v build -output img -size 200M linuxkit.yml
$  ./bin/linuxkit -v push aws -bucket dave-tucker-linuxkit linuxkit.img

This expects you to have AWS_REGION exported in your environment as well as the standard AWS variables and/or a ~/.aws/credentials file via aws config

Docs will be included with the final PR.

Collaborator

dave-tucker commented Jun 2, 2017

Progress is in a branch here
I'm behind where I wanted to be due to issues with vendoring 🦀

I'm currently testing this now but it should manage to get you an AMI but YMMV.

Then I just need to plumb in the DescribeImages and RunInstances API calls...
Displaying output from run will be more complicated as the GetConsoleOutput is bufffered and just appears after a given period afaict.

Testing with raw images as follows:

$ moby -v build -output img -size 200M linuxkit.yml
$  ./bin/linuxkit -v push aws -bucket dave-tucker-linuxkit linuxkit.img

This expects you to have AWS_REGION exported in your environment as well as the standard AWS variables and/or a ~/.aws/credentials file via aws config

Docs will be included with the final PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment