Implemented snapshot loading capabilities

[sondavidb]

Signed-off-by: David Son davbson@amazon.com

Taking most of the code from #348 and modifying it a bit, I was able to implement the ability to load snapshots as an option within the Start function in machine.go via WithSnapshot. This fixes the issue addressed in #348, where machine.go exposing LoadSnapshot function in said PR was deemed suboptimal due to its guaranteed failure outside of the scope of Start.

The actual implemention in opts.go initially felt a bit messy, as I felt the extra added imports made it clunky, but @austinvazquez and I looked over it today and decided it was fine given that it ultimately did similar things compared to other options, but feel free to disagree below if you feel otherwise.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[sondavidb]

Made changes that included both a config file and an extra handler to load snapshots, so that we can choose when to call it during device setup.

I tried to do @kzys's approach of having LoadSnapshotHandler after AttachDrivesHandler but having either one in front of the other makes Firecracker unhappy and gives me an error saying that whatever handler is called second cannot be called after start config is set (e.g. AttachDrivesHandler -> LoadSnapshotHandler makes Firecracker complain about LoadSnapshotHandler, while LoadSnapshotHandler -> AttachDrivesHandler makes Firecracker complain about LoadSnapshot handler, both for the same reason). Based on this information, I feel like either a. Attaching drives is handled by loading the snapshot, or b. I'm not fully understanding how LoadSnapshot works. Leaning more towards b. as taking a look at line 171 in this file, it seems that snapshots need to have the drives attached before being loaded, but this doesn't make sense based on the testing done, as it does not let you attach drives before the snapshot is loaded (LoadSnapshot errors out), and doing so afterwards is both wrong and does not work (AttachDrives errors out)

In any case, the above seems to run fine, so take that as you will. I'm planning on adding some testing tomorrow to make sure that the proper snapshot is being loaded. This will be done as follows:

• Create MicroVM
• ssh into it
• Run sleep 492 (or some other weird uncommon value)
• Run CreateSnapshot
• Close the VM
• Create another MicroVM loaded from that snapshot
• ssh into that
• see if ps -aux | grep "sleep 492" returns more than one line

If said test succeeds, we can assume that said snapshot is properly loaded and forget about it, but I also would like to make sure I fully understand this to ensure we're not skipping over anything crucial.

[Kern--]

We switched from ptp to bridge in firecracker-containerd. Should we do the same here?

[austinvazquez]

I looked into this and it didn't work out-of-the-box. Mainly I didn't see the bridge plugin creating the IP route on the host like the ptp plugin does. So packets were not being routed to the bridge. More than likely I am miss configuring though. @sondavidb would you like to give it a shot?

[sondavidb]

From what I understand, the bridge plugin doesn't actually create any network interfaces, it only bridges two pre-existing connections. So, to refactor this would probably take moderate effort — entirely doable, but I would also like to finish this PR first to get the feature out, and then create a separate PR to refactor this. Given that it's only used in a test, I think it's not crucial to ship this change with the existing PR. What do you think @Kern--?

[Kern--]

Both the bridge plugin and ptp create veth pairs, so it should be just a slightly different config to switch, but I don't feel strongly enough about which we use to block on that. It's fine to ship this as is.

[sondavidb]

Worth mentioning the install step seems to be flaky — sometimes it doesn't set up the loop device during mounting (e.g. the install step in this build, line 83), but just restarting the test makes it run fine. Unsure of cause but good to document in my opinion.