updata/update_metadata: Adds ability to add waves using TOML files

[zmrow]

Issue number:
Fixes #596

Description of changes:
This commit adds to updata the ability to use TOML-formatted files
to add waves to an existing update. The --wave-file flag for
the add-wave subcommand allows a user to specify the path to the
file where waves are defined.

Structs for deserializing the TOML have been added to update_metadata. A method on add_waves was also added Manifest. This allows us to update the manifest.json using other Rust code.

Testing done:

• All unit tests continue to pass
• An built image boots locally
• Calling updata with a waves file with an invalid fleet_percentage fails as expected.

$ cat ./waves 
[[waves]]
start_after = 'in 12 hours'
fleet_percentage = 1

[[waves]]
start_after = 'in 1 day'
fleet_percentage = 10

[[waves]]
start_after = 'in 3 days'
fleet_percentage = 0

[[waves]]
start_after = 'in 6 days'
fleet_percentage = 100

$ cargo run --bin updata -- set-waves foobar --arch x86_64 --version 0.3.1 --variant aws-k8s-1.15 --wave-file ./waves
...
15:31:56 [ERROR] `fleet_percentage` must be a value between 1 - 100

$ cat ./waves 
[[waves]]
start_after = 'in 12 hours'
fleet_percentage = 1

[[waves]]
start_after = 'in 1 day'
fleet_percentage = 10

[[waves]]
start_after = 'in 3 days'
fleet_percentage = 101

[[waves]]
start_after = 'in 6 days'
fleet_percentage = 100

$ cargo run --bin updata -- set-waves foobar --arch x86_64 --version 0.3.1 --variant aws-k8s-1.15 --wave-file ./waves
...
15:33:17 [ERROR] `fleet_percentage` must be a value between 1 - 100

• Calling updata with waves out of order fails as expected

$ cat ./waves 
[[waves]]
start_after = 'in 1 day'
fleet_percentage = 1

[[waves]]
start_after = 'in 12 hours'
fleet_percentage = 10

[[waves]]
start_after = 'in 3 days'
fleet_percentage = 25

[[waves]]
start_after = 'in 6 days'
fleet_percentage = 100

$ RUST_BACKTRACE=1 cargo run --bin updata -- set-waves foobar --arch x86_64 --version 0.3.1 --variant aws-k8s-1.15 --wave-file ./waves
...
15:35:59 [ERROR] Waves are not ordered: bound 20 occurs before bound 0

• Calling updata with a valid waves file usin a fake manifest.json works a treat:

$ cat foobar
{
  "updates": [
    {
      "variant": "aws-k8s-1.15",
      "arch": "x86_64",
      "version": "0.3.0",
      "max_version": "0.3.1",
      "waves": {},
      "images": {
        "boot": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-boot.ext4.lz4",
        "root": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-root.ext4.lz4",
        "hash": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-root.verity.lz4"
      }
    },
  ],
  "migrations": {}
}

$ cat ./waves
[[waves]]
start_after = 'in 12 hours'
fleet_percentage = 1

[[waves]]
start_after = 'in 1 day'
fleet_percentage = 10

[[waves]]
start_after = 'in 3 days'
fleet_percentage = 25

[[waves]]
start_after = 'in 6 days'
fleet_percentage = 100

$ cargo run --bin updata -- set-waves foobar --arch x86_64 --version 0.3.0 --variant aws-k8s-1.15 --wave-file ./waves
...

$ cat foobar
{
  "updates": [
    {
      "variant": "aws-k8s-1.15",
      "arch": "x86_64",
      "version": "0.3.0",
      "max_version": "0.3.1",
      "waves": {
        "0": "2020-04-03T11:08:59.483930025Z",
        "20": "2020-04-03T23:08:59.483949148Z",
        "204": "2020-04-05T23:08:59.483958597Z",
        "512": "2020-04-08T23:08:59.483967129Z"
      },
      "images": {
        "boot": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-boot.ext4.lz4",
        "root": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-root.ext4.lz4",
        "hash": "bottlerocket-aws-k8s-1.15-x86_64-0.3.0-faaec6e4-root.verity.lz4"
      }
    },
  ],
  "migrations": {}
}

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[zmrow]

Oops - fixed a tangling test data file in the wrong format and fixed the test.

[tjkirch]

We still do the same checks on waves, right? That they're in order, seed doesn't go down, etc.

I'm confused about testing, going to ask about that

[tjkirch]

Your testing in the description has fleet_percentage = 0 which should be rejected, I think, not only because 0 is explicitly rejected but because the waves are out of order... can you please update the testing, and make sure we're doing proper wave sanity checks?

[zmrow]

Gah - sorry about that. I was using a file with fleet_percentage = 0 when manually testing to ensure the program choked when it saw a zero. I'll fix the testing section of the description.

In regards to update order, I haven't changed the validation we're doing. update_metadata includes a function validate_updates which makes sure that the dates are sane and in ascending order. I only removed the ordered_waves() test because the purpose of the test is already covered in the set_waves() test and is therefore redundant.

Let me know if you have any other questions.

[zmrow]

Updated to address @tjkirch 's comments.

Also updated the main description with additional testing.

[tjkirch]

No blockers, but I think it'd be good to confirm that seeds don't go down, just like we confirm times don't go down. That could prevent some issues where users think the percentages are incremental from the previous value, perhaps.

[sam-aws]

🏄‍♂️

[zmrow]

Improve error messages for unordered waves and invalid fleet_percentage.

Add an additional comment to the validate_updates function that explains we will catch dates and fleet_percentages that are out of order.