[mig scheduler] Query AWS api to authorize agents connecting from aws (Bugzilla #1117147)

[jvehent]

Migrated from https://bugzilla.mozilla.org/show_bug.cgi?id=1117147
Assigned to: Aaron Meihm [:alm]

On 2015-01-02 09:54:23 -0800, Julien Vehent [:ulfr] wrote:

MIG needs to allow releng spot instances to register. They are ephemeral, such that manual whitelisting requires constant verification work. Instead, I�d like the scheduler to query the aws api to verify that an agent runs on an instance that exists and can be allowed to register.

Proposed algorithm:

1. mig-agent retrieves local instance meta-data by querying http://169.254.169.254/latest/meta-data/ and sends that information to the scheduler [1].
2. the scheduler queries AWS to look for an instance that matches the meta-data sent by the agent. It should, at a minimum, compare the following:
   • placement (region + AZ)
   • instance-id
   • instance-type
   • mac address
3. if a match is found, the agent is allowed to register. If not, a warning is logged.

The implementation must allow the scheduler to query multiple AWS accounts. We could use hostnames mapping to tie a hostname with one (or more) AWS access key.
For example:
{
"domains": [
{
"match": ".+.dmz.use1.mozilla.com$",
"lookup": {
"aws": {
"accesskey": "12345",
"secretkey": "abcdef"
}
}
},
{
"match": ".+.dmz.use1.mozilla.com$",
"lookup": {
"aws": {
"accesskey": "98271",
"secretkey": "ffffff"
}
}
}
]
}

The scheduler would go through the list of domain regexes, and attempt to verify the instance against each aws account that matches the regex until a match is found.

[1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-categories

[jvehent]

We've decided against this, and in fact disabled whitelisting entirely, since agents are already strongly authenticated.