Ability to tag EC2 instances ... to make for easier tracking/reaping #83

Open
chilts opened this Issue Jul 15, 2013 · 8 comments

Comments

Projects
None yet
3 participants
Contributor

chilts commented Jul 15, 2013

Ideas from @mostlygeek:

Some ideas I had:

  • tweak AWSBox code to tag instances with:
  • AWSBOX (makes it easier to find them in AWS)
  • AWSBOX_OWNER = email address ...
  • AWSBOX_SCHEDULE = to start/stop them at specific times, ie: when the owner is actually awake to use them
  • AWSBOX_TTL = 1 week, some time that the awsbox lives for. Automatic reaping (termination) of the EC2 box
  • others?

Once we figure this out, let's make another issue to be able to both list them and perhaps cull them too. :)

The AWSBOX and AWSBOX_OWNER should be easy. What do you think would be a good format for the schedule. It's something that will require a time zone and a really simple key like:

To keep it simple for AWSBOX_SCHEDULE. Either 24/7 or 12/7 or 12/5, and a timezone adjustment.

For the 12 hours/ 7 days, and 12 hours/5 days (M-F) it will be from 6am to 6pm UTC.
Then people can adjust to their working hours:

Example:
AWS_SCHEDULE=24/7
AWS_SCHEDULE=12/5,-8 (6am to 6pm, monday to friday, pacific standard time)

people can use the hour offset value to tweak to their working day.

Lots of options :), maybe 0111110,6-18,-8 ... which is: day of week, hours of day, time zone ... i could go on forever :)

Contributor

chilts commented Jul 16, 2013

@mostlygeek I think the AWSBOX_SCHEDULE is much harder and we should probably push that to a separate issue ... perhaps once we've done plain tags.

(Also, unless the instance is backed by a EBS, this won't work. Finally, what will be the 'thing' that starts and stops the instance ... let's take these to another issue.)

Just chatting about this, I think we have two "types" of tags. Active and passive.

  • Passive tags: AWSBOX, AWSBOX_OWNER, etc
  • Active tags: AWSBOX_SCHEDULE, AWSBOX_TTL, etc

The active tags need to be separated out and dealt with differently (and probably have more thought put in).

Am looking forward to doing some of this later today.

Contributor

chilts commented Jul 16, 2013

Please see #84 for Active tags. This issue is for the more straightforward passive tags. :)

Contributor

chilts commented Jul 19, 2013

After a conversation on IRC, we (@chilts and @mostlygeek) have decided to go and set all $AWSBOX_* environment vars as tags on the EC2 instance.

I'll do this first, then if any --tag command line args are set, they will override the the env vars.

@mostlygeek mostlygeek added a commit to mostlygeek/awsbox that referenced this issue Jul 19, 2013

@mostlygeek mostlygeek #83 improve compatibility with EC2 CLI env. variables 1e1c731
Contributor

chilts commented Jul 22, 2013

@mostlygeek now all of the AWSBOX* env vars are automatically read into the 'tags' list. If the same tag is specified on the command line the the env var is overridden.

The code is on this branch:

Contributor

lloyd commented Sep 26, 2013

shit, I didn't realize there was code around this feature. from discussions on list it's sounding like @mostlygeek is leaning toward a vastly simplified approach. He proposes a reaper algorithm of:

- has received less than X bytes/day, stop it. Runs once / day
- has been stopped for 10 days, send an email?
- has been stopped for 15 days, terminate it. 

it sounds like the awsbox tag and the email tag, and proper application restart upon reboot are all that are required to support this.

So... we implement those teensy things, release awsbox 0.6.2, and then go drink beers?

Contributor

lloyd commented Sep 26, 2013

Also, a nice bit that the reaper can defer to is termination protection. If we set the convention for when to use it, then it might be a nice tool to use that's minimal invention.

So, clearly restated. The reaper may use the following information about an instance to make decisions / nag:

  1. termination protection flag
  2. awsbox tag
  3. email tag
  4. time running
  5. instance activity

does this sound sufficient?

I've rethought the algorithm for how the reaper works. I think the new one is easier / better as it looks at activity rather than just time the box has been alive.

Do you think the send email step is essential? I rather leave it out to make the reaper dumber. Then I don't need to keep track of state like, "email sent", which I sort of hacked in using ec2-tags.

  • the reaper will only work on instances tagged with AWSBOX
  • I'm good w/ time running and instance activity

I'm 50/50 on the termination protection flag. Its all to easy to set no termination and then forget about the instances. If it is a dev machine, and it hasn't gotten any traffic for like 12 hours, then stopping it will be fine, since it is still there. I would like to start with this and if it sucks, let's add more logic then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment