Skip to content
Reaper culls leftover AWS resources
Go Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
aws Merge pull request #88 from mozilla-services/remove-scaling-and-force… Oct 4, 2016
config remove scaling for instances and asgs, remove forcestop option, remov… Oct 4, 2016
events Merge pull request #90 from mozilla-services/better-reaperevent-logging Oct 20, 2016
filters fix some go vet formatting stuff Jul 11, 2016
prices add ohio region us-east-2 to prices Oct 26, 2016
reapable remove scaling for instances and asgs, remove forcestop option, remov… Oct 4, 2016
reaper Merge pull request #91 from mozilla-services/default-to-not-match Oct 20, 2016
reaperlog make mozlog optional Oct 26, 2016
state fix initial state to avoid race condition Aug 3, 2015
token remove scaling for instances and asgs, remove forcestop option, remov… Oct 4, 2016
vendor Update AWS SDK to 1.2.5 Jul 14, 2016
.gitignore add vendor into the repo Apr 6, 2016
CODE_OF_CONDUCT.md Add Mozilla Code of Conduct file Mar 28, 2019
Dockerfile expose port 8000 Jul 14, 2016
FILTERS.md Merge remote-tracking branch 'origin/master' into add-volumes, had to… Apr 7, 2016
LICENSE add license May 22, 2015
README.md remove scaling for instances and asgs, remove forcestop option, remov… Oct 4, 2016
circle.yml reorder circle push to dockerhub priority Aug 9, 2016
glide.lock Update AWS SDK to 1.2.5 Jul 14, 2016
glide.yaml Update AWS SDK to 1.2.5 Jul 14, 2016
main.go

README.md

AWS Reaper

About

The Reaper terminates forgotten AWS resources.

Reaper workflow:

  1. Find all enabled resources, filter them, then fire events based on config
    • Event types include sending emails, posting events to Datadog (Statsd), tagging resources on AWS, stopping or killing resources, and more
    • Reaper uses the Owner tag on resources to notify the owner of the resource with options to Ignore (for a time), Whitelist, Terminate, or Stop each resource they own
  2. Report statistics about the resources that were found
  3. Terminate or Stop abandoned resources after a set amount of time

Caution This app is experimental because:

  • doesn't have a lot of tests (Read: any), will add when SDK is updated

Building

  • install glide per https://github.com/Masterminds/glide
  • checkout repo
  • install dependencies: glide install
  • build binary: go build main.go
  • run binary: ./reaper -config config/default.toml
  • for command line options: ./reaper -help

Command Line Flags

  • config: required flag, the path to the Reaper config file. string (no default value)
  • dryrun: run Reaper in dryrun (no-op) mode. Events will not be triggered. boolean (default: true)
  • withoutCloudformationResources: skip checking for Cloudformation Resource dependencies (throttled by AWS, so it takes ages). boolean (default: false)

Creating a configuration file

Reaper configuration files should be in toml format. See config/default.toml for an example config.

  • Top level options
    • LogFile: the full filepath of the file that logs are written to. string
    • WhitelistTag: a string that will be used to tag resources that have been whitelisted. Defaults to REAPER_SPARE_ME. (string)
    • DefaultOwner: all unowned resources will be assigned to this owner. Can be an email address, or can be a username if DefaultEmailHost is specified. string
    • DefaultEmailHost: resources that do not have a complete email address as their owner will have this appended. Should be of the form "domain.tld". Works with DefaultOwner in the following way: DefaultOwner@DefaultEmailHost. string
    • EventTag: a tag that is added to all events that support tagging. Should be of the form key1:value1,key2:value2. string
  • HTTP options (under [HTTP])
    • TokenSecret: the secret key used to secure web requests. string
    • ApiURL: used to generate URLs for Reaper's HTTP API. Should be of the form protocol://host:port. string
    • Listen: where the HTTP server will listen for requests. Should be of the form host:port. string
    • Token: TODO
    • Action: TODO
  • Logging (under [Logging])
    • Extras: enables or disables extra logging, such as dry run notifications for EventReporters not triggering. boolean
  • States (under [States])
    • Interval: the interval between Reaper's scans for resources. The time format must be a duration parsable by Go's time.ParseDuration. See: http://godoc.org/time#ParseDuration. Example: 1h. string
    • FirstStateDuration: the length of the first state assigned to resources that match filters. The time format must be a duration parsable by Go's time.ParseDuration. See: http://godoc.org/time#ParseDuration. Example: 1h. string
    • SecondStateDuration: the length of the second state assigned to resources that match filters. The time format must be a duration parsable by Go's time.ParseDuration. See: http://godoc.org/time#ParseDuration. Example: 1h. string
    • ThirdStateDuration: the length of the third state assigned to resources that match filters. After the Third state elapses, resources move to a permanent final state. The time format must be a duration parsable by Go's time.ParseDuration. See: http://godoc.org/time#ParseDuration. Example: 1h. string
  • Events (under [Events])
    • Datadog ([Events.Datadog])
      • Enabled: enables or disables the Datadog EventReporter. Note: Datadog statistics and Event depend on this. boolean
      • Triggers: states for which Datadog will trigger Reapable Events. Can be any/all/none of first, second, third, final, or ignore. []string
    • Tagger ([Events.Tagger])
      • Enabled: enables or disables the Tagger EventReporter. boolean
      • Triggers: states for which Tagger will trigger Reapable Events. Can be any/all/none of first, second, third, final, or ignore. []string
    • Reaper ([Events.Reaper])
      • Enabled: enables or disables the Reaper EventReporter. boolean
      • Triggers: states for which Reaper will trigger Reapable Events. Can be any/all/none of first, second, third, final, or ignore. []string
      • Mode: when the Reaper EventReporter is triggered on a Reapable Event, it will Stop or Terminate Reapables per this flag. Note: modes must be capitalized. string
    • Email ([Events.Email])
      • Enabled: enables or disables the Email EventReporter. boolean
      • Triggers: states for which Email will trigger Reapable Events. Can be any/all/none of first, second, third, final, or ignore. []string
      • Host: the mailserver Reaper will use
      • AuthType: the type of authentication used by the mailserver. Should be one of none, md5 or plain. string
      • Port: the port used by the mailserver. int
      • Username: the username to use for the mailserver. string
      • Password: the password to use for the nmailserver. string
      • From: the address that Reaper will send mail from, must be parsable by Go's mail.ParseAddress. See: http://godoc.org/net/mail#ParseAddress. string
  • All Supported AWS Resource types have these properties
    • Enabled: enables or disables reporting of this resource type. Note: resources will still be queried for as they inform Reaper about the dependencies of other resources. boolean
    • FilterGroups (under [ResourceType.FilterGroups]): FilterGroups are sets of filters that can be applied to resources. In order for a resource to match a FilterGroup, it must match all filters in the FilterGroup. If an resource matches any FilterGroup, it has satisfied Reaper's filters. []FilterGroup
      • Example FilterGroup:

        [ResourceType.FilterGroups.Example]
                [ResourceType.FilterGroups.Example.Filter1]
                    function = "IsDependency"
                    arguments = ["false"]
                [ResourceType.FilterGroups.Example.Filter2]
                    function = "Running"
                    arguments = ["true"]
        
      • In this example, we see a FilterGroup named "Example" that has two Filters, Filter1 and Filter2.

      • A FilterGroup is a []Filter, and a Filter has two components, a function and arguments. The function is the name of the filtering function for the associated resource type (string), and arguments is a slice of arguments to that function ([]string).

  • Currently supported AWS Resource types:
    • SecurityGroups (under [SecurityGroups])
    • Cloudformations (under [Cloudformations])
    • AutoScalingGroups (under [AutoScalingGroups])
    • Instances (under [Instances])
    • Volumes (under [Volumes])
You can’t perform that action at this time.