A simple and easy to use tool designed to significantly lower your Amazon AWS costs by automating the use of spot instances.
We believe that AWS EC2 is often pricier than it should be, and that the pricing models that can significantly reduce costs are hard to be reliably used by humans and are better handled by automation.
We developed a novel, simple but effective way to make it much more affordable for a significant number of existing setups within minutes, with minimal configuration changes, negligible additional infrastructure and runtime costs, safely and securely and without any vendor lock-in.
This already allows a large number of companies and individuals to significantly reduce their infrastructure costs or get more bang for the same buck. They can now easily get access to cheap compute capacity so they can spend their scarce resources developing innovative products that hopefully make the world a better place.
Once installed and enabled by tagging existing on-demand AutoScaling groups, AutoSpotting gradually replaces their on-demand instances with spot instances that are usually much cheaper, at least as large and identically configured to the group's members, without changing the group configuration in any way. For your peace of mind, you can also keep running a configurable number of on-demand instances given as percentage or absolute number.
This can be seen in action below, you can click to expand the animation:
It implements some complex logic aware of spot and on demand prices, including for different spot products and configurable discounts for reserved instances or large volume customers. It also considers the specs of all instance types and automatically places bids to instance types and prices chosen based on flexible configuration set globally or overridden at the group level using additional tags.
A single installation can handle all enabled groups in parallel across all available AWS regions, but can be restricted to fewer regions if desired.
Your groups will then monitor and use these spot instances just like they would do with your on-demand instances. They will automatically join your load balancer and start receiving traffic once passing the health checks.
The savings it generates are often in the 60-80% range, but sometimes even up to 90%, like you can see in the graph below.
The entire logic described above is implemented in a Lambda function deployed using CloudFormation or Terraform stacks that can be installed and configured in just a few minutes.
The stack assigns the function the minimal set of IAM permissions required for it to work and has no admin-like cross-account permissions. The entire code base can be audited to see how these permissions are being used and even locked down further if your audit discovers any issues. This is not a SaaS, there's no component that calls home and reveals details about your infrastructure.
The Lambda function is written in the Go programming language and the code is compiled as a static binary compressed and uploaded to S3. For evaluation or debugging purposes, the same binary can run out of the box locally on Linux machines or as a Docker container. Some people even run these containers on their existing Kubernetes clusters assuming the other resources provided by the stack are implemented in another way on Kubernetes.
The stack also consists of a Cron-like CloudWatch event, that runs the Lambda function periodically to take action against the enabled groups. Between runs your group is entirely managed by AutoScaling (including any scaling policies you may have) and load balancer health checks, that can trigger instance launches or replacements using the original on-demand launch configuration. These instances will be replaced later by better priced spot instances when they are available on the spot market.
Read here for more information and implementation details.
Frequently asked questions about the project are answered in the FAQ, please read this first before asking for support.
If you have additional questions not covered there, they can be easily added to the crowdsourced source of the FAQ and we'll do our best to answer them either there or on Gitter.
Just like in the above animation, it's as easy as launching a CloudFormation (or
Terraform)
stack and setting the (configurable) spot-enabled tag on the AutoScaling
groups where you want it enabled to true.
All the required infrastructure and configuration will be created automatically, so you can get started as fast as possible.
For more detailed information you can read this document
Community support is available on the gitter chat room, where the main authors and other users are likely to help you solve issues with these official binaries.
Unlike multiple commercial products in this space that cost a lot of money and attempt to lock you in, this project is fully open source and developed in the open by a vibrant community.
It was largely developed by volunteers who spent countless hours of their own spare time to make it easy for you to use. If you find it useful and you appreciate the work they put in it, please consider contributing to the development effort as well.
You can just try it out and give feedback, report issues, improve the documentation, write some code or assign a developer to work on it, or even just spread the word among your peers who might be interested in it. Any amount of help would be greatly appreciated and would make a huge difference to the project.
You can also contribute financially, we gladly accept recurrent tips on Patreon, regardless of the amount. These donations will pay for hosting infrastructure of the easy to install binaries and the project website, and will also encourage further development by the main author.
If you can, try also convincing your organization to use the official easy to install binaries for longer term, for a small fraction of their cost savings (see below for more details).
Note: Non-trivial code should be submitted according to the contribution guidelines.
The stack mentioned above can be used to conveniently get started within minutes, without setting up a build environment or any additional infrastructure so it can save you some time that you can better spend doing something else. It also requires some resources paid by the main author from his own pocket, so unfortunately he can't affort to sponsor it to everyone out there for free.
That's why the binaries and the automated installation scripts used by the above stack are distributed under a proprietary license that allows them to be used free of charge for a limited evaluation period, no longer than 14 days.
Further use is possible, but you have to contribute at least 5% of the generated savings as long as you keep using these binaries. This can be paid on a monthly basis through Patreon.
The license also forbids non-paying users to automate the installation of these official builds on a recurring basis, in order to reduce the costs incurred to the main author.
Any income left after paying for the infrastructure costs motivates the main author to allocate a proportional amount of time on a monthly basis for improving the software and supporting existing users.
Note:
- even though these builds are usually stable enough, they may not have been thoroughly tested yet and come with best effort community support.
- the authors of significant code contributions and/or their employers are allowed to use these builds free of charge without any time limit. The original author grants these exemptions in written form upon request, and decides on a case by case basis if the contribution is significant enough.
Carefully tested builds are also available for a slightly larger cut of your savings. They come with support from the main author, who will do his best to help you configure and successfully run AutoSpotting on your environment.
Your feature requests and issues will also be treated with higher priority.
These builds require a monthly contribution of at least 10% of the monthly savings, also paid through Patreon.
Please get in touch on gitter if you are interested in getting access to the stable builds.
It is recommended to use the binaries mentioned above, which are easy to install for you, support further development of the software and even allow you to get support from the main author.
But if you have some special needs that require some customizations or you don't want to rely on the author's infrastructure or contribute anything for longer term use of the software, you can always build and run your customized binaries that you maintain on your own, just keep in mind that those won't be supported in any way.
More details are available here
Autospotting is already used by hundreds of individuals and organizations around the world, and we estimate to already save them more than $1.000.000. Some of them we know of are mentioned in the list of notable users.
The following deserve a special mention for contributing significantly to the development effort (listed in alphabetical order):
This software is distributed under the terms of the MIT license.
The official binaries are licensed under this proprietary license.