Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Cheapest price #396

Open
wants to merge 7 commits into
from

Conversation

Projects
None yet
4 participants
Contributor

FinchPowers commented May 14, 2014

When creating spot instances with no specified zone, select the zone with the lowest bid price.

I noticed in your change that you are iterating over a limited set of zones and only in us-east-1.

  • def get_spot_cheapest_zone(self, instance_type):
  •    for z in ['a', 'c', 'd']:
    
  •        zone = 'us-east-1' + z
    

In my experience, the zone identifiers change between instance types and users. I have personally seen us-east-1b (extremely common), us-east-1e, and us-east-1f in the past. I recommend supporting any region (not just Virginia) and performing an API call to Amazon to determine the available zones for the instance type of interest.

Contributor

FinchPowers commented May 15, 2014

@darnells Cool! I'm open to this. Quickly looking at boto I didn't see any way to be able to list the zones where I'm able to create a node. If you want to add more to that PR go for it or if you have tips for me, that would be great.

@jtriley jtriley commented on an outdated diff May 16, 2014

starcluster/awsutils.py
@@ -1585,6 +1590,22 @@ def show_console_output(self, instance_id):
else:
log.info("No console output available...")
+ def get_spot_cheapest_zone(self, instance_type):
+ # Find cheapest zone
+ min_price = 9999
+ min_zone = None
+ for z in ['a', 'c', 'd']:
+ zone = 'us-east-1' + z
@jtriley

jtriley May 16, 2014

Owner

@FinchPowers We can't assume users are in us-east-1 region - use self.get_zones() instead. You can iterate through the list and get the zone name:

for z in self.get_zones():
    name = z.name # e.g. us-east-1a
    ...

@jtriley, thanks for pointing out the correct way to query available zones. I'm more biologist than computer scientist/developer, so I have limited experience with boto.

@FinchPowers, I've been sitting on an idea related to your current work. At one point, I was interested in selecting the cheapest, most stable zone for an instance type. Simply selecting the cheapest instantaneous price does not consider observable volatility within a zone (and experience has shown there is plenty of it), which increases the odds of being outbid.

My thought was to rank the zones by mu+n*sigma over a period of days, where mu is the hourly average, sigma is the standard deviation, n is a constant (like 2), and the lowest value wins. This way more expensive but stable zones may be ranked better than highly volatile but currently cheap zones.

In my hands, the price history returned by Amazon was only when the prices changed and the resolution of the data was dependent on length of date range, number of zones or instance types in the query, etc. I created some prototype code that extrapolates the price across unrepresented hours and simplifies the data by only considering the maximum price within an hour (rather than incorporating multiple price changes per hour).

If you are interested in this idea, we can discuss it in further detail (I am sure the idea can be improved) and I can share some code snippets (prototype quality). If not, your current idea is still quite useful and I look forward to it (hopefully) being included in StarCluster.

Contributor

FinchPowers commented May 20, 2014

@darnells I'm always open to improvements. Here are my first thoughts.

  • Your formula may return a low value even if the current price is high. What then?
  • What would be the proper value for n? If I understand, a low value means tolerance for fluctuations and a high value means less tolerance.

We can both agree that the goal is to minimize the overall run time cost of cloud instances. I am betting on savings over long periods (tens or hundreds of hours), where price variance concerns are more important. Clearly, there are multiple approaches that can be used. Perhaps the ideal solution is to create an interface where users can provide their own zone selection process in addition to some canned versions.

In your scenario, I expect the variance would be high (low mean but price spike occurring), which would be penalized by the standard deviation term. If the non-optimal zone is selected, then the max bid price acts as a kill switch to prevent exorbitant bids at the cost of losing the instance. In a perfect world, the load balancer could switch between spot and on-demand instances depending on current spot market trends, but that is clearly for another day.

As for a value for n, I would suggest making it a parameter with a default value of 2. For n = 0, there is no penalty for price variance. As n increases, so does the penalty. A value of n = 2 is close to the two-sigma value for a 95% confidence interval (but the sample size is neglected in this score). To me, it "feels about right" but YMMV.

Maybe it makes more sense to just calculate the 95% upper confidence limit (UCL) and pick the zone with the lowest bound value. It's more complicated to calculate: UCL = mu + T * sigma / sqrt(n), where mu is the sample mean, sigma is the sample standard deviation, n is the sample size, and T is the Student's t value with alpha = 0.05 and n-1 degrees of freedom. This assumes that hourly spot prices are independent and come from the same population (a stretch since available resources vary over time, but maybe reasonable enough). I expect the central limit theorem would apply when sampling over 4 days (96 hourly data points).

How do you deal with the EBS volumes you are trying to mount being localized to a given zone?

Contributor

FinchPowers commented Sep 12, 2014

I guess I don't have that.
The way I work is my master node has all the necessary binaries. The other nodes mount them via nfs. All other files are saved/stored in S3.

Then it might make sense to check for the usage of volumes and perhaps even what zone the volume can be used in and then use that zone automatically (a possible added benefit of the PR).

FinchPowers added some commits Mar 24, 2015

VPC cluster checks for vpc prices fix
Conflicts:

	starcluster/awsutils.py
	starcluster/cluster.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment