Changing strategy lists

[drvinceknight]

Closes #664.

This renames strategies to all_strategies, and ordinary_strategies to strategies. I've kept ordinary_strategies in there tough just to not break anyones code. It it worth not doing that and just 'ripping off' the bandaid?

Also, @marcharper mentioned having the meta strategies in a separate list. Here I've got them still in the one unique list (strategies) but have also created another list recursive_strategies. We could call that expensive_strategies to be more generic perhaps?

Have included a tutorial also, just listing these. I ended up putting that in further capabilities.

[marcharper]

How do you feel about the name recursive_strategies? I'm not sure that's technically accurate but I understand what you are going for. Perhaps meta_strategies is fine?

Also we could simply measure the runtime of all of these strategies. The meta strategies are obviously at least an order of magnitude more expensive by construction, and the deterministic strategies are typically the fastest. Perhaps there is some natural clustering.

[drvinceknight]

How do you feel about the name recursive_strategies? I'm not sure that's technically accurate but I understand what you are going for. Perhaps meta_strategies is fine?

Yeah I don't think recursive_strategies is quite right. I thought about leaving it as meta_strategies but wanted to leave it open for 'something else' that would have long run times. How do you feel about something like 'long_run_time_strategies? It's pretty explicit although a bit of a mouthful...

Checking the run times of individual strategies would be cool :) I'm assuming you're not meaning to include that as an automatic classification. Perhaps a nice notebook to include? Or script to run and have travis report it? It could print out a rank of the strategies by run time (against some pre defined set of strategies) along with the run time itself. We could then use that to (if need be) add another strategy to the high_cost_strategies (just trying out another name there).

[marcharper]

I'll see what I can do, I have some code laying around that basically does this already without the timing part (for the Axelrod Examples plots), but it may take a while to run.

[drvinceknight]

but it may take a while to run.

So that might exclude travis... We could just run that as and when we feel it's needed... But yeah, sounds good, want to open it as a separate issue and stick with the meta strategies in there for now? (Subject to finding the right name for the list...)

[drvinceknight]

FWIW my vote would (currently) go to: long_run_time_strategies.

[drvinceknight]

Have just pushed 3 more commits:

1. e91ca26 and cd7140e just change the name to long_run_time_strategies
2. 411720c is used in the doc commit: it adds an option for the doctests to include ... as the output (here's an SO question with some info: http://stackoverflow.com/questions/17092215/how-enable-ellipsis-when-calling-python-doctest).

EDIT:

In the use case here it (the ...) at least can be used to check that the calls of the lists actually work (thought of it when changing the name of the long run time strategies which would have been easy to leave incorrect in the docs).

[drvinceknight]

Here's what the docs look like with ...:

[marcharper]

I'm computing the run times now, if we want to wait a bit. I suspect that MetaHunter is not too intense and could go in the standard list (plus it's a good strategy). We could also look at the big-O complexities in terms of e.g. turns for various strategies.

[drvinceknight]

Happy to wait :) 👍

[marcharper]

Runtimes: Each strategy plays each other strategy for 100 turns, 20 repetitions, and the mean and std dev is computed for each strategy (over all opponents). All times are in seconds.

As I suspected the MetaHunter is fine but the other meta strategies are among the slower strategies.

Here's the notebook code:

import timeit

import numpy as np
import pandas as pd

import axelrod as axl

strategies = axl.strategies
reps = 20

def play_matches(s1, s2, reps=20):
    match = axl.Match(players=(s1(), s2()), turns=100)
    for _ in range(reps):
        match.play()

data = []        

for s1 in strategies:
    times = []
    for s2 in strategies:
        t = timeit.timeit(lambda: play_matches(s1, s2, reps=reps), number=1)
        times.append(t / float(reps))
    data.append((str(s1()), np.mean(times), np.std(times)))

df = pd.DataFrame(data, columns=["Player Name", "Mean Time", "Std Time"])
df.sort_values(by="Mean Time", inplace=True, ascending=False)

df.to_csv("runtimes.csv")

[drvinceknight]

Neat. We should pop that code in the notebook repo :)

Have pushed again, removing MetaHunter from the long run strategies.