Re factor/implement first tournament strategies #1275

drvinceknight · 2019-12-01T07:25:20Z

#1273 noted a number of potential implementation errors in the first tournament strategies.

This is a first draft of addressing these so we can discuss particular implementations.

I believe that a number of errors were a result of confusion between first tournament and second tournament code so I've renamed all the strategies: FirstBy<author> and SecondBy<author>. (I'm open to other suggestions).

In some places I've added a number of things to the docstrings to make explicit the assumptions made when descriptions are not clear.

Closes #1273

Happy to change the prefix etc...

Also make minor docstring amendments. Also made some notes regarding Downing.

I believe the logic was slightly faulty for Tullock: the first 11 moves was correct however once past there we should only be considering the 10 previous moves.

@id428

As noted by @id428 we were not giving a "true" fresh start so I've implemented that. I also added the two final defections (when the game length is known).

This is a complete rewrite of the Downing strategy. To be able to do this I've used the description in Downing's 1975 paper. This description itself is not sufficiently clear and so I've had to make some further assumptions which I've clearly documented. Note: there was documentation claiming that there was a bug in the implementation in the original tournament. I believe this was a mistake due to a misinterpretation of one online set of slides where they commented that there was a mistake in the implementation. This however was not a bug and was actually described quite a lot in Axelrod's original tournament: the strategy was implemented to act a particular way in the first two rounds and this had the result of making the strategy a king maker. This however was not a bug, just a particular interpretation of the overall decision rule described in Downing's 1975 paper.

@id428

Note that this was not the specific error that @id428 pointed out but having reviewed the papers and source code I found this one minor inaccuracy. @id428 made a point that the strategy should cooperate twice after it's round of retaliations but I do not see this in any of the descriptions of the strategy. Once the strategy has finished retaliating, all the texts indicate that it cooperates again but ready to retaliate.

drvinceknight · 2019-12-01T07:26:53Z

I've hopefully kept my commits quite modular and in some cases added text describing my approach to the commit message. For example, Downing required me to sift through Downing's 1975 paper: eccd7e2.

marcharper

What do you think about having a classifier entry for "in Axelrod's first tournament" and "in Axelrod's second tournament"? Then we can also easily add lists of first and second tournament strategies along side short_run_term_strategies for ease of use (and possibly a nice tutorial example, maybe even an advanced tutorial comparing to the Fortran implementations with fingerprints). We'd also need to add these classifiers to TFT.

axelrod/strategies/axelrod_first.py

axelrod/strategies/calculator.py

axelrod/tests/strategies/test_axelrod_first.py

marcharper · 2019-12-01T18:37:24Z

axelrod/strategies/axelrod_first.py

            return D
        return C

+class FirstByDowning(Player):


Since RevisedDowning is in the second tournament, can we also preserve that implementation and/or compare to the fortran implementation?

Sorry I meant to write that in my PR: we need to consider what we do with RevisedDowning. As it's in the second tournament is it worth waiting until we translate https://github.com/Axelrod-Python/TourExec/blob/v0.2.0/src/strategies/K59R.f or implement RevisedDowning as a modification of the strategy in this PR?

The current RevisedDowning looks really similar to the Fortran code. If they are basically the same I'm in favor of leaving RevisedDowning (eliminating the revised bool) as the second tournament implementation. Maybe comparing the fingerprints will tell us if it's essentially correct or not?

Yeah good call.

I'm struggling to get axelrod_fortran to work on my current machine (I blame an OS update), could you or @meatballs if you get time paste fingerprints for "k59r".

Something like:

import axelrod as axl import axelrod_fortran as axlf downing = axlf.Player("k59r") ashlock_fp = axl.AshlockFingerprint(strategy=downing) data = ashlock_fp.fingerprint() # This will take a little while p = ashlock_fp.plot() p.savefig("k59r_ashlock_fingerprint.png") transitive_fp = axl.TransitiveFingerprint(strategy=downing) data = transitive_fp.fingerprint() p = transitive_fp.plot() p.savefig("k59r_transitive_fingerprint.png")

Here are the equivalent for RevisedDowning:

Ashlock:

Transitive:

They are really similar but not 100% identical

Gosh they're incredibly similar though. I'm happy and looking at the history of RevisedDowning you implemented it from the Fortran code and I suspect you did it right.

Let's keep it and then tweak it in a follow-up PR

Fine by me. 👍

axelrod/strategies/axelrod_first.py

drvinceknight · 2019-12-02T09:12:31Z

What do you think about having a classifier entry for "in Axelrod's first tournament" and "in Axelrod's second tournament"? Then we can also easily add lists of first and second tournament strategies along side short_run_term_strategies for ease of use (and possibly a nice tutorial example, maybe even an advanced tutorial comparing to the Fortran implementations with fingerprints). We'd also need to add these classifiers to TFT.

Yeah I really like this idea. Perhaps a classifier is not worth doing as it's not something dynamic (we will never implement another strategy from the first tournament) - perhaps a hard coded list is the way to go?

Also - unrelated - I think we should just go ahead remove the cheating strategies (I know I've been on the other side of this for a long time), if only to clear up the classifiers. But that's a discussion for another time...

drvinceknight · 2019-12-02T09:12:49Z

Thanks @meatballs I think I've addressed all those.

marcharper · 2019-12-03T03:56:54Z

Yeah I really like this idea. Perhaps a classifier is not worth doing as it's not something dynamic (we will never implement another strategy from the first tournament) - perhaps a hard coded list is the way to go?

Hard-coded lists seem fine in this case.

Also - unrelated - I think we should just go ahead remove the cheating strategies (I know I've been on the other side of this for a long time), if only to clear up the classifiers. But that's a discussion for another time...

Sure, let's (eventually) at least silo them sufficiently so that no one inadvertently uses them.

marcharper · 2019-12-04T05:43:20Z

axelrod/tests/strategies/test_axelrod_second.py

@@ -2034,7 +2034,7 @@ def test_strategy(self):

 class TestSeconodByDowning(TestPlayer):


Seconod -> Second

Are we sure we don't want to call this one RevisedDowning ? (Here is where the classifier would separate the naming concerns from the inclusion in the first or second tournament...)

Yup, that'll be similar to TitForTat, Grudger etc... I'll change it.

a6471e8 renames this and moves it to its own file so that the description at the top of axelrod_second.py is still accurate.

Nikoleta-v3

Following a meeting with @drvinceknight today going through the paper Downing (1975) I have made a few comments on the implementation of Downing. Please let me know if something does not make sense 👍

Nikoleta-v3 · 2019-12-04T16:44:39Z

axelrod/strategies/axelrod_first.py

+    S, P(C_o | C_s) and the conditional probability that O will choose C
+    following D by S, P(C_o, D_s)."
+
+    Throughout the paper the strategy (S) assumes that the opponent (D) is


I believe you meant (O)

Nikoleta-v3 · 2019-12-04T16:48:02Z

axelrod/strategies/axelrod_first.py

+
+        EV_TOT = #CC(EV_CC) + #CD(EV_CD) + #DC(EV_DC) + #DD(EV_DD)
+
+    I.E. The player aims to maximise the expected value of being in each state


After our conversation today I feel (not sure) that this might need to re-written. #CC is not a state but the number of times the strategy S cooperated twice...

Yeah I agree, thanks @Nikoleta-v3 I'll work on this tomorrow (long day!).

Nikoleta-v3 · 2019-12-04T17:33:38Z

axelrod/strategies/axelrod_first.py

+    Then the opponent's first cooperation counts as a cooperation in response to
+    the non existent cooperation of round 0. The total number of cooperations in
+    response to a cooperation is 1. We need to take in to account that extra
+    phantom cooperation to estimate the probability alpha=P(C|C) as 1 / 1 = 1.


I have one question, why we don't start with alpha=P(C|C) = 0.5 as stated above?

Because that doesn't necessarily always imply 2 defections as an opening which is one of the "stronger" points made in the various literature. (But I'm guessing here.)

I'll investigate more and get back to you (and add to the docstring as well) 👍

6c46483 adds to the docstring on this topic and also no the other points you raised. Let me know what you think.

The explanation is really good now! One minor comment would be to change P(C | C) to P(C_o | C_s) to be consistent.

Yup good call.

I have moved this to it's own module/file.

drvinceknight · 2019-12-05T21:26:22Z

b9b00a2 adds a tutorial that just reproduces the first tournament (or fails to). I thought I might as well do this after your suggestion @marcharper, let me know what you think.

FYI, I'm currently running some code to iterate through random seeds to see if we can get the same results as Axelrod reported (no luck so far but it did give me the couple of examples I use in the tutorial). Here is the code I'm using to do that:

import axelrod as axl
import pandas as pd
import csv

def get_players():
    first_tournament_participants_ordered_by_reported_rank = [s() for s in axl.axelrod_first_strategies]
    return first_tournament_participants_ordered_by_reported_rank

def obtain_ranked_names(players, seed=0):
    axl.seed(seed)
    tournament = axl.Tournament(players=players, 
                                turns=200, 
                                repetitions=5)  # Axelrod's original tournament ran with 5 repetitions
    results = tournament.play(progress_bar=False)
    return results.ranked_names

def count_matches(ranked_names, ranked_players):
    first_tournament_ranked_names = [str(p) for p in ranked_players]
    return sum(reported == reproduced for reported, reproduced in zip(first_tournament_ranked_names, ranked_names))

def write_data(seed, number, winner, tit_for_tat_rank, filename, mode="a"):
    with open(filename, mode) as f:
        writer = csv.writer(f)
        writer.writerow([seed, number, tit_for_tat_rank, winner])
        
def check_seed(seed, players, filename):
    ranked_names = obtain_ranked_names(players=players, seed=seed)
    number_of_matches = count_matches(ranked_names=ranked_names, ranked_players=players)
    
    write_data(
        seed=seed, 
        number=number_of_matches, 
        tit_for_tat_rank=ranked_names.index("Tit For Tat"),
        winner=ranked_names[0], 
        filename=filename)
    
    return number_of_matches

players = get_players()
number_of_players = len(players)
filename = "seed_search.csv"

try:
    seed = pd.read_csv("seed_search.csv")["seed"].max() + 1
except FileNotFoundError:
    seed = 0
    write_data(seed="seed", 
               number="number", 
               winner="winner", 
               tit_for_tat_rank="tit_for_tat_rank", 
               filename=filename, 
               mode="w",
              )

while check_seed(
    seed=seed, 
    players=players, 
    filename=filename,
) != number_of_players:
    seed += 1

Nikoleta-v3 · 2019-12-05T21:29:27Z

axelrod/strategies/axelrod_first.py

+    playing C and the second D etc...
+    In this case the author uses an argument based on the sequence of plays by
+    the player (S) so #CC denotes the number of times the player plays C twice
+    in a row. This is then used to


drvinceknight · 2019-12-09T07:22:55Z

@Nikoleta-v3 could you confirm you're happy with this now when you get a moment?

Nikoleta-v3 · 2019-12-09T10:06:27Z

Looks good to me @drvinceknight!

drvinceknight · 2019-12-09T15:22:57Z

@marcharper @meatballs when either of your have time, I believe this is good to go now.

marcharper · 2019-12-09T16:26:43Z

LGTM. @meatballs want to take a final look since there were some post-approval changes?

drvinceknight added 16 commits November 18, 2019 20:10

Rename second tournament strategies to SecondBy

7520eec

Happy to change the prefix etc...

Modify name of Davis and Feld.

b5e8310

Also make minor docstring amendments. Also made some notes regarding Downing.

Add clone argument to Graaskamp.

0bf356c

First first by Grofman.

8c59778

Modify docstring and name of Joss.

d4e1552

Fix Nydegger.

33c903f

Rename Shubik but make notes for further investigation.

aa417a1

Rename and fix Tullock.

3dc60fb

I believe the logic was slightly faulty for Tullock: the first 11 moves was correct however once past there we should only be considering the 10 previous moves.

Rename Unnamed strategy.

75ee7c5

Rename Stein and Rapoport.

b54c272

Add a note to Graaskamp.

061e7f2

Revise Tideman and Cheruzzi.

0dde6e2

As noted by @id428 we were not giving a "true" fresh start so I've implemented that. I also added the two final defections (when the game length is known).

Adjust tests.

659d997

Adjust doctests.

ce1d6e6

drvinceknight mentioned this pull request Dec 1, 2019

Possible Implementation errors for first tournament strategies #1273

Closed

marcharper requested changes Dec 1, 2019

View reviewed changes

Make minor modifications suggested by @marcharper.

6e6a573

meatballs reviewed Dec 2, 2019

View reviewed changes

Address comments from Owen.

d8cee8d

meatballs approved these changes Dec 2, 2019

View reviewed changes

drvinceknight added 4 commits December 3, 2019 18:07

Address comments from @marcharper (RevisedDowning)

f4e162b

Update docs.

ecbd9c0

Add a list with all First strategies.

0c31588

Fix a typo.

e6b9349

Fix failing test.

e7cf5ad

marcharper reviewed Dec 4, 2019

View reviewed changes

Nikoleta-v3 reviewed Dec 4, 2019

View reviewed changes

drvinceknight added 4 commits December 5, 2019 09:12

Rename SecondByDowning -> RevisedDowning.

a6471e8

I have moved this to it's own module/file.

Add docstring about alpha=beta=1/2 in 1st 2 rounds.

6c46483

Correct name of RevisedDowning in docs.

e396646

Write tutorial using Axelrods first strategies.

b9b00a2

Run black on plotting script.

49e8983

Nikoleta-v3 reviewed Dec 5, 2019

View reviewed changes

drvinceknight added 3 commits December 5, 2019 21:43

Address @Nikoleta-v3's comments.

d24c304

Move test file to correct location.

468a829

Documentation modification.

36e82a2

marcharper approved these changes Dec 6, 2019

View reviewed changes

meatballs merged commit ea5b4b9 into master Dec 11, 2019

meatballs deleted the fix-first-tournament-second-tournament-confusion branch December 11, 2019 08:51

drvinceknight mentioned this pull request Dec 16, 2019

Update strategies to reflect new names in axelrod. Axelrod-Python/axelrod-fortran#81

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re factor/implement first tournament strategies #1275

Re factor/implement first tournament strategies #1275

drvinceknight commented Dec 1, 2019

drvinceknight commented Dec 1, 2019

marcharper left a comment

marcharper Dec 1, 2019

drvinceknight Dec 2, 2019

marcharper Dec 3, 2019

drvinceknight Dec 3, 2019

marcharper Dec 3, 2019 •

edited

drvinceknight Dec 3, 2019

marcharper Dec 4, 2019

drvinceknight Dec 4, 2019

drvinceknight commented Dec 2, 2019

drvinceknight commented Dec 2, 2019

marcharper commented Dec 3, 2019

marcharper Dec 4, 2019

drvinceknight Dec 4, 2019

drvinceknight Dec 5, 2019

Nikoleta-v3 left a comment

Nikoleta-v3 Dec 4, 2019

Nikoleta-v3 Dec 4, 2019 •

edited

drvinceknight Dec 4, 2019

Nikoleta-v3 Dec 4, 2019

drvinceknight Dec 4, 2019

drvinceknight Dec 4, 2019

drvinceknight Dec 5, 2019

Nikoleta-v3 Dec 5, 2019

drvinceknight Dec 5, 2019

drvinceknight commented Dec 5, 2019

Nikoleta-v3 Dec 5, 2019

drvinceknight commented Dec 9, 2019

Nikoleta-v3 commented Dec 9, 2019

drvinceknight commented Dec 9, 2019

marcharper commented Dec 9, 2019

		@@ -2034,7 +2034,7 @@ def test_strategy(self):

		class TestSeconodByDowning(TestPlayer):


		EV_TOT = #CC(EV_CC) + #CD(EV_CD) + #DC(EV_DC) + #DD(EV_DD)

		I.E. The player aims to maximise the expected value of being in each state

Re factor/implement first tournament strategies #1275

Re factor/implement first tournament strategies #1275

Conversation

drvinceknight commented Dec 1, 2019

drvinceknight commented Dec 1, 2019

marcharper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcharper Dec 3, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Dec 2, 2019

drvinceknight commented Dec 2, 2019

marcharper commented Dec 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nikoleta-v3 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nikoleta-v3 Dec 4, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Dec 5, 2019

Choose a reason for hiding this comment

drvinceknight commented Dec 9, 2019

Nikoleta-v3 commented Dec 9, 2019

drvinceknight commented Dec 9, 2019

marcharper commented Dec 9, 2019

marcharper Dec 3, 2019 •

edited

Nikoleta-v3 Dec 4, 2019 •

edited