Strategy transformers

[marcharper]

This PR introduces strategy transformers, which are ways of transforming the behavior of a strategy without rewriting the strategy's class. There is a generic transformer factory that will take a user-defined function and wrap the Player's strategy method.

For example, using FlipTransformer, we can turn Cooperator into Defector:

new_class = FlipTransformer(axelrod.Cooperator)
player = new_class() # It's the same as Defector now

Similarly we could turn AntiCycler into DefectingAntiCycler, playing

D DC DDC DDDC DDDDC ...

Instead of

C CD CCD CCCD CCCCD ...

Another transform adds TFT style retaliation to any other strategy:

RUA = RetailiateUntilApologyTransformer()
TFT = RUA(axelrod.Cooperator)
player = TFT()

This actually does transform Cooperator exactly into TitForTat (it passes all the tests for TFT, which specify TFT completely). For other strategies, it only affects what follows an opponent's defection, otherwise the player's desired plays are simply passed through.

There are additional transformers for the following modifications:

• Adding noise (similar to the noisy tournament but only to one of the players)
• Adding probabilistic defection forgiveness
• Defecting on the last N moves (only if the proper tournament attribute is available), or more generally finishing with any given sequence of moves
• Starting with any specific sequences of moves then playing as the strategy intended
• Internally tracking intended history (e.g. to infer ambient noise with)

If you've been wondering if MetaHunter would be better with a little TFT style retaliation, there's no need to write a new strategy! Moreover the transformations can be chained or composed, so you can add as many transformations to any strategy. The transformations are preserved when the player is cloned.

These transforms can also be used as class decorators. As an example in the library, I modified BackStabber's implementation:

@FinalTransformer([D, D, D]) # End with three defections
class BackStabber(Player):
...
    def strategy(self, opponent):
        if not opponent.history:
            return C
        if opponent.defections > 3:
            return D
        return C

while removing these two lines from the strategy method:

-        if len(opponent.history) > (self.tournament_attributes['length'] - 3):
-            return D

Now it works even if the tournament length is not known, and defects as intended if the length is available.

Some ideas for additional transforms:

• A grudger style never forgive
• Automatic noise detection
• Automatic memory_depth inference
• Adding hunting behavior to other strategies
• Add other types of retaliation (e.g. TitForTwoTats)

I only ran into one significant issue while testing -- the fact that the classifier dict is sometimes a class variable made it difficult to test, since it would overwrite the original class variable (in the super class). One workaround would be to set the classifiers in a method rather than have them as class variables.

Once #372 is merged I'll add a page of usage documentation.

[drvinceknight]

This looks interesting and obviously fits the goal of the library of facilitating the study of IPDs.

I need to get my head around it a bit more I think. I haven't looked at the code yet, so just thinking about the problem this solves: can you help me out a bit more than you already have. Are you thinking the (main) use case is for people using the library and wanting to create on the fly strategies? Or are these (mainly) to help create strategies?

Very much in favour, for example the @FinalTransformer([D, D, D]) is a lovely decorator and it helps 'idiot proof' as you say:

Now it works even if the tournament length is not known, and defects as intended if the length is available.

Just my usual initial slowness at getting my head around it 👍 😄

Once #372 is merged I'll add a page of usage documentation.

This is a perfect example of a 'further_topics' tutorial I think?
This is where the new modular tutorial docs I think come in to their own, instead of a big terrible monster being creating (as the previous docs were) we can make modular adjustments...

Also: anything we can do about 0.1% drop in coverage? (Not a disaster obviously)

[marcharper]

I'm sure that I can get the coverage up.

This should prevent code duplication and expand the possible strategies the library can produce substantially. There are already many strategies that are e.g. TFT but defect on round one instead.

Plus you can, for example, run a tournament of many strategies and apply a transform to all of them. What happens if everyone defects on the first round? If everyone retaliates like TFT?

[drvinceknight]

This should prevent code duplication and expand the possible strategies the library can produce substantially. There are already many strategies that are e.g. TFT but defect on round one instead.

Plus you can, for example, run a tournament of many strategies and apply a transform to all of them. What happens if everyone defects on the first round? If everyone retaliates like TFT?

Have had more time to think about it: big big fan.

Will look through the code itself as soon as possible.

[marcharper]

Have had more time to think about it: big big fan.

Glad to hear about it, as I'm fairly certain these transformers are the key to studying the IPD with category theory. I'll write something up eventually...

[drvinceknight]

Oh wow: that sounds awesome! Category theory is something I know very
little about.

On Fri, Oct 23, 2015 at 5:42 PM Marc Harper, PhD notifications@github.com
wrote:

Have had more time to think about it: big big fan.

Glad to hear about it, as I'm fairly certain these transformers are the
key to studying the IPD with category theory. I'll write something up
eventually...

—
Reply to this email directly or view it on GitHub
#380 (comment)
.

[meatballs]

Category theory is something I know very
little about.

I can only aspire to attain such a level of knowledge

[drvinceknight]

Be careful what you wish for :)

On Fri, 23 Oct 2015, 18:35 Owen Campbell notifications@github.com wrote:

Category theory is something I know very
little about.

I can only aspire to attain such a level of knowledge

—
Reply to this email directly or view it on GitHub
#380 (comment)
.

[marcharper]

I can only aspire to attain such a level of knowledge

It's nbd really, you just have to waste years of your life in mathematics graduate school, postponing other life goals and the development of a viable career in the meantime. Or learn a bit of Haskell I suppose.

[drvinceknight]

Lol. Slightly related. I've seen a quote somewhere, something like:

'graduate school is reducing current income in order to reduce future
income'

On Fri, 23 Oct 2015, 18:52 Marc Harper, PhD notifications@github.com
wrote:

I can only aspire to attain such a level of knowledge

It's nbd really, you just have to waste years of your life in mathematics
graduate school, postponing other life goals and the development of a
viable career in the meantime. Or learn a bit of Haskell I suppose.

—
Reply to this email directly or view it on GitHub
#380 (comment)
.

[meatballs]

Shouldn't this be two defections? The orginal condition was for opponent history length to be greater than 197, so 198 is the first occurrence. If the the history is 198 long, then we are on turn 199.

[marcharper]

Thanks -- I fixed it, and updated the tests.

[meatballs]

This is an excellent piece of work and opens up all sorts of possibilities. Other than my comment on backstabber, I'm more than happy to see this go in.

[drvinceknight]

This comment isn't quite right? It's do seq on last len(seq) actions right? The default is default...

[marcharper]

Good catch.

[drvinceknight]

Thought as much :)

On Sat, 24 Oct 2015, 22:59 Marc Harper, PhD notifications@github.com
wrote:

In axelrod/strategies/strategy_transformers.py
#380 (comment):

• return transformer

+def final_sequence(player, opponent, action, seq):

• """Play the moves in seq first, ignoring the strategy's
• moves until the list is exhausted."""
• length = player.tournament_attributes["length"]

• if length < 0: # default is -1

•    return action
  
• index = length - len(player.history)
• if index <= len(seq):

•    return seq[-index]
  

• return action

+# Defect on last N actions

Good catch -- that's what it was initially but then I generalized to an
arbitrary sequence.

—
Reply to this email directly or view it on GitHub
https://github.com/Axelrod-Python/Axelrod/pull/380/files#r42938684.

[drvinceknight]

Just some minor things I've commented on. I think the main thing this is missing is an advanced tutorial or further topics. I'd also suggest that a pointer gets put in the contribution docs to that tutorial but as you say I guess that is easiest if you wait for #372?

I completely agree with @meatballs: this is great and leaves open the possibility of further transformations to be added :) (that could be explained in the tutorial perhaps?)

[marcharper]

I will certainly improve the documentation -- both the docstrings and an advanced tutorial, both of which are needed since we're metaprogramming now. But I do want #372 to hit first so I don't have merge conflicts on the docs (and I wanted to make sure that you all liked the idea).

[drvinceknight]

Love the idea.

On Sat, 24 Oct 2015, 23:14 Marc Harper, PhD notifications@github.com
wrote:

I will certainly improve the documentation -- both the docstrings and an
advanced tutorial, both of which are needed since we're metaprogramming
now. But I do want #372
#372 to hit first so I
don't have merge conflicts on the docs (and I wanted to make sure that you
all liked the idea).

—
Reply to this email directly or view it on GitHub
#380 (comment)
.

[drvinceknight]

blank line needed here (before the code)

[drvinceknight]

Yeah this is brilliant. Just some blank lines to tidy things a bit.

I really like the tutorial.

Brilliant contribution, very exciting.

[drvinceknight]

👍

[langner]

This is very nice!