Fix bug in classification of transformers #972

drvinceknight · 2017-04-14T16:12:26Z

Our transformers were not reclassifying the strategies (mostly we weren't trying but in one case we were but it was not tested and not working). See #971

This implements the ability to pass a reclassifier function to the strategy transformer factory so that we can reclassify based on input parameters (for example if using the Noisy transformer but passing noise of 0 or 1 then the classifier does not change).

I've also written such reclassifiers for all relevant transformers EDIT: although technically the Mixed strategy transformer isn't as smart as it could be but I suggest this gets merged under the bug fix rule and an issue gets opened for that transformer. (I've set it to blankly say that it make strategies stochastic which is what we want for any sensible use case). I've done that one too now.

Closes #971

eric-s-s · 2017-04-14T16:39:55Z

axelrod/strategy_transformers.py

+    if sum(probability) > 1:
+        probability = tuple([i / sum(probability) for i in probability])
+
+    if probability[0] in [0, 1]:


will this fail on probability=(0.0, 0.5) or the very useless (0.0, 0.0)?

I don't think so because of the normalisation step that happens before. I'm fixing some remaining failures and will take a look at the (0, 0) case! 👍

You were right, that did fall over on those cases :) In 149e2e6 I've added more tests and a fix.

- Final transformer reclassification not tested (or working) - Random transformers not updated the stochastic classification.

Implement a reclassifier option that can be passed to the Transformer factor. This allows for smart reclassification of strategies based on the input parameters. - Also added a test to ensure that original class is not touched.

This was an early attempt an in essence doesn't do anything.

marcharper · 2017-04-15T16:01:10Z

Do we have a test that catches the original issue with cloning these players?

eric-s-s · 2017-04-15T16:10:23Z

axelrod/tests/unit/test_strategy_transformers.py

@@ -349,6 +420,7 @@ def test_mixed(self):
        probability = (0, 0, 1)  # Note can also pass tuple
        strategies = [axelrod.TitForTat, axelrod.Grudger, axelrod.Defector]
        MD = MixedTransformer(probability, strategies)(axelrod.Cooperator)
+        self.assertFalse(MD.classifier["stochastic"])

        p1 = MD()
        # Against a cooperator we see that we only cooperate


i've been reading through this to try and wrap my head around decorators, which drive me up the wall. it has helped but it's also hard reading. This comment that sneaked in from a copy-paste made me laugh.

Removing that now :)

which drive me up the wall.

Tell me about it! There be dragons... :'(

drvinceknight · 2017-04-15T16:25:20Z

Do we have a test that catches the original issue with cloning these players?

I've incorporated a number of tests that check that players are now correctly classified.

This was the original issue (stochastic players were not being recognised as such and they were starting off the cache). EDIT: My first guess was cloning but as I commented later on in #971 it's nothing to do with cloning. It was just that we weren't classifying things properly.

I haven't specifically incorporated a test of the exact symptom but the new tests for the fingerprint (showing that the previous values weren't right) show that this fixes #971

meatballs

Very minor nitpicking stuff from me

meatballs · 2017-04-16T13:34:56Z

axelrod/strategy_transformers.py

@@ -37,6 +39,8 @@ def __call__(self, player, opponent, action)
        Any keyword arguments to pass to the wrapper
    name_prefix: string, "Transformed "
        A string to prepend to the strategy and class name
+    reclassifier: function,
+        A function to use to update the class's classifier.


Minor quibbles:

Somebody new to this might reasonably ask "What class?"

"to use" is redundant

How about: "A function which will update the classifier of the strategy being transformed"?

meatballs · 2017-04-16T13:36:27Z

axelrod/strategy_transformers.py

@@ -229,9 +242,14 @@ def noisy_wrapper(player, opponent, action, noise=0.05):
        return flip_action(action)
    return action

+def noisy_reclassifier(original_classifier, noise):
+    """Function to reclassify the strategy"""
+    if noise not in [0, 1]:


Really nitpicky comment: I'd use a tuple here as there is no need for a list

meatballs · 2017-04-16T13:36:48Z

axelrod/strategy_transformers.py

@@ -241,9 +259,15 @@ def forgiver_wrapper(player, opponent, action, p):
        return random_choice(p)
    return C

+def forgiver_reclassifier(original_classifier, p):
+    """Function to reclassify the strategy"""
+    if p not in [0, 1]:


same nitpicky tuple vs list comment as previous

- Better docstring description. - Use tuples instead of lists.

drvinceknight · 2017-04-16T13:52:01Z

Have addressed both those @meatballs. I still often slip in the bad habit of using lists when I shouldn't... 👍

meatballs · 2017-04-16T13:52:53Z

If only that were my only bad habit..... Vince Knight wrote:

…

Have addressed both those @meatballs <https://github.com/meatballs>. I still often slip in the bad habit of using lists when I shouldn't... 👍 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#972 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUrntC_Sya4ZVBOR-Lkk72-0CQ94nuSks5rwh0CgaJpZM4M93-7>.

marcharper

Just a question on memory_depth for the transformers

marcharper · 2017-04-17T02:38:55Z

axelrod/strategy_transformers.py

@@ -294,9 +318,15 @@ def final_sequence(player, opponent, action, seq):
        return seq[-index]
    return action

+def final_reclassifier(original_classifier, seq):


We probably need to update the memory depth as well (also for initial transformer)

Sure doing that now.

Also added a small extension of a test that ensures strategy transformers are completely covered by their unit tests (I assume this was being covered by another test somewhere else).

drvinceknight · 2017-04-17T06:52:34Z

I've added reclassification of the memory depth too.

I think this should go in now under the bug fix rule. If there's anything remaining we can pick it up as other issues (I don't think there are but there might be other reclassifiers that need to be added but the main bug was being caused by the stochastic ones).

drvinceknight added the bug label Apr 14, 2017

drvinceknight mentioned this pull request Apr 14, 2017

Bug with strategy transformers #971

Closed

eric-s-s reviewed Apr 14, 2017

View reviewed changes

drvinceknight added 3 commits April 14, 2017 20:05

Write failing tests: classification of transformers

413ad00

- Final transformer reclassification not tested (or working) - Random transformers not updated the stochastic classification.

Make tests pass.

ca5fa33

Implement a reclassifier option that can be passed to the Transformer factor. This allows for smart reclassification of strategies based on the input parameters. - Also added a test to ensure that original class is not touched.

Adjust incorrect fingerprint examples.

63b3179

drvinceknight force-pushed the 971 branch from 149e2e6 to 63b3179 Compare April 14, 2017 19:07

drvinceknight added the ready-for-review label Apr 14, 2017

drvinceknight added 3 commits April 14, 2017 20:15

Remove redundant line.

e0e9154

This was an early attempt an in essence doesn't do anything.

Remove unnecessary line.

37c88e5

Add proper classification of mixer.

ad11de5

eric-s-s reviewed Apr 15, 2017

View reviewed changes

Fix copy and paste comment.

e67b28b

eric-s-s mentioned this pull request Apr 15, 2017

refactor for fingerprint #970

Closed

meatballs requested changes Apr 16, 2017

View reviewed changes

Address Owen's comments.

80b5a4a

- Better docstring description. - Use tuples instead of lists.

meatballs approved these changes Apr 16, 2017

View reviewed changes

marcharper requested changes Apr 17, 2017

View reviewed changes

Add reclassification of memory depth.

2e97acb

Also added a small extension of a test that ensures strategy transformers are completely covered by their unit tests (I assume this was being covered by another test somewhere else).

marcharper approved these changes Apr 17, 2017

View reviewed changes

marcharper merged commit c210af6 into master Apr 17, 2017

drvinceknight mentioned this pull request Apr 17, 2017

Refactor the static methods to functions in fingerprint #974

Closed

drvinceknight deleted the 971 branch April 19, 2017 17:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bug in classification of transformers #972

Fix bug in classification of transformers #972

drvinceknight commented Apr 14, 2017 •

edited

Loading

eric-s-s Apr 14, 2017

drvinceknight Apr 14, 2017

drvinceknight Apr 14, 2017

marcharper commented Apr 15, 2017

eric-s-s Apr 15, 2017

drvinceknight Apr 15, 2017

drvinceknight Apr 15, 2017

drvinceknight commented Apr 15, 2017 •

edited

Loading

meatballs left a comment

meatballs Apr 16, 2017

meatballs Apr 16, 2017

meatballs Apr 16, 2017

drvinceknight commented Apr 16, 2017

meatballs commented Apr 16, 2017 via email

marcharper left a comment

marcharper Apr 17, 2017

drvinceknight Apr 17, 2017

drvinceknight commented Apr 17, 2017

Fix bug in classification of transformers #972

Fix bug in classification of transformers #972

Conversation

drvinceknight commented Apr 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcharper commented Apr 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Apr 15, 2017 • edited Loading

meatballs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Apr 16, 2017

meatballs commented Apr 16, 2017 via email

marcharper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Apr 17, 2017

drvinceknight commented Apr 14, 2017 •

edited

Loading

drvinceknight commented Apr 15, 2017 •

edited

Loading