Skip to content
Permalink
master
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time
---
title: Is behavioral flexibility manipulatable and, if so, does it improve flexibility and problem solving in a new context?
author:
- '[Logan CJ](http://CorinaLogan.com)^1^*'
- 'Bergeron L^2^'
- '[Blaisdell AP](http://pigeonrat.psych.ucla.edu)^3^'
- '[Breen AJ](http://www.alexisbreen.com)^1^'
- 'Folsom M^1^'
- 'Johnson-Ulrich Z^2^'
- '[Lukas D](http://dieterlukas.mystrikingly.com)^1^*'
- '[MacPherson M](http://maggiepmacpherson.com)^2^'
- 'Rowney C^1^'
- '[Seitz B](https://benjaminseitz.wixsite.com/mysite)^3^'
- 'Sevchik A^4^'
- '[McCune KB](https://www.kelseymccune.com)^2^'
date: '`r Sys.Date()`'
always_allow_html: yes
output:
html_document:
toc: true
toc_depth: 4
toc_float:
collapsed: false
code_folding: hide
github_document:
toc: true
pdf_document:
keep_tex: yes
latex_engine: xelatex
md_document:
toc: true
bibliography: MyLibrary.bib
csl: https://raw.githubusercontent.com/citation-style-language/styles/master/apa.csl
urlcolor: blue
header-includes:
- \usepackage[left]{lineno}
- \linenumbers
- \usepackage{fancyhdr}
---
Open... ![](logoOpenAccess.png){width=5%} access ![](logoOpenCode.png){width=5%} [code](https://github.com/corinalogan/grackles/blob/master/Files/Preregistrations/g_flexmanip.Rmd) ![](logoOpenPeerReview.png){width=5%} peer review ![](logoOpenData.png){width=5%} [data]()
\addtolength{\headheight}{0.1cm}
\pagestyle{fancyplain}
\lhead{\includegraphics[height=1.2cm]{logoPCIecology.png}}
\renewcommand{\headrulewidth}{0pt}
 
**Affiliations:** 1) Max Planck Institute for Evolutionary Anthropology, Leipzig, Germany 2) University of California Santa Barbara, USA 3) University of California Los Angeles, USA, 4) Arizona State University, Tempe, AZ USA. *Corresponding author: corina_logan@eva.mpg.de
```{r setup, include=FALSE}
library(knitr)
knitr::opts_chunk$set(tidy.opts=list(width.cutoff=70),tidy=TRUE)
#Make code wrap text so it doesn't go off the page when Knitting to PDF
knitr::opts_chunk$set(echo = TRUE)
#sets global options to display code along with the results https://exeter-data-analytics.github.io/LitProg/r-markdown.html
```
 
<img width="50%" src="logoPCIecology.png">
**Cite as:** Logan CJ, MacPherson M, Rowney C, Bergeron L, Seitz B, Blaisdell AP, Folsom M, Johnson-Ulrich Z, McCune K. 2019. [Is behavioral flexibility manipulatable and, if so, does it improve flexibility and problem solving in a new context?](http://corinalogan.com/Preregistrations/g_flexmanip.html) (http://corinalogan.com/Preregistrations/g_flexmanip.html) In principle acceptance by *PCI Ecology* of the version on 26 Mar 2019 [https://github.com/corinalogan/grackles/blob/master/Files/Preregistrations/g_flexmanip.Rmd](https://github.com/corinalogan/grackles/blob/master/Files/Preregistrations/g_flexmanip.Rmd).
<img width="5%" src="logoOpenAccess.png"> <img width="5%" src="logoOpenCode.png"> <img width="5%" src="logoOpenPeerReview.png">
**This preregistration has been pre-study peer reviewed and received an In Principle Recommendation by:**
Aurélie Coulon (2019) Can context changes improve behavioral flexibility? Towards a better understanding of species adaptability to environmental changes. *Peer Community in Ecology*, 100019. [10.24072/pci.ecology.100019](https://doi.org/10.24072/pci.ecology.100019). Reviewers: Maxime Dahirel and Andrea Griffin
## ABSTRACT
Behavioral flexibility, the ability to adapt behavior to new circumstances, is thought to play an important role in a species' ability to successfully adapt to new environments and expand its geographic range. However, behavioral flexibility is rarely directly tested in species in a way that would allow us to determine how it works and how we can make predictions about a species' ability to adapt their behavior to new environments. We use great-tailed grackles (a bird species) as a model to investigate this question because they have rapidly expanded their range into North America over the past 140 years. We attempted to manipulate grackle behavioral flexibility (color tube reversal learning) to determine whether their flexibility is generalizable across contexts (touch screen reversal learning and multi-access box), whether it is repeatable within individuals and across contexts, and what learning strategies they employ. We found that we were able to manipulate flexibility: birds in the experimental group reversed in fewer trials by the end of their serial reversals compared to control birds who had only one reversal. Flexibility WAS WAS NOT repeatable within individuals. We found that flexibility WAS WAS NOT generalizable to a multi-access box context, but not to a touchscreen context. The responses to the touchscreen reversal experiment appeared to answer a different question - one that we were not aware we were asking. Further research on the cues that confound the test on the touchscreen are needed to understand this result. They employed X learning strategies. Our findings allow us to understand more about what flexibility is and how it works.
## [Video summary](https://youtu.be/bALXB2S4OpI)
## INTRODUCTION
Behavioral flexibility, the ability to adapt behavior to new circumstances (see @mikhalevich_is_2017 for the theoretical background on this definition), is thought to play an important role in a species' ability to successfully adapt to new environments and expand its geographic range [e.g., @lefebvre1997feeding; @sol2000behavioural; @sol2002behavioural; @sol2005big; @sol2007big]. This research predicts that behavioral flexibility (hereafter referred to as flexibility) should positively relate with innovativeness. However, these predictions are based on species-level data and proxies for flexibility and for innovation when examining such relationships [see @logan2018beyond]. Flexibility is rarely directly tested in species that are rapidly expanding their geographic ranges in a way that would allow us to determine how flexibility works and how we can make predictions about a species' ability to adapt their behavior to new areas Those investigations that examine the relationship between flexibility and innovation (or problem solving) in species that are expanding their range show mixed results, with these variables correlating positively [e.g., grey squirrels: @chow2016practice], negatively [e.g., Indian mynas: @griffin2013tracking], or not at all [e.g., great-tailed grackles: @logan2016behavioral]. One way to improve our understanding of whether and how flexibility is related to innovativeness is to perform a manipulative experiment on one of the variables to determine whether there is an associated change in the other.
We aimed to determine whether flexibility is manipulatable and, if so, whether there are associated differences in flexibility in a new context and in innovativeness in the manipulated group of great-tailed grackles (*Quiscalus mexicanus*), a bird species that is flexible [@logan2016behavioral] and rapidly expanding its geographic range [@wehtje2003range]. We attempted to manipulate grackle flexibility using serial reversals of color tube preferences to determine whether their flexibility is generalizable across contexts (touch screen reversal learning and multi-access box solution switching), whether it is repeatable within individuals and across contexts, and what learning strategies they employ.
If grackle flexibility is manipulatable using serial reversals, this would provide conservation managers with a useful tool for managing populations in this, and potentially other, species. If the manipulation works in grackles, it has the potential to also be effective in other species, which would be particularly useful for endangered species because individuals that are more flexible might be able to adapt better to new environments. If the flexibility manipulation is not successful, this could indicate either that we did not manipulate the right aspect of flexibility (e.g., perhaps training them to solve a variety of different types of tasks quickly would be more effective) or that grackle flexibility is not a trait that is trainable.
## HYPOTHESES
#### H1: Behavioral flexibility, as measured by reversal learning using colored tubes, is manipulatable.
**Prediction 1:** Individuals improve their flexibility on a serial reversal learning task using colored tubes by generally requiring fewer trials to reverse a preference as the number of reversals increases (manipulation condition). Their flexibility on this test will have been manipulated relative to control birds who do not undergo serial reversals. Instead, individuals in the control condition will be matched to manipulated birds for experience (they will experience a similar number of trials), but there will be no possibility of a functional tube preference because both tubes will be the same color and both will contain food, therefore either choice will be correct.
**P1 alternative 1:** If the number of trials to reverse a preference does not correlate with or positively correlates with reversal number, which would account for all potential correlation outcomes, this suggests that some individuals may prefer to rely on information acquired previously (i.e., they are slow to reverse) rather than relying on current cues (e.g., the food is in a new location) [e.g., @manrique_repeated_2013; @griffin2014innovation; @liu2016learning; but see @homberg2007serotonin].
#### H2: Manipulating behavioral flexibility (improving reversal learning speed through serial reversals using colored tubers) improves flexibility (rule learning and/or switching) and problem solving in a new context (two distinct multi-access boxes and serial reversals on a touch screen).
**P2:** Individuals that have improved their flexibility on a serial reversal learning task using colored tubes (requiring fewer trials to reverse a preference as the number of reversals increases) are faster to switch between new methods of solving (latency to solve or attempt to solve a new way of accessing the food [locus]), and learn more new loci (higher total number of solved loci) on multi-access box flexibility tasks, and are faster to reverse preferences in a serial reversal task using a touch screen than individuals in the control group where flexibility has not been manipulated. The positive correlation between reversal learning performance using colored tubes and a touch screen (faster birds have fewer trials) and the multi-access boxes (faster birds have lower latencies) indicates that all three tests measure the same ability even though the multi-access boxes require inventing new rules to solve new loci (while potentially learning a rule about switching: "when an option becomes non-functional, try a different option") while reversal learning requires switching between two rules ("choose light gray" or "choose dark gray") or learning the rule to "switch when the previously rewarded option no longer contains a reward". Serial reversals eliminate the confounds of exploration, inhibition, and persistence in explaining reversal learning speed because, after multiple reversals, what is being measured is the ability to learn one or more rules. If the manipulation works, this indicates that flexibility can be influenced by previous experience and might indicate that any individual has the potential to move into new environments (see relevant hypotheses in preregistrations on [genetics](https://github.com/corinalogan/grackles/blob/master/EasyToReadFiles/g_flexgenes.md) (R1) and [expansion](https://github.com/corinalogan/grackles/blob/master/EasyToReadFiles/g_expansion.md) (H1)).
**P2 alternative 1:** If the manipulation does not work in that those individuals in the experimental condition do not decrease their reversal speeds more than control individuals, then this experiment will elucidate whether general individual variation in flexibility relates to flexibility in new contexts (two distinct multi-access boxes and serial reversals on a touch screen) as well as problem solving ability (multi-access boxes). The prediction is the same in P2, but in this case variation in flexibility is constrained by traits inherent to the individual (some of which will be tested in a separate [preregistration](https://github.com/corinalogan/grackles/blob/master/EasyToReadFiles/g_exploration.md)), which suggests that certain individuals will be more likely to move into new environments.
**P2 alternative 2:** If there is no correlation between reversal learning speed (colored tubes) and the latency to solve/attempt a new locus on the multi-access boxes, this could be because the latency to solve not only measures flexibility but also innovativeness. In this case, an additional analysis will be run with the latency to solve as the response variable, to determine whether the fit of the model (as determined by the lower AIC value) with reversal learning as an explanatory variable is improved if motor diversity (the number of different motor actions used when attempting to solve the multi-access box) is included as an explanatory variable. If the inclusion of motor diversity improves the model fit, then this indicates that the latency to solve a new locus on the multi-access box is influenced by flexibility (reversal learning speed) and innovation (motor diversity).
**P2 alternative 3:** If there is a negative correlation or no correlation between reversal learning speed on colored tubes and reversal learning speed on the touch screen, then this indicates that it may be difficult for individuals to perceive and/or understand images on the touch screen in contrast with physical objects (colored tubes) [e.g., @ohara2015advantage].
![Figure 1. A visual illustration of Hypothesis 1 (A), Hypothesis 2 (B), Hypothesis 3 (C1 and C2), and Hypothesis 4 (D).](g_flexmanipFig1.png)
#### H3a: Behavioral flexibility within a context is repeatable within individuals.
Repeatibility of behavioral flexibility is defined as the number of trials to reverse a color preference being strongly negatively correlated within individuals with the number of reversals.
**P3a:** Individuals that are faster to reverse a color preference in the first reversal will also be faster to reverse a color preference in the second, etc. reversal due to natural individual variation.
**P3a alternative:** There is no repeatibility in behavioral flexibility within individuals, which could indicate that performance is state dependent (e.g., it depends on their fluctuating motivation, hunger levels, etc.). We will determine whether performance on colored tube reversal learning related to motivation by examining whether the latency to make a choice influenced the results. We will also determine whether performance was related to hunger levels by examining whether the number of minutes since the removal of their maintenance diet from their aviary plus the number of food rewards they received since then influenced the results.
#### H3b: The consistency of behavioral flexibility in individuals across contexts (context 1=reversal learning on colored tubes, context 2=multi-access boxes, context 3=reversal learning on touch screen) indicates their ability to generalize across contexts.
Individual consistency of behavioral flexibility is defined as the number of trials to reverse a color preference being strongly positively correlated within individuals with the latency to solve new loci on each of the multi-access boxes and with the number of trials to reverse a color preference on a touch screen (total number of touch screen reversals = 5 per bird).
*If P3a is supported (repeatability of flexibility within individuals)...*
**P3b:** ...and flexibility is correlated across contexts, then the more flexible individuals are better at generalizing across contexts.
**P3b alternative 1:** ...and flexibility is not correlated across contexts, then there is something that influences an individual's ability to discount cues in a given context. This could be the individual's reinforcement history (tested in P3a alternative), their reliance on particular learning strategies (one alternative is tested in H4), or their motivation (tested in P3a alternative) to engage with a particular task (e.g., difficulty level of the task).
#### H4: Individuals should converge on an epsilon-first learning strategy (learn the correct choice after one trial) as they progress through serial reversals.
**P4:** Individuals will prefer a mixture of learning strategies in the first serial reversals (an *epsilon-decreasing* strategy where individuals explore both options extensively before learning to prefer the rewarded option, and an *epsilon-first* strategy where the correct choice is consistently made after the first trial), and then move toward the epsilon-first learning strategy. The epsilon-first strategy works better later in the serial reversals where the reward is all or nothing because individuals will have learned the environment is changing in predictable ways [@bergstrom2004shannon]: only one option is consistently rewarded, and if the reward isn't in the previously rewarded option, it must be in the other option.
**P4 alternative 1:** Individuals will continue to prefer a mixture of learning strategies, and/or they do not converge on the more functional epsilon-first learning strategy, regardless of how many reversals they participate in. This pattern could suggest that the grackles do not attend to functional meta-strategies, that is, they do not learn the overarching rule (once food is found in the non-preferred tube, one must switch to preferring that tube color), but rather they learn each preference change as if it was new.
## ASSOCIATED PREREGISTRATION
Our methods and analysis plans are described in the peer-reviewed preregistration of this article, which is included below as the [Methods](#methods). We moved the hypotheses from the preregistration below to the section above to improve flow for the reader.
### DEVIATIONS FROM THE PREREGISTRATION
**In the middle of data collection**
1) 10 April 2019: We are discontinuing the reversal learning experiment on the touchscreen because it appears to measure something other than what we are trying to test and it requires a huge time investment for each bird (which consequently reduces the number of other tests they are available to participate in). This is not necessarily surprising because this is the first time touchscreen tests have been conducted in this species, and also the first time this particular reversal experiment has been conducted on a touchscreen with birds (to our knowledge). So we had no idea going into it what was going to happen. We are basing this decision on data from four grackles (2 in the flexibility manipulation group and 2 in the flexibility control group; 3 males and 1 female). All four of these individuals show highly inconsistent learning curves and require hundreds of trials to form each preference when compared to the performance of these individuals on the color tube reversal experiment. It appears that there is a confounding variable with the touchscreen such that they are extremely slow to learn a preference as indicated by passing our criterion of 17 correct trials out of the most recent 20. One confound could be that they must discriminate between shapes rather than colors. Shapes are known to require a few more trials for a preference to develop, but nothing on the order of what we found here [e.g., @shaw2015wild: mean=40 trials color, mean=55 trials shape in New Zealand robins; @isden2013performance: mean=6 trials color, mean=10 trials shape in spotted bowerbirds]. Another confound could be that they find it somehow rewarding simply to touch the screen and have something happen, regardless of whether they receive a food reward (e.g., they touch the screen on one of the stimuli and the screen goes blank because the trial ends). It is unclear what this confounding variable could be and we will not investigate it further given that these individuals need to complete a large test battery. The time investment to complete the touchscreen reversal experiment is significant: of the birds in the manipulation group which undergo 5 reversals on the touchscreen, Mole took 4 months and Habanero is on a similar track (he is not done yet). The birds in the control group (Queso and Tapa) only undergo only 1 reversal on the touchscreen, so it overall requires less time for these individuals. We will not include the data from this experiment when conducting the cross-test comparisons in the Analysis Plan section of this preregistration. Instead, in the results section of the resulting article, we will provide summary results for this experiment and qualitatively compare it with performance on the color tube reversal test to explain why we removed this experiment.
2) 16 April 2019: Because we are discontinuing the touchscreen reversal learning experiment, we will add an additional but distinct multi-access box task, which will allow us to continue to measure flexibility across three different experiments. There are two main differences between the current multi-access box, which is made of plastic, and the additional multi-access box added, which is made of wood. First, the wooden multi-access box is a natural log in which we carved out 4 compartments. As a result, the apparatus and solving options are more comparable to what grackles experience in the wild, though each compartment is covered by clear plastic doors that require different behaviors to open. Furthermore, there is only one food item available in the plastic multi-access box and the bird could use any of 4 loci to reach it. In contrast, the wooden multi-access box has a piece of food in each of the 4 separate compartments. With this difference, we can determine whether grackles are better able to inhibit manipulating a non-functional loci when food is no longer present.
**Post data collection, pre-data analysis**
3) We completed our simulation to explore the lower boundary of a minimum sample size and determined that **our sample size for the Arizona study site is above the minimum** (see details and code in [Ability to detect actual effects](http://corinalogan.com/Preregistrations/g_flexmanip.html#ability_to_detect_actual_effects) (17 April 2020).
4) Please see our [Alternative Analyses](#alternative-analyses) section where we describe how we **changed the analysis for P2** and that we are replacing this analysis with the new models in the [Ability to detect actual effects](#ability-to-detect-actual-effects) section (14 May 2020). We also describe here that we realized that Condition (manipulated or control) does not need to be a variable in our models because the manipulated birds have, by definition, faster reversal speeds.
5) We originally planned on testing only **adults** to have a better understanding of what the species is capable of, assuming the abilities we are testing are at their optimal levels in adulthood, and so that we could increase our statistical power by eliminating the need to include age as an independent variable in the models. Because the grackles in Arizona were extremely difficult to catch, we ended up testing two juveniles: Taco and Chilaquile. We did not conduct the full test battery with Taco or put him in the flexibility manipulation or control groups (he received 1 reversal and then moved on to the next test) because he was the first juvenile and we wanted to see whether his performance was different from adult performances. His performances were similar to the adults, therefore we decided to put Chilaquile in the full test battery. Chilaquile's performances were also similar to the adults, therefore we decided not to add age as an independent variable in the models to avoid reducing our statistical power.
**Post data collection, mid-data analysis**
6) We **log transformed** the response variable and changed the GLMM distribution from Poisson to Gaussian in the [P3a analysis](#p3a-repeatable-within-individuals-within-a-context-reversal-learning) (24 Aug 2021).
## RESULTS
Data are publicly [available]() at the Knowledge Network for Biocomplexity [@logan2021flexmanipdata].
Although 22 grackles completed their color tube initial discrimination, only 20 grackles participated in one or more reversals (Table 1). The rest of the tests began only after a bird's reversal experiment was complete (see the order of tests for each bird in Table 2). Interobserver reliability analyses (unregistered) showed that the reversal learning and multiaccess box (plastic and wooden) experiments were highly repeatable across live coders and video coders (see details in Analysis Plan > Interobserver reliability).
**Table 1.** Summarized results per bird in the reversal learning (tube and touchscreen) and multiaccess box (plastic and wooden) experiments. Reversals to pass indicates how many serial reversals it took a bird to pass criterion if they were in the flexibility manipulation condition.
```{r summary, eval=TRUE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d <- data.frame(d)
colnames(d) <- c("Bird","Batch","Sex","Trials to learn (tube)","Trials to first reversal (tube)","Trials to last reversal (tube)","Reversals to pass","Total loci solved MAB plastic","Total loci solved MAB wooden","Average latency to attempt new solution (MAB plastic)","Average latency to attempt new solution (MAB wooden)","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
library(kableExtra)
knitr::kable(d) %>%
kable_styling(full_width = T, position = "left")
```
**Table 2.** An overview of all color marked grackles at the Arizona field site (2018-2021), whether the bird was brought into the aviaries (number 1-8) or not ("-"), the order of the aviary experiments that individual began (though they did not necessarily complete that experiment) is denoted by numbers 1 through 8, whether the experiment was offered to the bird (- = not offered to that bird even though they were in the aviaries because they did not participate enough to pass habituation/training), and whether data for the variables that were collected in the wild are present (Y) or not (blank). Data and results from the various columns belong with different articles: reversal learning and MAB are reported here; detour and go no-go are reported in @logan2021inhibition; causal is reported in @blaisdell2021causal; demonstrator training is reported in @mccune2019soclearn; exploration and boldness in the aviaries and in the wild are reported in @mccune2019exploration; focal follows are reported here and in @mccune2020spaceuse; radio telemetry is reported in @mccune2020spaceuse, @berens2019condition, and @logan2019flexforaging; GPX tracks are reported in @mccune2020spaceuse; and nest and territory checks are reported in @berens2019condition.
```{r order, eval=TRUE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_AllGrackleExpOrder.csv"), header=F, sep=",", stringsAsFactors=F)
d <- data.frame(d)
colnames(d) <- c("Bird","ID","Sex","Age","Reversal learning (tube)","MAB plastic","MAB log","Detour","Reversal learning (touchscreen)","Go no-go (touchscreen)","Causal (touchscreen)","Demonstrator training","Exploration and boldness (aviaries)","Exploration and boldness (wild)","Focal follows","Radio telemetry","GPX tracks","Had nest (females) or had nest on territory (males)")
library(kableExtra)
knitr::kable(d) %>%
kable_styling(full_width = T, position = "left")
```
Because the wooden multiaccess box was added after in principle acceptance, we conducted an unregistered analysis to determine whether the plastic and wooden multiaccess box results correlated with each other, which would indicate that these tests are interchangeable. We found that they did not correlate with each other on either variable measured: the average latency to attempt a new locus (switching; Pearson's r=0.74, 95% CI=-0.19-0.97, t=2.18, df=4, p=0.09) or the total number of loci solved (problem solving; Pearson's r=0.51, 95% CI=-0.09-0.84, t=1.86, df=10, p=0.09). Therefore, these two tests are not interchangeable and we analyzed them separately.
### P1: reversal speed gets faster with serial reversals
There was a significant negative correlation between the number of trials to reverse (average=71 trials, sd=28) and the reversal number for those grackles in the flexibility manipulation condition (n=9, which included Memela who did not pass the manipulation condition; Figure B). When this model was compared with the null model, the null model had a higher Akaike weight, however it was not high enough to indicate the two models were reliably different from each other (Table B). Therefore, we conclude that there was no effect of a linear relationship between the number of trials to reverse and reversal number.
**Unregistered analysis:** While there may not be an effect when one examines all reversals, there is a difference between manipulated and control reversal speeds when comparing their last reversals (for the control birds, their last reversal is their first reversal): the Akaike weight of the full model was 0.94, which means that including condition in the model explains the bulk of the variation in the number of trials to reverse in the last reversal (Table C). This analysis includes 19 grackles (8 manipulated condition - only those who actually passed the manipulation, 11 control condition) who had an overall average of 62 trials in their last reversal (sd=32).
**Table B.** Does the number of trials to reverse decrease with increasing reversal number? Not more than would be expected by chance: the Akaike weight of null model was higher than that of the the full model, though not >0.89, which indicates that neither model is more reliable than the other.
```{r p1table, eval=F, warning=F, results='asis', echo=T, include=T}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_reverse.csv"), header=T, sep=",", stringsAsFactors=F)
d <- d[!d$ID=="Fajita" & !d$ID=="Empanada",] #remove Fajita because she was a pilot bird
#remove NAs from the variables that will be in the model
d <- subset(d,!(is.na(d["TrialsToReverse"])))
d <- subset(d,!(is.na(d["ReverseNumber"])))
#include only those birds in the reversal tubes experiment and only those in the manipulation condition bc only these will have more than one reversal (and thus something to correlate)
d <- d[d$TubesOrTouchscreen=="TUBES" & d$ExperimentalGroup=="Manipulation",]
#factor variables
d$Batch <- as.factor(d$Batch)
d$ID <- as.factor(d$ID)
# GLMM
library(MCMCglmm)
prior = list(R = list(R1 = list(V = 1, nu = 0)), G = list(G1 = list(V = 1,
nu = 0), G2 = list(V = 1, nu = 0)))
serial <- MCMCglmm(TrialsToReverse ~ ReverseNumber, random = ~ID+Batch,
family = "poisson", data = d, verbose = F, prior = prior,
nitt = 300000, thin = 500, burnin = 90000)
#reverse number significantly negatively correlates with trials to reverse, as expected due to the manipulation
#summary(serial)
#Did fixed effects converge (<0.1)? Yes
#autocorr(serial$Sol)
#Did random effects converge (<0.1)? Yes except for 2 values: 0.11 and 0.12
#autocorr(serial$VCV)
#AIC calculation
library(MuMIn)
options(na.action = "na.fail")
base <- dredge(MCMCglmm(TrialsToReverse ~ ReverseNumber, random = ~ID+Batch,
family = "poisson", data = d, verbose = F, prior = prior,
nitt = 300000, thin = 500, burnin = 90000))
library(kableExtra)
knitr::kable(base)
#UNREGISTERED ANALYSIS: compare control vs manipulated group reversal speeds using only last reversals
#prior = list(R = list(R1 = list(V = 1, nu = 0)), G = list(G1 = list(V = 1,
# nu = 0), G2 = list(V = 1, nu = 0)))
#serial <- MCMCglmm(TrialsToReverse ~ ReverseNumber, random = ~ID+Batch,
# family = "poisson", data = d, verbose = F, prior = prior,
# nitt = 300000, thin = 500, burnin = 90000)
#reverse number significantly negatively correlates with trials to reverse, as expected due to the manipulation
#summary(serial)
#Did fixed effects converge (<0.1)? Yes
#autocorr(serial$Sol)
#Did random effects converge (<0.1)? Yes except for 2 values: 0.11 and 0.12
#autocorr(serial$VCV)
```
```{r p1fig, eval=T, warning=F, results='asis', echo=T, include=T}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_reverse.csv"), header=T, sep=",", stringsAsFactors=F)
d <- d[!d$ID=="Fajita" & !d$ID=="Empanada",] #remove Fajita because she was a pilot bird
#remove NAs from the variables that will be in the model
d <- subset(d,!(is.na(d["TrialsToReverse"])))
d <- subset(d,!(is.na(d["ReverseNumber"])))
#include only those birds in the reversal tubes experiment and only those in the manipulation condition bc only these will have more than one reversal (and thus something to correlate)
d <- d[d$TubesOrTouchscreen=="TUBES" & d$ExperimentalGroup=="Manipulation",]
#n, mean, sd
#length(levels(d$ID)) #9
#mean(d$TrialsToReverse) #71
#sd(d$TrialsToReverse) #28
#figure
plot(d$TrialsToReverse ~ d$ReverseNumber)
```
**Figure C.** Individuals in the manipulated condition (who received serial reversals) did not decrease their reversal passing speeds with increasing reversal number (n=9 grackles in the manipulated condition).
**Table C.** Do individuals in the manipulated condition pass their last reversal in fewer trials than control individuals? Yes: the Akaike weight of full model was >0.89, indicating that it is more reliable than the null model.
```{r p1last, eval=T, warning=F, results='asis', echo=T, include=T}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d <- data.frame(d)
colnames(d) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden)","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Remove NAs
d <- subset(d,!(is.na(d["TrialsLastReversal"])))
# exclude the bird who didn't pass serial
d <- d[!d$Bird=="Memela",]
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
d$ReversalsToPass <- as.factor(d$ReversalsToPass)
levels(d$ReversalsToPass)[c(1,2,3,4)] <- c("Control","Manipulated","Manipulated","Manipulated")
#UNREGISTERED ANALYSIS: compare control vs manipulated group reversal speeds using only last reversals
last <- glm(d$TrialsLastReversal~d$ReversalsToPass)
#manipulated group has significantly fewer trials to reverse in last reversal, as expected due to the manipulation
#summary(last)
#AIC calculation
library(MuMIn)
options(na.action = "na.fail")
aw <- dredge(glm(d$TrialsLastReversal~d$ReversalsToPass))
library(kableExtra)
knitr::kable(aw)
#the full model has an Akaike weight >0.9 so it is reliable. This means that condition explains differences in the number of trials to pass the last reversal, with the manipulated group being faster than the control group
```
```{r p1figlast, eval=T, warning=F, results='asis', echo=T, include=T}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d <- data.frame(d)
colnames(d) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden)","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Remove NAs
d <- subset(d,!(is.na(d["TrialsLastReversal"])))
# exclude the bird who didn't pass serial
d <- d[!d$Bird=="Memela",]
#n, mean, and sd
#length(d$TrialsLastReversal) #19
#mean(d$TrialsLastReversal) #62
#sd(d$TrialsLastReversal) #32
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
d$ReversalsToPass <- as.factor(d$ReversalsToPass)
levels(d$ReversalsToPass)[c(1,2,3,4)] <- c("Control","Manipulated","Manipulated","Manipulated")
#figure
plot(d$TrialsLastReversal ~ d$ReversalsToPass)
```
**Figure D.** Individuals in the manipulated condition (who received serial reversals) passed their last reversal in fewer trials than individuals in the control condition (who only received 1 reversal). n=19 grackles: 8 in the manipulated condition, 11 in the control condition.
### P2: serial reversals improve rule switching & problem solving on the MAB
#### Problem solving: number of loci solved on the multiaccess box (plastic) ~ trials to reverse
Grackles that were faster to reverse a preference (in their last reversal: average 62 trials, sd=34) solved more loci on the plastic multiaccess box (average=2 loci, sd=1.6; Figure E; Table C: Model 2; n=15 grackles: 6 in manipulated condition, 9 in control condition; this number excludes Mole and Habanero who were, due to experimenter error, given the fully put together box during habituation and could have learned how to solve the loci at that time). The original model (Table C: Model 1) included aviary batch, however this ended up confounding the analysis because control and manipulated individuals, while randomly assigned to these conditions, ended up in particular batches as a result of their willingness to participate (Table C: Model 3). As such, grackles who did not participate in their condition in a given batch were replaced in the aviary batch. Therefore, we removed batch from the model. We additionally examined whether first reversal (rather than last reversal) or condition (control or manipulated) better explained the relationship between reversal performance and number of loci solved: they did not. There was no correlation between the number of loci solved and which reversal condition a grackle was randomly assigned to, which indicates that the conditions did not differ from each other (Table C: Model 4). There was also no correlation between the number of trials to reverse in the first reversal (average=75 trials, sd=31) and the number of loci solved on the multiaccess box (Table C: Model 5; unregistered analysis). A correlation was determined to be present if the prediction interval for the slope (b) in the model output did not cross zero (Table C).
```{r p2mabplastic, eval=F, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
library(rethinking)
library(rstan)
library(formatR)
# LOAD the data and column names
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d <- data.frame(d)
colnames(d) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden)","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Exclude Mole and Habanero from this analysis because they were given the put together plastic box during habituation (due to experimenter error)
d <- d[!d$Bird=="Mole" & !d$Bird=="Habanero",]
# Remove NAs
d <- subset(d,!(is.na(d["TotalLociSolvedMABplastic"])) & !(is.na(d["TrialsLastReversal"])))
# n=15: 6 in manipulated group, 9 in control group
#length(d$TotalLociSolvedMABplastic)
# make Batch a factor (assigned Taco to batch 3 because 3a doesn't work with the model)
d$Batch <- as.factor(d$Batch)
levels(d$Batch)[c(3)] <- c("3")
# look at the data
#hist(d$TotalLociSolvedMABplastic)
#mean(d$TotalLociSolvedMABplastic) #2
#sd(d$TotalLociSolvedMABplastic) #1.6
#hist(d$TrialsLastReversal)
#mean(d$TrialsLastReversal) #61.5
#sd(d$TrialsLastReversal) #34.2
#translating the actual data (rather than the simulated data) into effect sizes (see equation below in "translated the simulation output into effect sizes"): SDx / SDy = 1.6/34.2 actual data (note: simulated data were 1.5/21=0.7)
#sd(d$TotalLociSolvedMABplastic)/sd(d$TrialsLastReversal) #=0.5
#cor.test(d$TotalLociSolvedMABplastic,d$TrialsLastReversal,alternative=c("two.sided"),method = c("pearson"),conf.level = 0.95)
#corr = r =-0.24
#solve equation for beta:
#-0.24/(sd(d$TotalLociSolvedMABplastic)/sd(d$TrialsLastReversal)) #-4.98 = beta
# RUN MODELs on the actual data
dat <- list(locisolved = d$TotalLociSolvedMABplastic,
trials = standardize(d$TrialsLastReversal),
batch = d$Batch
)
# MODEL 1: includes batch
m1 <- ulam( alist(
locisolved ~ dbinom(4,p) , #4 loci, p=probability of solving a locus
logit(p) <- a[batch] + b*trials , #batch=random effect, standardize trials so 0=mean
a[batch] ~ dnorm(0,1) , #each batch gets its own intercept
b ~ dnorm(0,0.4) #our prior expectation for b is that it is around 0, can be negative or positive, and should not be larger than 1. normal distribution works for binomial (Rethinking p.341)
) , data=dat , chains=4 , log_lik=TRUE )
#precis(m1,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a[1] 0.04 0.46 -0.70 0.78 2304 1
#a[2] 0.29 0.36 -0.30 0.87 2456 1
#a[3] -0.78 0.55 -1.65 0.08 2510 1
#b -0.22 0.25 -0.63 0.18 2364 1
#mean loci solved varies by batch, b=slope for correlation between number of loci solved and number of trials to pass last reversal - there is a negative relationship, but (in all cases) the confidence interval crosses 0 so this indicates no correlation because we can't say with certainty that it is different from zero. n_eff (number of independent samples obtained) is high, Rhat (indicator of model convergence) is good (p.281 Rethinking)
#Result: total number of loci solved is not associated with the number of trials to pass criterion on the last reversal
#model details: 2000 samples from 4 chains
#show(m1)
#no correlations between variables across batches
#pairs(m1)
#check the chain - fuzzy caterpillars = looks good
#traceplot(m1)
#check the chain a different way - "histograms overlap and stay within the same range" ≠ looks good (p.285 Rethinking)
#trankplot(m1)
# plot the results on the outcome scale (p.330 Rethinking). V1-3 = batch 1-3, value = probability of solving a locus
#postm1 <- extract.samples(m1)
#p_batch <- inv_logit( postm1$a )
#plot( precis( as.data.frame(p_batch) ) , xlim=c(0,1) )
# contrasts between batches: the log odds differences in solving a locus between batches. Value = log odds of solving a locus (p.331 & 341 Rethinking)
#diffsm1 <- list(
# b13 = postm1$a[,1] - postm1$a[,2],
# b14 = postm1$a[,1] - postm1$a[,3],
# b34 = postm1$a[,2] - postm1$a[,3] )
#labsdif <- c("Batch 1-3","Batch 1-4","Batch 3-4")
#plot( precis(diffsm1), xlim=c(-3,3), labels=labsdif)
# contrasts between batches on the outcome scale (p.341 Rethinking). Value = difference in number of loci solved. This shows that the batches are actually not different from each other in how many loci they solved because all values cross zero.
diffsm1c <- list(
diff_b13 <- inv_logit( postm1$a[,1]) - inv_logit( postm1$a[,2]),
diff_b14 <- inv_logit( postm1$a[,1]) - inv_logit( postm1$a[,3]),
diff_b34 <- inv_logit( postm1$a[,2]) - inv_logit( postm1$a[32]) )
#precis( list( diff_b13 , diff_b14 , diff_b34 ) )
#check posterior for p to look at the distribution of probabilities that are probable
#p <- inv_logit(postm1$a) #convert from logit to actual probability
#dens(p,adj=0.1)
#HPDI(p) #most mass is below 0.5
#median(p) #0.49
#result: the prior was a normal curve that peaked at 0.5. The posterior: the mean y axis point where the intercept is is 0.49 (meaning they solve on average 50% of the loci = 2), which means this is when trials to reverse is at the average
# MODEL 2: check to see if including batch has an influence on the estimate of b by removing batch
mwobatch <- ulam( alist(
locisolved ~ dbinom(4,p) ,
logit(p) <- a + b*trials, #standardize trials so 0=mean
a ~ dnorm(0,0.5) ,
b ~ dnorm(0,2)
) , data=dat , chains=4 , log_lik=TRUE )
precis(mwobatch,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a -0.02 0.24 -0.40 0.35 1466 1
#b -0.46 0.31 -0.97 -0.01 1383 1
#the confidence interval does NOT cross 0, which indicates batch differ in their composition of control and manipulated indviduals. n_eff is high, but lower than for m1, Rhat is good. Report both: there is a potential confound of batch, which is differently composed (m1).
#check posterior for p to look at the distribution of probabilities that are probable
postmwobatch <- extract.samples(mwobatch)
#p2 <- inv_logit(postmwobatch$a) #convert from logit to actual probability
#dens(p2,adj=0.1)
#HPDI(p2)
#median(p2) #0.50, narrower than prior & m1
#result: The posterior: the mean y axis point where the intercept is is 0.50 (meaning they solve on average 50% of the loci = 2), which means this is when trials to reverse is at the average
# MODEL 3: see whether the confound of batch in m1 is an issue of the composition of control and manipulated individuals in a particular batch that is causing the batch differences in the MAB models
#trialsbatch <- ulam( alist(
# trials ~ normal(mu,sigma),
# mu <- a[batch],
# a[batch] ~ dnorm(0,0.5) ,
# sigma ~ dexp(1)
#) , data=dat , chains=4 , log_lik=TRUE )
#precis(trialsbatch,depth=2)
#batches differ, which suggests that this is a confound in m1: can't accurately estimate the relationship between loci solved and trials to reverse because the batches already contain the difference. So need to EXCLUDE batch from the MAB model and use mwobatch
# MODEL 4: see whether the flexibility manipulation actually had an effect on MAB performance by replacing batch with condition (control, manipulated) and REMOVING trials
#mean(d$TrialsFirstReversal) #74.7
#sd(d$TrialsFirstReversal) #30.7
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
#d$ReversalsToPass <- as.factor(d$ReversalsToPass)
#levels(d$ReversalsToPass)[c(1,2,3)] <- c("Control","Manipulated","Manipulated")
#dat2 <- list(locisolved = d$TotalLociSolvedMABplastic,
# trials = standardize(d$TrialsFirstReversal),
# condition = d$ReversalsToPass
# )
mcondition <- ulam( alist(
locisolved ~ dbinom(4,p) , #4 loci, p=probability of solving a locus
logit(p) <- a[condition] , #condition=random effect, standardize trials so 0=mean
a[condition] ~ dnorm(0,1) #each condition gets its own intercept
) , data=dat2 , chains=4 , log_lik=TRUE )
#precis(mcondition,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a[1] -0.11 0.32 -0.62 0.4 1311 1
#a[2] 0.15 0.39 -0.46 0.8 1222 1
# contrasts between conditions: the log odds differences in solving a locus between batches. Value = log odds of solving a locus (p.331 & 341 Rethinking)
postmcondition <- extract.samples(mcondition)
diffsmcondition <- postmcondition$a[,1] - postmcondition$a[,2]
#labsdifc <- c("Control-Manipulated")
#plot( precis(diffsmcondition), xlim=c(-1.5,1), labels=labsdifc)
# contrasts between conditions on the outcome scale (p.341 Rethinking)
#precis( diffsmcondition )
#Both of thse results show that the condiitons are actually not different from each other in how many loci they solved because all values cross zero. This means that mwobatch shows that it is some combination of the flexibility manipulation (training those individuals who were not already fast) and other previous experience (not making much of a difference in reversal speeds for those who were already fast) that led to differences on the MAB. This suggests the important variable is the ability to be flexible, which birds could have from the beginning or could be manipulated in the experiment. It is the effect of the experiment on this ability, not the something else about the experiment (e.g., differences in motivation, exploration, etc.)
# MODEL 5: see whether the flexibility manipulation actually had an effect on MAB performance by replacing trials to LAST reversal with trials to FIRST reversal
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
#dat3 <- list(locisolved = d$TotalLociSolvedMABplastic,
# trials = standardize(d$TrialsFirstReversal),
# batch = d$Batch
# )
#mwobatch2 <- ulam( alist(
# locisolved ~ dbinom(4,p) ,
# logit(p) <- a + b*trials , #standardize trials so 0=mean
# a ~ dnorm(0,0.5) ,
# b ~ dnorm(0,2)
#) , data=dat , chains=4 , log_lik=TRUE )
#precis(mwobatch2,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 0.00 0.24 -0.37 0.39 1208 1
#b -0.44 0.30 -0.94 0.02 1273 1
#b crosses zero so there is no correlation between number of loci solved and number of trials to reverse on the first reversal. This means that reversal in general doesn't correlate with MAB loci. You actually have to DO something to flexibility to make it matter for MAB loci
# VISUALIZE: plot trials to pass last reversal against number of loci solved on the mwobatch model (p249 Rethinking panel on the left)
# figure out xlim: -0.96 - 2.65
#range(dat$trials)
# dat$locisolved is on the outcome scale, but the model output is on the logit scale so transform dat$locisolved to a probability by dividing by 4 total loci
dat$locisolvedp <- (dat$locisolved/4)
# draw 50 lines from the prior
trials_seq <- seq( from=-1.13 , to=2.9 , length.out=30 )
op <- par(mfrow=c(1,1), mar=c(5.9,4.9,2,0.9))
plot(dat$locisolvedp ~ dat$trials , pch=16 , col="black" ,
xlab="Trials to pass last reversal (standardized: mean=0)" , ylab="Probability of solving a locus on the multiaccess box (plastic)" , xlim=c(-1.2,3.1), cex.lab=2, cex.axis=2, cex=2 )
mu <- link( mwobatch , data=data.frame( trials=trials_seq ) )
mu_mean <- apply( mu , 2 , mean )
mu_ci <- apply( mu , 2 , PI , prob=0.97 )
lines( trials_seq , mu_mean , lwd=2 )
shade( mu_ci , trials_seq , col=col.alpha(rangi2,0.3) )
par(op)
# contrasts between batches on the outcome scale (p.341 Rethinking). Value = difference in number of loci solved. This shows that the batches are actually not different from each other in how many loci they solved because all values cross zero
labsdif1 <- c("m1: Batch 1-3","m1: Batch 1-4","m1: Batch 3-4")
labsdif2 <- c("m4: Control-Manipulated")
op <- par(mfrow=c(2,1), mar=c(4,4,2,0.2))
plot( precis(diffsm1c, diffsmcondition), xlim=c(-0.5,0.5), xlab="Difference in number of loci solved", labels=labsdif1)
plot( precis(diffsmcondition), xlim=c(-1.5,0.5), xlab="Difference in number of loci solved", labels=labsdif2)
par(op)
```
![](g_flexmanip_figp2mabplastic.png){width=100%}
**Figure E.** The probability of solving a locus on the plastic multiaccess box is negatively correlated with the number of trials to pass their last reversal (n = 15 grackles, shading = 97% prediction interval).
**Table C.** Model outputs for the number of loci solved on the plastic (models 1-5) and wooden (models 6-8) multiaccess boxes. SD=standard deviation, the 89% prediction intervals are shown, n_eff=effective sample size, Rhat4=an indicator of model convergence (1 is ideal), b=the slope of the relationship between loci solved and number of trials to pass reversal.
```{r p2outputs, eval=TRUE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
table <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_table_modeloutputs.csv"), header=T, sep=",", stringsAsFactors=F)
table <- data.frame(table)
colnames(table) <- c("","Mean","SD","5.5%","94.5%","n_eff","Rhat4")
library(kableExtra)
knitr::kable(table) %>%
kable_styling(full_width = T, position = "left")
```
#### Problem solving: number of loci solved on the multiaccess box (wooden) ~ trials to reverse
The estimate for the association between the number of loci solved on the wooden multiaccess box (average=3.2, sd=1.3) and the number of trials to reverse a preference (in their last reversal: average=59 trials, sd=38) crossed zero (Figure F; Model 6, Table C; n=12 grackles: 6 in manipulated condition, 6 in control condition). This could mean that there is no association, however our simulations showed that if the effect is small, we would expect that with our sample size, we are not able to reliably estimate that such a small effect is different from zero (correlation test suggests effect size of 0.2; Table 2p). Aviary batch was excluded because of what we learned in the previous section. We did find a correlation between the number of loci solved and which reversal condition a grackle was randomly assigned to, indicating the reversal manipulation appears to have affected the birds. The model estimates that manipulated birds solved on average 1.2 loci more than birds in the control condition (Table C: Model 7, wooden; 89% prediction intervals 0.34-2.14; n=12 grackles: 6 in manipulated condition, 6 in control condition). However, there is no association between the number of trials to reverse in the first reversal (average=74 trials, sd=34) and the number of loci solved on the multiaccess box (Table C: Model 8, wooden; unregistered analysis). A correlation was determined to be present if the prediction interval for the slope (b) in the model output did not cross zero (Table C).
```{r p2mablog, eval=F, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
library(rethinking)
library(rstan)
library(formatR)
# LOAD the data and column names
d2 <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d2 <- data.frame(d2)
colnames(d2) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden)","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Remove NAs
d2 <- subset(d2,!(is.na(d2["TotalLociSolvedMABwooden"])) & !(is.na(d2["TrialsLastReversal"])))
# n=12: 6 in manipulated group, 6 in control group
#length(d2$TotalLociSolvedMABwooden)
# make Batch numeric (assigned Taco to batch 3 because 3a doesn't work with the model)
d2$Batch <- as.factor(d2$Batch)
levels(d2$Batch)[c(2)] <- c("3")
# look at the data
#hist(d2$TotalLociSolvedMABwooden)
#mean(d2$TotalLociSolvedMABwooden) #3.2
#sd(d2$TotalLociSolvedMABwooden) #1.3
#mean(d2$TrialsLastReversal) #59.4
#sd(d2$TrialsLastReversal) #38.0
#translating the actual data (rather than the simulated data) into effect sizes (see equation below in "translated the simulation output into effect sizes"): SDx / SDy = 1.3/38 actual data (note: simulated data were 1.5/21=0.7)
#sd(d2$TotalLociSolvedMABwooden)/sd(d2$TrialsLastReversal) #=0.03
#cor.test(d2$TotalLociSolvedMABwooden,d2$TrialsLastReversal,alternative=c("two.sided"),method = c("pearson"),conf.level = 0.95)
#corr = r = 0.20 = this is the effect size!!!
#solve equation for beta:
#0.20/(sd(d2$TotalLociSolvedMABwooden)/sd(d2$TrialsLastReversal)) #6.08 = beta, which is larger than the estimated -5
# RUN MODELs on the actual data
datw <- list(locisolved = d2$TotalLociSolvedMABwooden,
trials = standardize(d2$TrialsLastReversal),
batch = d2$Batch
)
# MODEL 6: same as model 2 in previous section
m6 <- ulam( alist(
locisolved ~ dbinom(4,p) ,
logit(p) <- a + b*trials, #standardize trials so 0=mean
a ~ dnorm(0,0.5) ,
b ~ dnorm(0,2)
) , data=datw , chains=4 , log_lik=TRUE )
precis(m6,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 1.04 0.28 0.61 1.48 1423 1
#b 0.39 0.41 -0.24 1.09 1517 1
#the confidence interval for b (the slope) crosses 0, which indicates that there is no correlation between number of loci solved and trials to reverse
#In case you rerun the model and want to add the new estimates for a and b to the table (Table C referenced above), use this code
# This will go to the respective columns 2-7 that have the output information, in lines 21 and 22 which have the information for the a and for the b from this model, and fill it in with the precis output. The first line of the precis output is the information for a so it goes in line 21, the second line in the precis output is for b, so it goes into line 22. We also want to round the output values to what is in the table, 2 decimal places
#table[21,2:7]<-round(precis(m6,depth=2)[1,],2)
#table[22,2:7]<-round(precis(m6,depth=2)[2,],2)
# For the number of effective samples, we actually want no decimal places, so we can adjust that at the end
#table$n_eff<-round(table$n_eff,0)
# You can save the changed table as
# write.csv(table,file="g_flexmanip_table_modeloutputs.csv")
#check posterior for p to look at the distribution of probabilities that are probable
postm6 <- extract.samples(m6)
#p2 <- inv_logit(postm6$a) #convert from logit to actual probability
#dens(p2,adj=0.1)
#HPDI(p2)
#median(p2) #0.72, narrower than prior & m1, shifted to the right
#result: The posterior: the mean y axis point where the intercept is is 0.72 (meaning they solve on average 72% of the loci = 3), which means this is when trials to reverse is at the average
#model details: 2000 samples from 4 chains
#show(m1)
#no correlations between variables across batches
#pairs(m6)
#check the chain - fuzzy caterpillars = looks good
#traceplot(m6)
#check the chain a different way - "histograms overlap and stay within the same range" ≠ looks good (p.285 Rethinking)
#trankplot(m6)
# MODEL 7: see whether the flexibility manipulation actually had an effect on MAB performance by replacing batch with condition (control, manipulated) and REMOVING trials
#mean(d2$TrialsFirstReversal) #73.6
#sd(d2$TrialsFirstReversal) #34.1
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
d2$ReversalsToPass <- as.factor(d2$ReversalsToPass)
levels(d2$ReversalsToPass)[c(1,2,3)] <- c("Control","Manipulated","Manipulated")
dat2 <- list(locisolved = d2$TotalLociSolvedMABplastic,
trials = standardize(d2$TrialsFirstReversal),
condition = d2$ReversalsToPass
)
m7 <- ulam( alist(
locisolved ~ dbinom(4,p) , #4 loci, p=probability of solving a locus
logit(p) <- a[condition] , #condition=random effect, standardize trials so 0=mean
a[condition] ~ dnorm(0,1) #each condition gets its own intercept
) , data=dat2 , chains=4 , log_lik=TRUE )
#precis(m7,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a[1] -0.45 0.40 -1.10 0.18 1161 1
#a[2] 0.77 0.41 0.13 1.44 1302 1
#Condition does not correlate with performance on the MAB bc both CIs cross zero
# contrasts between conditions: the log odds differences in solving a locus between batches. Value = log odds of solving a locus (p.331 & 341 Rethinking)
postm7 <- extract.samples(m7)
diffsm7 <- postm7$a[,1] - postm7$a[,2]
labsdifc <- c("Control-Manipulated")
plot( precis(diffsmcondition), xlim=c(-1.5,1), labels=labsdifc)
# contrasts between conditions on the outcome scale (p.341 Rethinking)
precis( diffsm7 )
#conditons are actually not different from each other in how many loci they solved because CIs cross zero. This means that m7 shows that it is some combination of the flexibility manipulation (training those individuals who were not already fast) and other previous experience (not making much of a difference in reversal speeds for those who were already fast) that led to differences on the MAB. This suggests the important variable is the ability to be flexible, which birds could have from the beginning or could be manipulated in the experiment. It is the effect of the experiment on this ability, not the something else about the experiment (e.g., differences in motivation, exploration, etc.)
# MODEL 8: see whether the flexibility manipulation actually had an effect on MAB performance by replacing trials to LAST reversal with trials to FIRST reversal
dat3 <- list(locisolved = d2$TotalLociSolvedMABplastic,
trials = standardize(d2$TrialsFirstReversal),
batch = d2$Batch
)
m8 <- ulam( alist(
locisolved ~ dbinom(4,p) ,
logit(p) <- a + b*trials , #standardize trials so 0=mean
a ~ dnorm(0,0.5) ,
b ~ dnorm(0,2)
) , data=dat3 , chains=4 , log_lik=TRUE )
precis(m8,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 0.11 0.26 -0.30 0.52 1221 1
#b -0.50 0.35 -1.09 0.04 1234 1
#b crosses zero so there is no correlation between number of loci solved and number of trials to reverse on the first reversal. This means that reversal in general doesn't correlate with MAB loci. You actually have to DO something to flexibility to make it matter for MAB loci
# VISUALIZE: plot trials to pass last reversal against number of loci solved on the mwobatch model (p249 Rethinking panel on the left)
# figure out xlim: -0.96 - 2.65
#range(datw$trials)
# datw$locisolved is on the outcome scale, but the model output is on the logit scale so transform datw$locisolved to a probability by dividing by 4 total loci
datw$locisolvedp <- (datw$locisolved/4)
# draw 50 lines from the prior
trialsw_seq <- seq( from=-0.96 , to=2.7 , length.out=30 )
# the plot will automatically be turned into a png file so we need to set the working directory to where this rmd file is so that g_flexmanip_figp2mablog.png will also save there
library(rstudioapi)
setwd(dirname(rstudioapi::getActiveDocumentContext()$path))
# if this does not work, in RStudio click Session > Set working directory > To source file location
op <- par(mfrow=c(1,1), mar=c(5.9,4.9,2,0.9))
#png("g_flexmanip_figp2mablog.png")
plot(jitter(datw$locisolvedp) ~ jitter(datw$trials) , pch=1 , col="black" ,
xlab="Trials to pass last reversal (standardized: mean=0)" , ylab="Probability of solving a locus on the multiaccess box (wooden)" , xlim=c(-1.2,3.1), cex.lab=2, cex.axis=2, cex=2 )
mu <- link( m6 , data=data.frame( trials=trialsw_seq ) )
mu_mean <- apply( mu , 2 , mean )
mu_ci <- apply( mu , 2 , PI , prob=0.97 )
lines( trialsw_seq , mu_mean , lwd=2 )
shade( mu_ci , trialsw_seq , col=col.alpha(rangi2,0.3) )
#dev.off()
par(op)
#but the png doesn't look good so I will make the png by hand again
```
![](g_flexmanip_figp2mablog.png){width=100%}
**Figure F.** The probability of solving a locus on the wooden multiaccess box is not correlated with the number of trials to pass their last reversal (n = 12 grackles, shading = 97% prediction interval; estimate of slope includes zero).
#### Rule switching: latency to attempt a new locus on the multiaccess box (plastic) ~ trials to reverse
Grackles that were faster to reverse a preference (in their last reversal: average 52 trials, sd=23) were also faster to attempt to solve a new locus on the plastic multiaccess box (after just having passed criterion on a different locus; average=208 seconds, sd=226; Figure G; Table C: Model 9; n=11 grackles: 6 in manipulated condition, 5 in control condition; Pizza, Taquito, Chalupa, and Pollito completed this experiment, but they only solved 0 locus, therefore they did not have any locus switching times to analyze; Marisco and Taco also completed this experiment and solved 1 locus, but did not attempt another locus after that, thus they also do not have any switching times to analyze). We excluded aviary batch from all analyses because of our findings in the first section of Results P2. We also found that individuals in the flexibilty manipulation had faster switch latencies than those in the control condition (Table C: Model 10). There was a positive correlation between the number of trials to reverse in the first reversal (average=70 trials, sd=21) and the average switch latency on the multiaccess box (Table C: Model 11; unregistered analysis). A correlation was determined to be present if the prediction interval for the slope (b) in the model output did not cross zero (Table C).
```{r p2mabplasticlat, eval=F, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
library(rethinking)
library(rstan)
library(formatR)
# LOAD the data and column names
d3 <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d3 <- data.frame(d3)
colnames(d3) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Remove NAs
d3 <- subset(d3,!(is.na(d3["AverageLatencyAttemptNewLocusMABplastic"])) & !(is.na(d3["TrialsLastReversal"])))
# n=11: 5 in manipulated group, 6 in control group
#length(d3$AverageLatencyAttemptNewLocusMABplastic)
# make Batch a factor
d3$Batch <- as.factor(d3$Batch)
# look at the data
#hist(d3$AverageLatencyAttemptNewLocusMABplastic)
#mean(d3$AverageLatencyAttemptNewLocusMABplastic) #208
#sd(d3$AverageLatencyAttemptNewLocusMABplastic) #226
#mean(d3$TrialsLastReversal) #52
#sd(d3$TrialsLastReversal) #23
#mean(d3$TrialsFirstReversal) #70
#sd(d3$TrialsFirstReversal) #21
#translating the actual data (rather than the simulated data) into effect sizes (see equation below in "translated the simulation output into effect sizes")
#sd(d3$AverageLatencyAttemptNewLocusMABplastic)/sd(d3$TrialsLastReversal) #=9.9
#cor.test(d3$AverageLatencyAttemptNewLocusMABplastic,d3$TrialsLastReversal,alternative=c("two.sided"),method = c("pearson"),conf.level = 0.95)
#corr = r = 0.52
#solve equation for beta:
#0.52/(sd(d3$AverageLatencyAttemptNewLocusMABplastic)/sd(d3$TrialsLastReversal)) #0.05 = beta
# RUN MODELs on the actual data
library("Rcpp")
library("rstan")
library(rethinking)
library(ggplot2)
# MODEL 9: batch was excluded because of what was learned in the previous sections
dl <- list(trials = standardize(as.numeric(d3$TrialsLastReversal)),
latency = as.integer(d3$AverageLatencyAttemptNewLocusMABplastic),
batch = as.integer(d3$Batch)
)
mplat1 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a + b * trials,
a ~ dnorm(1, 1),
b ~ dnorm(0, 1),
phi ~ dexp(1)
), data = dl, chains=4, log_lik = TRUE, messages = FALSE)
precis(mplat1,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 4.93 0.30 4.45 5.41 1235 1.01
#b 0.46 0.29 0.00 0.92 1363 1.00
#phi 0.93 0.35 0.44 1.55 1476 1.00
#the confidence interval for b (the slope) touches 0 but does not cross it (which would be indicated by a sign change), which indicates that there is likely a positive correlation between MAB switch latency and trials to reverse
#check posterior for p to look at the distribution of probabilities that are probable
postmplat1 <- extract.samples(mplat1)
p3 <- exp(postmplat1$a) #convert from log to number of seconds
dens(p3,adj=0.1)
HPDI(p3) #76-209
median(p3) #139, narrower and shifted left than the curve from the simulations
#result: The posterior: the mean y axis point where the intercept is is 139 (meaning they switch on average at a latency of 139 seconds), which means this is when trials to reverse is at the average. The actual median is smaller than what we estimated the mean would be in the simulations (300s)
#model details: 2000 samples from 4 chains
#show(mplat1)
#no correlation
#pairs(mplat1)
#check the chain - fuzzy caterpillars = looks good
#traceplot(mplat1)
#check the chain a different way - "histograms overlap and stay within the same range" ≠ looks good (p.285 Rethinking)
#trankplot(mplat1)
# MODEL 10: see whether the flexibility manipulation actually had an effect on MAB performance by replacing batch with condition (control, manipulated) and REMOVING trials
#mean(d2$TrialsFirstReversal) #73.6
#sd(d2$TrialsFirstReversal) #34.1
# make ReversalsToPass a factor that has only 2 levels: level 1 = manipulated, level 2 = control
d3$ReversalsToPass <- as.factor(d3$ReversalsToPass)
levels(d3$ReversalsToPass)[c(1,2,3)] <- c("Manipulated","Manipulated","Manipulated")
dl2 <- list(latency = as.integer(d3$AverageLatencyAttemptNewLocusMABplastic),
condition = as.integer(d3$ReversalsToPass)
)
mplat2 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a[condition],
a[condition] ~ dnorm(1, 1),
phi ~ dexp(1)
), data = dl2, chains=4, log_lik = TRUE, messages = FALSE)
precis(mplat2,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a[1] 4.07 0.39 3.46 4.68 1027 1.00
#a[2] 5.18 0.39 4.50 5.76 1006 1.00
#phi 0.91 0.41 0.37 1.63 925 1.01
#Condition correlates with performance on the MAB bc neither a[] CI crosses zero, a1 manipulated has a lower mean than a2 control
# contrasts between conditions: the log odds differences in solving a locus between batches. Value = log odds of solving a locus (p.331 & 341 Rethinking)
postmplat2 <- extract.samples(mplat2)
diffmplat2 <- postmplat2$a[,1] - postmplat2$a[,2]
labsdif2 <- c("Manipulated-Control")
plot( precis(diffmplat2), xlim=c(-2,0.5), labels=labsdif2)
# contrasts between conditions on the outcome scale (p.341 Rethinking)
precis( diffmplat2 )
#conditons are different from each other in their average latency to switch because CI does not cross zero. This means that the manipulated individuals are faster to switch than control individuals. This suggests that the experience involved in the flexibility manipulation had a direct effect on mab performance
# MODEL 11: see whether the flexibility manipulation actually had an effect on MAB performance by replacing trials to LAST reversal with trials to FIRST reversal
dl3 <- list(locisolved = d3$AverageLatencyAttemptNewLocusMABplastic,
trials = standardize(d3$TrialsFirstReversal)
)
mplat3 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a + b * trials,
a ~ dnorm(1, 1),
b ~ dnorm(0, 1),
phi ~ dexp(1)
), data = dl, chains=4, log_lik = TRUE, messages = FALSE)
precis(mplat3,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 4.93 0.29 4.46 5.39 1488 1
#b 0.46 0.28 0.02 0.93 1211 1
#phi 0.94 0.36 0.44 1.60 1447 1
#b does not cross zero so there is a positive correlation between average switch latencies and number of trials to reverse on the first reversal. This means that reversal in general correlates with MAB loci and that the flexibility manipulation is not needed to enhance or make this relationship
# VISUALIZE: plot trials to pass last reversal against number of loci solved on the mwobatch model (p249 Rethinking panel on the left)
# figure out xlim: -1.28 - 2.10
#range(dl$trials)
# draw 50 lines from the prior
trialsp_seq <- seq( from=-1.29 , to=2.11 , length.out=30 )
op <- par(mfrow=c(1,1), mar=c(5.9,4.9,2,0.9))
plot(dl$latency ~ dl$trials , pch=16 , col="black" ,
xlab="Trials to pass last reversal (standardized: mean=0)" , ylab="Avg latency (s) to attempt new locus on multiaccess box (plastic)" , xlim=c(-1.2,2.05), cex.lab=2, cex.axis=2, cex=2 )
mu <- link( mplat1 , data=data.frame( trials=trialsp_seq ) )
mu_mean <- apply( mu , 2 , mean )
mu_ci <- apply( mu , 2 , PI , prob=0.97 )
lines( trialsp_seq , mu_mean , lwd=2 )
shade( mu_ci , trialsp_seq , col=col.alpha(rangi2,0.3) )
par(op)
```
![](g_flexmanip_figp2latmabplastic.png){width=100%}
**Figure G.** The average latency (seconds) to attempt to solve a different locus on the plastic multiaccess box after having previously successfully solved a locus is positively correlated with the number of trials to pass their last reversal (n = 11 grackles, shading = 97% prediction interval).
#### Rule switching: latency to attempt a new locus on the multiaccess box (wooden) ~ trials to reverse
There was no correlation between the number of trials to reverse a preference (in their last reversal: average 60 trials, sd=38) and the latency to attempt to solve a new locus on the wooden multiaccess box (after just having passed criterion on a different locus; average=463 seconds, sd=481; Figure H; Table C: Model 12; n=11 grackles: 5 in manipulated condition, 6 in control condition; Diablo also completed this experiment and solved 1 locus, but did not attempt another locus after that, thus he does not have any switching times to analyze). We excluded aviary batch from all analyses because of our findings in the first section of Results P2. We additionally found that there was no difference in average latency to switch between individuals in the flexibilty manipulation and those in the control condition (Table C: Model 13). There was a negative correlation between the number of trials to reverse in the first reversal (average=73 trials, sd=34) and the average switch latency on the multiaccess box (Table C: Model 14; unregistered analysis). A correlation was determined to be present if the prediction interval for the slope (b) in the model output did not cross zero (Table C).
```{r p2mabloglat, eval=F, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
library(rethinking)
library(rstan)
library(formatR)
library("Rcpp")
library("rstan")
library(ggplot2)
# LOAD the data and column names
d4 <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
d4 <- data.frame(d4)
colnames(d4) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","Notes")
# Remove NAs
d4 <- subset(d4,!(is.na(d4["AverageLatencyAttemptNewLocusMABwooden"])) & !(is.na(d4["TrialsLastReversal"])))
# n=11: 5 in manipulated group, 6 in control group
#length(d4$AverageLatencyAttemptNewLocusMABwooden)
# make Batch a factor (assigned Taco to batch 3 because 3a doesn't work with the model)
d4$Batch <- as.factor(d4$Batch)
levels(d4$Batch)[c(3)] <- c("3")
# look at the data
#hist(d4$AverageLatencyAttemptNewLocusMABwooden)
#mean(d4$AverageLatencyAttemptNewLocusMABwooden) #463
#sd(d4$AverageLatencyAttemptNewLocusMABwooden) #481
#mean(d4$TrialsLastReversal) #60
#sd(d4$TrialsLastReversal) #38
#mean(d4$TrialsFirstReversal) #73
#sd(d4$TrialsFirstReversal) #34
#translating the actual data (rather than the simulated data) into effect sizes (see equation below in "translated the simulation output into effect sizes")
#sd(d4$AverageLatencyAttemptNewLocusMABwooden)/sd(d4$TrialsLastReversal) #=12.8
#cor.test(d4$AverageLatencyAttemptNewLocusMABwooden,d4$TrialsLastReversal,alternative=c("two.sided"),method = c("pearson"),conf.level = 0.95)
#corr = r = -0.26
#solve equation for beta:
#-0.26/(sd(d4$AverageLatencyAttemptNewLocusMABwooden)/sd(d4$TrialsLastReversal)) #0.02 = beta
#looking at table 2 for beta=0 and using the "Range of MAB loci solved" as 0-4 because they were able to solve all 4, the regression coefficient is 0.43.
# RUN MODELs on the actual data
# MODEL 12: batch was excluded because of what was learned in the previous sections
dlw <- list(trials = standardize(as.numeric(d4$TrialsLastReversal)),
latency = as.integer(d4$AverageLatencyAttemptNewLocusMABwooden)
)
mwlat1 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a + b * trials,
a ~ dnorm(1, 1),
b ~ dnorm(0, 1),
phi ~ dexp(1)
), data = dlw, chains=4, log_lik = TRUE, messages = FALSE)
precis(mwlat1,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 5.75 0.28 5.28 6.18 1049 1.00
#b -0.41 0.32 -0.86 0.15 1281 1.01
#phi 1.04 0.42 0.48 1.77 1456 1.00
#the confidence interval for b (the slope) crosses zero, indicating that there is no correlation between MAB switch latency and trials to reverse
#check posterior for p to look at the distribution of probabilities that are probable
postmwplat1 <- extract.samples(mwlat1)
#p4 <- exp(postmwplat1$a) #convert from log to number of seconds
#dens(p4,adj=0.1)
#HPDI(p4) #193-469
#median(p4) #315, as expected from the simulations
#result: The posterior: the mean y axis point where the intercept is is 315 (meaning they switch on average at a latency of 315 seconds), which means this is when trials to reverse is at the average. The actual median is the same as what we estimated the mean would be in the simulations (300s)
#model details: 2000 samples from 4 chains
#show(mwlat1)
#no correlation
#pairs(mwlat1)
#check the chain - fuzzy caterpillars = looks good
#traceplot(mwlat1)
#check the chain a different way - "histograms overlap and stay within the same range" ≠ looks good (p.285 Rethinking)
#trankplot(mwlat1)
# MODEL 13: see whether the flexibility manipulation actually had an effect on MAB performance by replacing batch with condition (control, manipulated) and REMOVING trials
#mean(d2$TrialsFirstReversal) #73.6
#sd(d2$TrialsFirstReversal) #34.1
# make ReversalsToPass a factor that has only 2 levels: level 1 = control, level 2 = manipulated
d4$ReversalsToPass <- as.factor(d4$ReversalsToPass)
levels(d4$ReversalsToPass)[c(1,2,3,4)] <- c("Control","Manipulated","Manipulated","Manipulated")
dl3 <- list(latency = as.integer(d4$AverageLatencyAttemptNewLocusMABwooden),
condition = as.integer(d4$ReversalsToPass)
)
mwlat2 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a[condition],
a[condition] ~ dnorm(1, 1),
phi ~ dexp(1)
), data = dl3, chains=4, log_lik = TRUE, messages = FALSE)
precis(mwlat2,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a[1] 5.31 0.42 4.61 5.95 701 1.00
#a[2] 5.34 0.44 4.61 6.00 620 1.01
#phi 0.66 0.32 0.25 1.25 806 1.00
#Condition correlates with performance on the MAB bc neither a[] CI crosses zero, a1 control and a2 manipulated have similar means
# contrasts between conditions: the log odds differences in solving a locus between batches. Value = log odds of solving a locus (p.331 & 341 Rethinking)
postmwlat2 <- extract.samples(mwlat2)
diffmwlat2 <- postmwlat2$a[,1] - postmwlat2$a[,2]
labsdif3 <- c("Control-Manipulated")
plot( precis(diffmwlat2), xlim=c(-1,1), labels=labsdif3)
# contrasts between conditions on the outcome scale (p.341 Rethinking)
precis( diffmwlat2 )
#conditons are not different from each other in their average latency to switch because CI crosses zero. Same interpretation as for MAB plastic loci solved
# MODEL 14: see whether the flexibility manipulation actually had an effect on MAB performance by replacing trials to LAST reversal with trials to FIRST reversal
dl4 <- list(latency = d4$AverageLatencyAttemptNewLocusMABwooden,
trials = standardize(d4$TrialsFirstReversal)
)
mwlat3 <- ulam(alist(
latency ~ dgampois(lambda, phi),
log(lambda) <- a + b * trials,
a ~ dnorm(1, 1),
b ~ dnorm(0, 1),
phi ~ dexp(1)
), data = dl4, chains=4, log_lik = TRUE, messages = FALSE)
precis(mwlat3,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
#a 5.71 0.26 5.28 6.12 1109 1
#b -0.50 0.28 -0.89 -0.01 1308 1
#phi 1.08 0.41 0.53 1.80 1347 1
#b does not cross zero so there is a negative correlation between average switch latencies and number of trials to reverse on the first reversal. This means that reversal in general correlates with MAB loci and that the flexibility manipulation is not needed to enhance or make this relationship
# VISUALIZE: plot trials to pass last reversal against number of loci solved on the mwobatch model (p249 Rethinking panel on the left)
# figure out xlim: -1 - 2.65
#range(dlw$trials)
# draw 50 lines from the prior
trialsw_seq <- seq( from=-1 , to=2.65 , length.out=30 )
op <- par(mfrow=c(1,1), mar=c(5.9,4.9,3,0.9))
plot(dlw$latency ~ dlw$trials , pch=16 , col="black" ,
xlab="Trials to pass last reversal (standardized: mean=0)" , ylab="Avg latency (s) to attempt new locus on multiaccess box (wooden)" , xlim=c(-1,2.65), cex.lab=2, cex.axis=2, cex=2 )
mu <- link( mwlat1 , data=data.frame( trials=trialsw_seq ) )
mu_mean <- apply( mu , 2 , mean )
mu_ci <- apply( mu , 2 , PI , prob=0.97 )
lines( trialsw_seq , mu_mean , lwd=2 )
shade( mu_ci , trialsw_seq , col=col.alpha(rangi2,0.3) )
par(op)
```
![](g_flexmanip_figp2latmablog.png){width=100%}
**Figure H.** The average latency (seconds) to attempt to solve a different locus on the wooden multiaccess box after having previously successfully solved a locus is positively correlated with the number of trials to pass their last reversal (n = 11 grackles, shading = 97% prediction interval).
### P2 alternative 2 (additional analysis): latency and motor diversity
Because there was no correlation between the number of trials to reverse in the last reversal and the latency to attempt a different locus on the wooden multiaccess box, we conducted this additional analysis to determine whether the model fit was improved when adding the number of motor actions as an explanatory variable. Adding the number of motor actions (wooden: average=13, sd=4) did not improve the model fit when examining the relationship between the latency to switch loci on the plastic and wooden multiaccess boxes (wooden: average=463, sd=481) and the number of trials to reverse in the last reversal (wooden: average=60, sd=38) because the Akaike weights were similar for both models (wooden: n=11 grackles: 5 in the manipulated group, 6 in the control group; Table Z).
```{r diversity1Wresults, eval=T, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
# WOODEN MULTI-ACCESS BOX (W)
dw <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_datasummary.csv"), header=F, sep=",", stringsAsFactors=F)
dw <- data.frame(dw)
colnames(dw) <- c("Bird","Batch","Sex","Trials to learn","TrialsFirstReversal","TrialsLastReversal","ReversalsToPass","TotalLociSolvedMABplastic","TotalLociSolvedMABwooden","AverageLatencyAttemptNewLocusMABplastic","AverageLatencyAttemptNewLocusMABwooden","Trials to learn (touchscreen)","Trials to first reversal (touchscreen)","MotorActionsPlastic","MotorActionsWooden","Notes")
# Remove NAs
dw <- subset(dw,!(is.na(dw["MotorActionsWooden"])) & !(is.na(dw["TrialsLastReversal"])) & !(is.na(dw["AverageLatencyAttemptNewLocusMABwooden"])))
# n=11: 5 in manipulated group, 6 in control group
#length(dw$AverageLatencyAttemptNewLocusMABwooden)
# look at the data
#hist(dw$AverageLatencyAttemptNewLocusMABwooden)
#mean(dw$AverageLatencyAttemptNewLocusMABwooden) #463
#sd(dw$AverageLatencyAttemptNewLocusMABwooden) #481
#hist(dw$MotorActionsWooden)
#mean(dw$MotorActionsWooden) #13
#sd(dw$MotorActionsWooden) #4
#mean(dw$TrialsLastReversal) #60
#sd(dw$TrialsLastReversal) #38
# GLM
motw <- glm(dw$AverageLatencyAttemptNewLocusMABwooden ~ dw$TrialsLastReversal + dw$MotorActionsWooden)
# AIC calculation
library(MuMIn)
options(na.action = "na.fail")
dredgemw <- dredge(glm(dw$AverageLatencyAttemptNewLocusMABwooden ~ dw$TrialsLastReversal + dw$MotorActionsWooden))
library(knitr)
kable(dredgemw, caption = "")
#Akaike weights = 0.71 null and <0.15 for the rest, therefore the models with or without motor actions are essentially the same
```
### P3a: reversal is repeatable within individuals within a context
Reversal learning performance was repeatable within individuals within a context (the context being reversal learning). We obtained a repeatability value of 0.13, which is significantly greater than that expected if birds are performing randomly in each reversal (p=0.001; see analysis details in the R code for Analysis Plan > P3a). Consequently and as preplanned, we did not need to conduct the analysis for P3a alternative to determine whether a lack of repeatability was due to motivation or hunger.
### P3b: repeatable across contexts
There was no consistency of flexibility in individuals across contexts: the latency to attempt a different locus on both multiaccess boxes did not correlate within individuals with the number of trials to reverse a preference in each reversal (n=8 grackles: only those in the manipulated condition because only they experienced more than one reversal; Memela was not included because she did not complete the reversal experiment and therefore was not offered the multiaccess box experiments).
```{r consistent1results, eval=FALSE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_reverse.csv"), header=T, sep=",", stringsAsFactors=F)
#remove NAs from the variables that will be in the model
d <- subset(d,!(is.na(d["TrialsToReverse"])))
d <- subset(d,!(is.na(d["ReverseNumber"])))
#include only those birds in the reversal tubes experiment
d <- d[d$TubesOrTouchscreen=="TUBES" & d$ExperimentalGroup=="Manipulation",]
#factor variable
d$ID <- as.factor(d$ID)
#remove pilot birds (Fajita and Empanada) and Memela who did not pass the reversal experiment and therefore was not offered the MAB experiments
d <- d[!d$ID=="Fajita" & !d$ID=="Empanada" & !d$ID=="Memela",]
#n=8
#length(unique(d$ID))
# GLMM color reversal tubes compared with multi-access box plastic
library(MCMCglmm)
prior = list(R = list(R1 = list(V = 1, nu = 0), R2 = list(V = 1, nu = 0)), G = list(G1 = list(V = 1,
nu = 0)))
rm <- MCMCglmm(LatencyMABplastic ~ ReverseNumber * TrialsToReverse, random = ~ID,
family = "poisson", data = d, verbose = F, prior = prior,
nitt = 130000, thin = 1000, burnin = 30000)
summary(rm)
# post.mean l-95% CI u-95% CI eff.samp pMCMC
#(Intercept) 2.08708 -4.45451 11.67734 100 0.66
#ReverseNumber 1.01476 -2.75484 5.49974 100 0.42
#TrialsToReverse 0.01693 -0.09999 0.11593 100 0.58
#ReverseNumber:TrialsToReverse -0.01159 -0.07061 0.03283 100 0.42
#nothing significant so no consistent individual differences across contexts on MAB plastic and trials to reverse
# GLMM color reversal tubes compared with multi-access box wooden
prior = list(R = list(R1 = list(V = 1, nu = 0), R2 = list(V = 1, nu = 0)), G = list(G1 = list(V = 1,
nu = 0)))
rmw <- MCMCglmm(LatencyMABwooden ~ ReverseNumber * TrialsToReverse, random = ~ID,
family = "poisson", data = d, verbose = F, prior = prior,
nitt = 130000, thin = 1000, burnin = 30000)
#summary(rmw)
# post.mean l-95% CI u-95% CI eff.samp pMCMC
#(Intercept) 3.622381 0.148743 7.810863 159.0 0.08 .
#ReverseNumber 0.211605 -1.843271 2.126334 100.0 0.88
#TrialsToReverse 0.032183 -0.019718 0.076067 147.9 0.14
#ReverseNumber:TrialsToReverse -0.004685 -0.037464 0.014299 100.0 0.62
#nothing significant so no consistent individual differences across contexts on MAB wooden and trials to reverse
```
### P4: serial reversal learning strategy
Three out of nine grackles switched from an epsilon-decreasing to an epsilon-first strategy in their last reversal (Diablo reversal 8, Burito reversal 8, and Chilaquile reversal 6). The rest continued to rely on an epsilon-decreasing strategy throughout their reversals.
```{r p4strategyfig, eval=F, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
d <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_reverseraw.csv"), header=T, sep=",", stringsAsFactors=F)
#Include only the manipulated birds because they received serial reversals
d <- d[d$ID=="Chalupa" | d$ID=="Mole" | d$ID=="Habanero" | d$ID=="Diablo" | d$ID=="Burrito" | d$ID=="Adobo" | d$ID=="Chilaquile" | d$ID=="Pollito" | d$ID=="Memela",]
#Exclude reversal 0 because this was the initial discrimination
d <- d[!d$Reversal==0,]
#Factor ID so I can make plots for each bird
d$ID <- factor(d$ID)
#levels(d$ID) #n=9 grackles in manipulated group, including Memela who did not complete the experiment
#make data frames for each bird for each of their reversals
ch1 <- d[d$ID=="Chalupa" & d$Reversal==1,]
ch2 <- d[d$ID=="Chalupa" & d$Reversal==2,]
ch3 <- d[d$ID=="Chalupa" & d$Reversal==3,]
ch4 <- d[d$ID=="Chalupa" & d$Reversal==4,]
ch5 <- d[d$ID=="Chalupa" & d$Reversal==5,]
ch6 <- d[d$ID=="Chalupa" & d$Reversal==6,]
ch7 <- d[d$ID=="Chalupa" & d$Reversal==7,]
ch8 <- d[d$ID=="Chalupa" & d$Reversal==8,]
mo1 <- d[d$ID=="Mole" & d$Reversal==1,]
mo2 <- d[d$ID=="Mole" & d$Reversal==2,]
mo3 <- d[d$ID=="Mole" & d$Reversal==3,]
mo4 <- d[d$ID=="Mole" & d$Reversal==4,]
mo5 <- d[d$ID=="Mole" & d$Reversal==5,]
mo6 <- d[d$ID=="Mole" & d$Reversal==6,]
mo7 <- d[d$ID=="Mole" & d$Reversal==7,]
ha1 <- d[d$ID=="Habanero" & d$Reversal==1,]
ha2 <- d[d$ID=="Habanero" & d$Reversal==2,]
ha3 <- d[d$ID=="Habanero" & d$Reversal==3,]
ha4 <- d[d$ID=="Habanero" & d$Reversal==4,]
ha5 <- d[d$ID=="Habanero" & d$Reversal==5,]
ha6 <- d[d$ID=="Habanero" & d$Reversal==6,]
di1 <- d[d$ID=="Diablo" & d$Reversal==1,]
di2 <- d[d$ID=="Diablo" & d$Reversal==2,]
di3 <- d[d$ID=="Diablo" & d$Reversal==3,]
di4 <- d[d$ID=="Diablo" & d$Reversal==4,]
di5 <- d[d$ID=="Diablo" & d$Reversal==5,]
di6 <- d[d$ID=="Diablo" & d$Reversal==6,]
di7 <- d[d$ID=="Diablo" & d$Reversal==7,]
di8 <- d[d$ID=="Diablo" & d$Reversal==8,]
bu1 <- d[d$ID=="Burrito" & d$Reversal==1,]
bu2 <- d[d$ID=="Burrito" & d$Reversal==2,]
bu3 <- d[d$ID=="Burrito" & d$Reversal==3,]
bu4 <- d[d$ID=="Burrito" & d$Reversal==4,]
bu5 <- d[d$ID=="Burrito" & d$Reversal==5,]
bu6 <- d[d$ID=="Burrito" & d$Reversal==6,]
bu7 <- d[d$ID=="Burrito" & d$Reversal==7,]
bu8 <- d[d$ID=="Burrito" & d$Reversal==8,]
ad1 <- d[d$ID=="Adobo" & d$Reversal==1,]
ad2 <- d[d$ID=="Adobo" & d$Reversal==2,]
ad3 <- d[d$ID=="Adobo" & d$Reversal==3,]
ad4 <- d[d$ID=="Adobo" & d$Reversal==4,]
ad5 <- d[d$ID=="Adobo" & d$Reversal==5,]
ad6 <- d[d$ID=="Adobo" & d$Reversal==6,]
cq1 <- d[d$ID=="Chilaquile" & d$Reversal==1,]
cq2 <- d[d$ID=="Chilaquile" & d$Reversal==2,]
cq3 <- d[d$ID=="Chilaquile" & d$Reversal==3,]
cq4 <- d[d$ID=="Chilaquile" & d$Reversal==4,]
cq5 <- d[d$ID=="Chilaquile" & d$Reversal==5,]
cq6 <- d[d$ID=="Chilaquile" & d$Reversal==6,]
po1 <- d[d$ID=="Pollito" & d$Reversal==1,]
po2 <- d[d$ID=="Pollito" & d$Reversal==2,]
po3 <- d[d$ID=="Pollito" & d$Reversal==3,]
po4 <- d[d$ID=="Pollito" & d$Reversal==4,]
po5 <- d[d$ID=="Pollito" & d$Reversal==5,]
po6 <- d[d$ID=="Pollito" & d$Reversal==6,]
po7 <- d[d$ID=="Pollito" & d$Reversal==7,]
po8 <- d[d$ID=="Pollito" & d$Reversal==8,]
me1 <- d[d$ID=="Memela" & d$Reversal==1,]
me2 <- d[d$ID=="Memela" & d$Reversal==2,]
me3 <- d[d$ID=="Memela" & d$Reversal==3,]
me4 <- d[d$ID=="Memela" & d$Reversal==4,]
me5 <- d[d$ID=="Memela" & d$Reversal==5,]
me6 <- d[d$ID=="Memela" & d$Reversal==6,]
me7 <- d[d$ID=="Memela" & d$Reversal==7,]
me8 <- d[d$ID=="Memela" & d$Reversal==8,]
me9 <- d[d$ID=="Memela" & d$Reversal==9,]
me10 <- d[d$ID=="Memela" & d$Reversal==10,]
me11 <- d[d$ID=="Memela" & d$Reversal==11,]
op <- par(mfrow=c(8,1), mar=c(4,4,1,0.2))
plot(ch1$Trial,ch1$NonOverlappingWindow4TrialBins, main="Chalupa", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch2$Trial,ch2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch3$Trial,ch3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch4$Trial,ch4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch5$Trial,ch5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch6$Trial,ch6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch7$Trial,ch7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ch8$Trial,ch8$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
par(op)
op <- par(mfrow=c(7,1), mar=c(4,4,1,0.2))
plot(mo1$Trial,mo1$NonOverlappingWindow4TrialBins, main="Mole", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo2$Trial,mo2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo3$Trial,mo3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo4$Trial,mo4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo5$Trial,mo5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo6$Trial,mo6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
plot(mo7$Trial,mo7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,111), pch=19)
par(op)
op <- par(mfrow=c(6,1), mar=c(4,4,1,0.2))
plot(ha1$Trial,ha1$NonOverlappingWindow4TrialBins, main="Habanero", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,91), pch=19)
plot(ha2$Trial,ha2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,91), pch=19)
plot(ha3$Trial,ha3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,91), pch=19)
plot(ha4$Trial,ha4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,91), pch=19)
plot(ha5$Trial,ha5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,91), pch=19)
plot(ha6$Trial,ha6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,91), pch=19)
par(op)
op <- par(mfrow=c(8,1), mar=c(4,4,1,0.2))
plot(di1$Trial,di1$NonOverlappingWindow4TrialBins, main="Diablo", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di2$Trial,di2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di3$Trial,di3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di4$Trial,di4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di5$Trial,di5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di6$Trial,di6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di7$Trial,di7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
plot(di8$Trial,di8$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,171), pch=19)
par(op)
op <- par(mfrow=c(8,1), mar=c(4,4,1,0.2))
plot(bu1$Trial,bu1$NonOverlappingWindow4TrialBins, main="Burrito", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu2$Trial,bu2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu3$Trial,bu3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu4$Trial,bu4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu5$Trial,bu5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu6$Trial,bu6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu7$Trial,bu7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
plot(bu8$Trial,bu8$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,121), pch=19)
par(op)
op <- par(mfrow=c(6,1), mar=c(4,4,1,0.2))
plot(ad1$Trial,ad1$NonOverlappingWindow4TrialBins, main="Adobo", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ad2$Trial,ad2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ad3$Trial,ad3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ad4$Trial,ad4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ad5$Trial,ad5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(ad6$Trial,ad6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
par(op)
op <- par(mfrow=c(6,1), mar=c(4,4,1,0.2))
plot(cq1$Trial,cq1$NonOverlappingWindow4TrialBins, main="Chilaquile", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(cq2$Trial,cq2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(cq3$Trial,cq3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(cq4$Trial,cq4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(cq5$Trial,cq5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(cq6$Trial,cq6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
par(op)
op <- par(mfrow=c(8,1), mar=c(4,4,1,0.2))
plot(po1$Trial,po1$NonOverlappingWindow4TrialBins, main="Pollito", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po2$Trial,po2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po3$Trial,po3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po4$Trial,po4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po5$Trial,po5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po6$Trial,po6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po7$Trial,po7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
plot(po8$Trial,po8$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,101), pch=19)
par(op)
op <- par(mfrow=c(11,1), mar=c(4,4,1,0.2))
plot(me1$Trial,me1$NonOverlappingWindow4TrialBins, main="Memela", xlab="Trials (reversal 1)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me2$Trial,me2$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 2)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me3$Trial,me3$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 3)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me4$Trial,me4$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 4)", ylab="Proportion correct choices", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me5$Trial,me5$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 5)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me6$Trial,me6$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 6)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me7$Trial,me7$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 7)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me8$Trial,me8$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me9$Trial,me9$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me10$Trial,me10$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
plot(me11$Trial,me11$NonOverlappingWindow4TrialBins, xlab="Trials (reversal 8)", ylab="", ylim=c(0,1), xlim=c(0,161), pch=19)
par(op)
```
![](g_flexmanip_fig_chalupa.png){width=100%}
![](g_flexmanip_fig_mole.png){width=100%}
![](g_flexmanip_fig_habanero.png){width=100%}
![](g_flexmanip_fig_diablo.png){width=100%}
![](g_flexmanip_fig_burrito.png){width=100%}
![](g_flexmanip_fig_adobo.png){width=100%}
![](g_flexmanip_fig_chilaquile.png){width=100%}
![](g_flexmanip_fig_pollito.png){width=100%}
![](g_flexmanip_fig_memela.png){width=100%}
There was no correlation between exploration (sampling ratio) or exploitation (acquisition ratio) and reversal number (sampling: reversal estimate=-0.09, SE=0.11, z=-0.86, p=0.39; acquisition: reversal estimate=0.00, SE=0.00, z=-0, p=1.00), indicating that the grackles did not use a particular strategy earlier or later in their serial reversals.
## DISCUSSION
**Serial reversals improved switch latencies on both multiaccess boxes**. Grackles that were faster to reverse a preference in their first and last reversals were also faster to attempt to solve a new locus on the **plastic** multiaccess box, and individuals in the manipulated condition had faster switch times than those in the control condition. This indicates that, while these variables already positively correlate before conducting a manipulation, the serial reversal flexibility manipulation improved multiaccess box switching latencies. In contrast, there was no difference between control and manipulated switching performance on the **wooden** multiaccess box. Instead, serial reversal experience changed the relationship between the number of trials to reverse and switch latencies from being negative in the first reversal to having no relationship in the last reversal. This indicates that the flexibility manipulation had some effect on changing the latency to switch to attempting different loci on the wooden multiaccess box. Because there was no correlation between trials to pass last reversal and the latency to attempt a different locus on the wooden multiaccess box, we added motor diversity (the number of motor actions used on the wooden multiaccess box) to the model as a measure of innovativeness to determine whether switch latencies might measure both flexibility and innovativeness. This addition did not improve the model fit. This indicates that innovativeness is likely not a confound in switch latencies, which were influenced by serial reversals.
**Serial reversals improved problem solving on both multiaccess boxes**. Grackles in the manipulated condition solved on average 1.2 more loci on the on the **wooden** multiaccess box than those birds in the control condition. The relationship for the **plastic** multiaccess box did show an improvement, but not as we originally predicted. We predicted (P2) that there would be differences in multiaccess box performance between individuals in the control and manipulated conditions in the reversal experiment, however this was not the case: there was no difference between the conditions. We had an alternative prediction (P2 alternative 1) for if the flexibility manipulation did not work, however the manipulation did work in that almost all individuals (8/9) in the manipulated condition met the passing criterion: they were faster at reversing preferences in their last reversals than control individuals were in their first reversals. This suggests that the flexibility manipulation trained individuals who were not already fast to become faster at reversing, and, for those individuals who were already fast at reversing when they began the experiment, that other previous experience led to differences in their performance on the plastic multiaccess box. This could indicate that the important variable is the ability to be flexible, which individuals could already possess or could be trained to improve on during the experimental manipulation. That there was no correlation between the number of trials to reverse in the first reversal and the number of loci solved on either multiaccess box indicates that flexibility is not an inherent ability, but one that is shaped by experience as demonstrated by the effect of the manipulation. If it was an inherent ability, the first reversals of the manipulated group would likely have resulted in a correlation with problem solving. Instead, what appears to be important is the current state of the bird as they attempt the multiaccess box, which is captured by the final reversal performance.
**Performance differed between the two multiaccess boxes**. The number of trials to reverse a preference in the first reversal positively correlated with switch latencies on the plastic multiaccess box, but negatively on the wooden multiaccess box, which calls into question what switching latencies actually measure. For the number of loci solved, there was negative correlation with trials to pass last reversal on the plastic multiaccess box (birds who reversed faster solved more loci), and a difference between manipulated and control birds on the wooden multiaccess box (manipulated birds solved more loci). The differences between the two multiaccess boxes could be due to a small sample size, which means that we were not able to detect an association with all reversal learning measures (first reversal, last reversal, control versus manipulated).
Examining only the manipulated grackles, there was **repeatability of flexibility performance within a context (reversal learning), but not across contexts (reversal learning and multiaccess boxes)**. Individuals who were faster at reversing a color preference in reversal 1 were also generally faster at reversing in subsequent reversals. However, individuals who were consistently faster at reversing in each reversal were not also those who were faster at switching latencies on either multiaccess box. Therefore, individual-level flexibility did not generalize to a new context, though the invididual's reinforcement history and motivation levels could play a role, and it is also possible that performance on the multiaccess boxes additionally relies on other cognitive abilities in which individuals differ.
While one third of the grackles switched from an exploratory **strategy** (epsilon-decreasing) to an exploitative strategy (epsilon-first) in their last reversal, there was no correlation between either strategy and reversal number, indicating that the grackles did not use a particular strategy earlier or later in their serial reversals. This could suggest that the grackles did not learn the overarching rule that once food is not present in the preferred tube, they must switch to preferring the other color. Instead, they may learn each preference change as if it was new.
We conducted a **post-hoc exploratory analysis** on the serial reversal learning data to better understand the effect the flexibility manipulation had on performance. We used the version of the Bayesian model that was developed by @blaisdell2021causal and modified by @logan2020xpop [see Analysis Plan > Flexibility analysis in @logan2020xpop for model specifications and validation]. This model uses data from every trial of reversal learning (rather than only using the total number of trials to pass criterion) and represents behavioral flexibility using two parameters: the learning rate of attraction to either option ($\phi$) and the rate of deviating from learned attractions ($\lambda$).
*Using simulations to check models estimating potential factors underlying performance in reversal tests*
We first run the Bayesian model on simulated data to better understand how the two parameters might lead to differences in performance and whether we would be able to detect meaningful differences between control and manipulated birds. The settings for the simulations were based on the previous analysis of data from birds in a different population (Santa Barbara, @blaisdell2021causal). When we used only the choices simulated individuals made during their one reversal, the estimated phi and lambda values did not match those individuals had been assigned. We realized that phi and lambda values were consistently shifted in a correlated way. With only a single performance, there was equifinality with multiple combinations of the two parameters phi and lambda leading to the performance of birds during the reversal. However, when we combined data from across at least one switch in the color of the rewarded option, either combining initial association learning with the first reversal or taking the choices from the last two reversals, the model accurately recovered the phi and lambda values that simulated individuals had been assigned. In terms of the influence of the two parameters phi and lambda on the number of trials birds needed to reverse an association, we see that the phi value assigned to simulated individuals have a stronger influence than the lambda values (estimated association with standardized values for phi: -21, 89%PI:-22 - -19; for lambda -14, 89PI: -16 - -13). In particular, low numbers of trials to reverse can be observed across the full range of lambda value, though when lambda is smaller than 8 simulated birds can need 150 or more trials to reverse a preference (Figure AB). In contrast, there is a more linear relationship between phi and the number of trials to reverse, with birds needing fewer trials the larger their phi.
![](g_flexmanip_figsimulationsphilambda.png)
**Figure AB.** In the simulations, the phi values assigned to individuals (green) have a clearer influence on the number of trials these individuals needed in their reversal than their lambda values (green). Both phi and lambda values have been standardised for direct comparison. In general, individuals need fewer trials if they have larger phi and lambda values. However, for individuals with relatively small lambda values are found across the range of performances, whereas there is a more clear distinction in phi.
```{r posthoc_simulationchecks, eval=FALSE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
################################################################################################
# Load previously simulated data from xpop
################################################################################################
# These two are the sets we decided on, with initial attractions at 0.1 and eight different phi and four different lambda combinations
simulatedreversaldata_attractionscores_1<-read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/gxpopbehaviorhabitat_SimulatedReversalData_Grackles_PhiLambda_Attraction02_Aug2021.csv"), header=T, sep=",", stringsAsFactors=F)
simulatedreversaldata_attractionscores_2<-read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/gxpopbehaviorhabitat_SimulatedReversalData_Grackles_PhiLambda_Attraction04_Aug2021.csv"), header=T, sep=",", stringsAsFactors=F)
# In both simulations, sites were counted from 1-16; for the second simulation we change this to 17-32
simulatedreversaldata_attractionscores_2$Site<-simulatedreversaldata_attractionscores_2$Site+16
# In both simulations, individuals were counted from 1-320; for the second population we change the ids to start at 321
simulatedreversaldata_attractionscores_2$Bird_ID<-simulatedreversaldata_attractionscores_2$Bird_ID+320
# We combine the two data sets for the further analyses
library(dplyr)
simulatedreversaldata_attractionscores<-bind_rows(simulatedreversaldata_attractionscores_1,simulatedreversaldata_attractionscores_2)
################################################################################################
# In the simulations, trials were counted continuously for each bird. We now want to change this so that it restarts counting trials from 1 upward once a bird switches to reversal.
for (birds in 1:length(unique(simulatedreversaldata_attractionscores$Bird_ID))){
currentbird<-unique(simulatedreversaldata_attractionscores$Bird_ID)[birds]
maximuminitial<-max(simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Bird_ID==currentbird & simulatedreversaldata_attractionscores$Reversal == "initial",]$Trial)
simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Bird_ID==currentbird & simulatedreversaldata_attractionscores$Reversal == "reversal",]$Trial<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Bird_ID==currentbird & simulatedreversaldata_attractionscores$Reversal == "reversal",]$Trial - maximuminitial
}
# We need to adjust the coding during the reversal learning so that "correct" now matches whether it is correct or not.
simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Choice==0,]$Choice<-2
# To use the model to estimate the phi and lambda parameters, we first need to change the column names to match these to the specifications in the model: change Bird_ID to id; change Reversal to Choice, change CorrectChoice to Correct, change Site to Expid
colnames(simulatedreversaldata_attractionscores)<-c("counter","id","Session","Trial","Reversal","Choice","Correct","Phi_mean","Lambda_mean","Site","Phi_sd","Lambda_sd","ThisBirdsPhi","ThisBirdsLambda","Attraction1","Attraction2")
# There are several simulated individuals who never reached the criterion during the initial learning phase. We need to remove these from the dataset
birdswithreversal<-as.data.frame(simulatedreversaldata_attractionscores %>% group_by(id) %>% summarise(experiments=length(unique(Reversal))))
birdswithreversal<-birdswithreversal[birdswithreversal$experiments==2,]
simulatedreversaldata_attractionscores<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$id %in% birdswithreversal$id,]
# Next, we need to change the ids of the birds to be continuous again so the STAN model will include them all
simulatedreversaldata_attractionscores$id<-as.integer(as.factor(simulatedreversaldata_attractionscores$id))
# We first focus only on the performance in the reversal trials
simulatedreversaldata_attractionscores_reversalphase<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Reversal=="reversal",]
# Let's start with 30 individuals for comparison
firstreversal_simulated<-simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$id %in% c(20,40,60,80,100,120,140,160,180,200,220,240,260,300,320,340,360,380,400,420,440,460,480,500,520,540,560,580,600,620),]
firstreversal_simulated$id<-as.numeric(as.factor(firstreversal_simulated$id))
# We can now extract the relevant data from the first reversal for the STAN model to estimate phi and lambda at the beginning
datfirstsimulated <- as.list(firstreversal_simulated)
datfirstsimulated$N <- nrow(firstreversal_simulated)
datfirstsimulated$N_id <- length(unique(firstreversal_simulated$id))
# Next, we also look at the estimation of the phi and lambda values based on their performance in the initial association learning phase
# We first focus only on the performance in the reversal trials
simulatedreversaldata_attractionscores_learningphase<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Reversal=="initial",]
# Let's start with 30 individuals for comparison
initiallearning_simulated<-simulatedreversaldata_attractionscores_learningphase[simulatedreversaldata_attractionscores_learningphase$id %in% c(20,40,60,80,100,120,140,160,180,200,220,240,260,300,320,340,360,380,400,420,440,460,480,500,520,540,560,580,600,620),]
initiallearning_simulated$id<-as.numeric(as.factor(initiallearning_simulated$id))
# We can now extract the relevant data from the first reversal for the STAN model to estimate phi and lambda at the beginning
datinitialsimulated <- as.list(initiallearning_simulated)
datinitialsimulated$N <- nrow(initiallearning_simulated)
datinitialsimulated$N_id <- length(unique(initiallearning_simulated$id))
# The STAN model is set up to have the inital attraction for each option set to 0.1, and that individuals only learn the reward of the option they chose in a given trial.
reinforcement_model_nonzeroattraction <- "
data{
int N;
int N_id;
int id[N];
int Trial[N];
int Choice[N];
int Correct[N];
}
parameters{
real logit_phi;
real log_L;
// Varying effects clustered on individual
matrix[2,N_id] z_ID;
vector<lower=0>[2] sigma_ID; //SD of parameters among individuals
cholesky_factor_corr[2] Rho_ID;
}
transformed parameters{
matrix[N_id,2] v_ID; // varying effects on stuff
v_ID = ( diag_pre_multiply( sigma_ID , Rho_ID ) * z_ID )';
}
model{
matrix[N_id,2] A; // attraction matrix
logit_phi ~ normal(0,1);
log_L ~ normal(0,1);
// varying effects
to_vector(z_ID) ~ normal(0,1);
sigma_ID ~ exponential(1);
Rho_ID ~ lkj_corr_cholesky(4);
// initialize attraction scores
for ( i in 1:N_id ) {
A[i,1] = 0.1; A[i,2] = 0.1';
}
// loop over Choices
for ( i in 1:N ) {
vector[2] pay;
vector[2] p;
real L;
real phi;
// first, what is log-prob of observed choice
L = exp(log_L + v_ID[id[i],1]);
p = softmax(L*A[id[i],1:2]' );
Choice[i] ~ categorical( p );
// second, update attractions conditional on observed choice
phi = inv_logit(logit_phi + v_ID[id[i],2]);
pay[1:2] = rep_vector(0,2);
pay[ Choice[i] ] = Correct[i];
A[ id[i] , Choice[i] ] = ( (1-phi)*(A[ id[i] , Choice[i] ]) + phi*pay[Choice[i]])';
}//i
}
"
# We run this model for the first reversal
m_firstsimulated <- stan( model_code = reinforcement_model_nonzeroattraction, data=datfirstsimulated ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sfirstsimulated <- extract.samples(m_firstsimulated)
firstreversal_simulatedlambda <- sapply(1 : datfirstsimulated$N_id, function(x) exp( mean(sfirstsimulated$log_L) + mean(sfirstsimulated$v_ID[ ,x, 1])))
firstreversal_simulatedphi <- sapply(1 : datfirstsimulated$N_id, function(x) inv_logit( mean(sfirstsimulated$logit_phi) + mean(sfirstsimulated$v_ID[ ,x, 2])))
# And we run this model for the initial learning phase
m_initialsimulated <- stan( model_code = reinforcement_model_nonzeroattraction, data=datinitialsimulated ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sinitialsimulated <- extract.samples(m_initialsimulated)
initiallearning_simulatedlambda <- sapply(1 : datinitialsimulated$N_id, function(x) exp( mean(sinitialsimulated$log_L) + mean(sinitialsimulated$v_ID[ ,x, 1])))
initiallearning_simulatedphi <- sapply(1 : datinitialsimulated$N_id, function(x) inv_logit( mean(sinitialsimulated$logit_phi) + mean(sinitialsimulated$v_ID[ ,x, 2])))
# We now can get back the phi and lambda values 30 individuals were assigned at the beginning of the simulation
simulatedphis<-unique(simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$id %in% c(20,40,60,80,100,120,140,160,180,200,220,240,260,300,320,340,360,380,400,420,440,460,480,500,520,540,560,580,600,620),]$ThisBirdsPhi)
simulatedlambdas<-unique(simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$id %in% c(20,40,60,80,100,120,140,160,180,200,220,240,260,300,320,340,360,380,400,420,440,460,480,500,520,540,560,580,600,620),]$ThisBirdsLambda)
#
# Some of the phi values estimated from the performance during the initial learning are estimated as higher than what the individuals had during the simulation.
plot(initiallearning_simulatedphi~simulatedphis,xlim=c(0,0.08),ylim=c(0,0.08))
abline(a=0,b=1)
# In contrast, some of the lambda values estimated from the performance during the initial learning are estimated as lower than what the individuals had during the simulation
plot(initiallearning_simulatedlambda~simulatedlambdas)
abline(a=0,b=1)
# The issue likely arises because the STAN model assumes that the phi and lambda values are correlated - whereas in the simulations they were allowed to vary independently from each other
plot(initiallearning_simulatedphi~initiallearning_simulatedlambda)
plot(simulatedphis~simulatedlambdas)
# In the simulation, we set some high lambda values and low phi values - because of the assumed correlation, the STAN model estimates higher phi values than simulated in cases when lambda was high, and lower lambda values than simulated when phi was low
plot(initiallearning_simulatedphi[simulatedlambdas<5]~simulatedphis[simulatedlambdas<5],xlim=c(0,0.08),ylim=c(0,0.08))
points(initiallearning_simulatedphi[simulatedlambdas>5]~simulatedphis[simulatedlambdas>5],xlim=c(0,0.08),ylim=c(0,0.08),col="red")
abline(a=0,b=1)
# The phi values for the first reversal are systematically underestimated. This likely results from the model assuming that the attraction scores for both options are initially equal, while in reality there will be a skewed attraction towards the option that was rewarded during the initial learning. According, the birds seem to change their behaviour very slowly (because in the estimation they do not have to overcome their initially biased attraction) leading to the model estimating much lower phi values than those that the birds really had.
plot(firstreversal_simulatedphi~simulatedphis,xlim=c(0,0.06),ylim=c(0,0.06))
abline(a=0,b=1)
# We can see how skewed the attraction scores were in the simulation at the beginning of the first reversal learning trial and use these values as priors in the STAN model (instead of the current setup where both attraction scores are set to be 0.1)
median(simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$Trial==1,]$Attraction1/simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$Trial==1,]$Attraction2)
median(simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$Trial==1,]$Attraction1)
median(simulatedreversaldata_attractionscores_reversalphase[simulatedreversaldata_attractionscores_reversalphase$Trial==1,]$Attraction2)
# Based on this we want to set it to 0.1 and 0.7
# Try different priors to reduce the correlation between estimated phis and lambdas
reinforcement_model_nonzeroattraction_alternativepriors <- "
data{
int N;
int N_id;
int id[N];
int Trial[N];
int Choice[N];
int Correct[N];
}
parameters{
real logit_phi;
real log_L;
// Varying effects clustered on individual
matrix[N_id,2] v_ID;
}
model{
matrix[N_id,2] A; // attraction matrix
logit_phi ~ normal(0,1);
log_L ~ normal(0,1);
// varying effects
to_vector(v_ID) ~ normal(0,1);
// initialize attraction scores
for ( i in 1:N_id ) {
A[i,1] = 0.1; A[i,2] = 0.1';
}
// loop over Choices
for ( i in 1:N ) {
vector[2] pay;
vector[2] p;
real L;
real phi;
// first, what is log-prob of observed choice
L = exp(log_L + v_ID[id[i],1]);
p = softmax(L*A[id[i],1:2]' );
Choice[i] ~ categorical( p );
// second, update attractions conditional on observed choice
phi = inv_logit(logit_phi + v_ID[id[i],2]);
pay[1:2] = rep_vector(0,2);
pay[ Choice[i] ] = Correct[i];
A[ id[i] , Choice[i] ] = ( (1-phi)*(A[ id[i] , Choice[i] ]) + phi*pay[Choice[i]])';
}//i
}
"
m_initialsimulated_alternativepriors <- stan( model_code = reinforcement_model_nonzeroattraction_alternativepriors, data=datinitialsimulated ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sinitialsimulatedalternativepriors <- extract.samples(m_initialsimulated_alternativepriors)
initiallearning_simulatedlambda_alternativepriors <- sapply(1 : datinitialsimulated$N_id, function(x) exp( mean(sinitialsimulatedalternativepriors$log_L) + mean(sinitialsimulatedalternativepriors$v_ID[ ,x, 1])))
initiallearning_simulatedphi_alternativepriors <- sapply(1 : datinitialsimulated$N_id, function(x) inv_logit( mean(sinitialsimulatedalternativepriors$logit_phi) + mean(sinitialsimulatedalternativepriors$v_ID[ ,x, 2])))
# Need to change the priors for the attraction scores 0.1 and 0.7
# Based on this information, we can now modify the STAN model to have the prior for the attraction for option set 1 (the option rewarded during the initial learning) to 0.7 and for option 2 set to 0.1, and that individuals only learn the reward of the option they chose in a given trial.
reinforcement_model_nonzeroattraction_skewedpriorattraction <- "
data{
int N;
int N_id;
int id[N];
int Trial[N];
int Choice[N];
int Correct[N];
}
parameters{
real logit_phi;
real log_L;
// Varying effects clustered on individual
matrix[2,N_id] z_ID;
vector<lower=0>[2] sigma_ID; //SD of parameters among individuals
cholesky_factor_corr[2] Rho_ID;
}
transformed parameters{
matrix[N_id,2] v_ID; // varying effects on stuff
v_ID = ( diag_pre_multiply( sigma_ID , Rho_ID ) * z_ID )';
}
model{
matrix[N_id,2] A; // attraction matrix
logit_phi ~ normal(0,1);
log_L ~ normal(0,1);
// varying effects
to_vector(z_ID) ~ normal(0,1);
sigma_ID ~ exponential(1);
Rho_ID ~ lkj_corr_cholesky(4);
// initialize attraction scores
for ( i in 1:N_id ) {
A[i,1] = 0.7; A[i,2] = 0.1';
}
// loop over Choices
for ( i in 1:N ) {
vector[2] pay;
vector[2] p;
real L;
real phi;
// first, what is log-prob of observed choice
L = exp(log_L + v_ID[id[i],1]);
p = softmax(L*A[id[i],1:2]' );
Choice[i] ~ categorical( p );
// second, update attractions conditional on observed choice
phi = inv_logit(logit_phi + v_ID[id[i],2]);
pay[1:2] = rep_vector(0,2);
pay[ Choice[i] ] = Correct[i];
A[ id[i] , Choice[i] ] = ( (1-phi)*(A[ id[i] , Choice[i] ]) + phi*pay[Choice[i]])';
}//i
}
"
# We run this model for the first reversal
m_firstsimulated_skewedpriorattraction <- stan( model_code = reinforcement_model_nonzeroattraction_skewedpriorattraction, data=datfirstsimulated ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sfirstsimulatedskewedpriorattraction <- extract.samples(m_firstsimulated_skewedpriorattraction)
firstreversalsimulated_lambda_skewedpriorattraction <- sapply(1 : datfirstsimulated$N_id, function(x) exp( mean(sfirstsimulatedskewedpriorattraction$log_L) + mean(sfirstsimulatedskewedpriorattraction$v_ID[ ,x, 1])))
firstreversalsimulated_phi_skewedpriorattraction <- sapply(1 : datfirstsimulated$N_id, function(x) inv_logit( mean(sfirstsimulatedskewedpriorattraction$logit_phi) + mean(sfirstsimulatedskewedpriorattraction$v_ID[ ,x, 2])))
plot(firstreversalsimulated_phi_skewedpriorattraction~simulatedphis,xlim=c(0,0.06),ylim=c(0,0.06))
# In these estimations based on the performance during single setups (either just the initial learning or the first reversal learning) the model always estimates that lambda and phi are correlated. This likely reflects equifinality - individuals can achieve the same performance with a range of phis and lambdas, and the model will slide to the middle along the line for each individual:
plot(x="lambda",y="phi",xlim=c(0,10),ylim=c(0,0.1))
# Individuals who needed a long time to learn the association will be in the bottom left corner
abline(a=0.04,b=-0.01,lty=2)
abline(a=0.06,b=-0.01,lty=2)
abline(a=0.08,b=-0.01,lty=2)
# Individuals who needed a short time to learn the association will be in the top right corner
abline(a=0.10,b=-0.01,lty=2)
abline(a=0.12,b=-0.01,lty=2)
abline(a=0.14,b=-0.01,lty=2)
points(x=1,y=0.03,cex=2)
points(x=2,y=0.04,cex=2)
points(x=3,y=0.05,cex=2)
points(x=4,y=0.06,cex=2)
points(x=5,y=0.07,cex=2)
points(x=6,y=0.08,cex=2)
abline(a=0.02,b=0.01,col="red",lwd=1.5)
points(initiallearning_simulatedphi~initiallearning_simulatedlambda,pch=2)
# Maybe the model can better separate the lambda and phi values when combining data from multiple runs - in the case of the simulations that means combining the data from the initial learning with the data of the first reversal
simulatedreversaldata_attractionscores_reversalphase<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$Reversal=="reversal",]
# Let's start with 30 individuals for comparison
initialandreversal_simulated<-simulatedreversaldata_attractionscores[simulatedreversaldata_attractionscores$id %in% c(20,40,60,80,100,120,140,160,180,200,220,240,260,300,320,340,360,380,400,420,440,460,480,500,520,540,560,580,600,620),]
initialandreversal_simulated$id<-as.numeric(as.factor(initialandreversal_simulated$id))
# We can now extract the relevant data from the first reversal for the STAN model to estimate phi and lambda at the beginning
datinitialandreversalsimulated <- as.list(initialandreversal_simulated)
datinitialandreversalsimulated$N <- nrow(initialandreversal_simulated)
datinitialandreversalsimulated$N_id <- length(unique(initialandreversal_simulated$id))
m_initialandreversal <- stan( model_code = reinforcement_model_nonzeroattraction, data=datinitialandreversalsimulated ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sinitialandreversal <- extract.samples(m_initialandreversal)
initialandreversal_lambda <- sapply(1 : datinitialandreversalsimulated$N_id, function(x) exp( mean(sinitialandreversal$log_L) + mean(sinitialandreversal$v_ID[ ,x, 1])))
initialandreversal_phi <- sapply(1 : datinitialandreversalsimulated$N_id, function(x) inv_logit( mean(sinitialandreversal$logit_phi) + mean(sinitialandreversal$v_ID[ ,x, 2])))
plot(initialandreversal_phi~simulatedphis)
abline(a=0,b=1)
plot(initialandreversal_lambda~simulatedlambdas)
abline(a=0,b=1)
plot(initialandreversal_phi~initialandreversal_lambda)
currentlocation<-getwd()
cmdstanlocation <- cmdstan_path()
setwd(cmdstanlocation)
# access the output file created by the model running the reinforcement model
write(reinforcement_model_nonzeroattraction_alternativepriors,file="myowntrial.stan")
file <- file.path(cmdstan_path(), "myowntrial.stan")
mod <- cmdstan_model(file)
options(mc.cores=4)
# RUN the model
fit <- mod$sample(
data = datinitialandreversalsimulated,
seed = 123,
chains = 4,
parallel_chains = 4,
refresh = 500
)
# Extract relevant variables
outcome<-data.frame(fit$summary())
rownames(outcome)<-outcome$variable
# Show the 90% compatibility intervals for the association between latency to switch loci on the plastic multi-access box and lambda and phi, and the interaction between lambda and phi from the reinforcement learning model
drawsarray<-fit$draws()
drawsdataframe<-as_draws_df(drawsarray)
drawsdataframe<-data.frame(drawsdataframe)
initialandreversal_lambda <- sapply(1 : datinitialandreversalsimulated$N_id, function(x) exp( mean(drawsdataframe$log_L) + mean(drawsdataframe[,x+3])))
initialandreversal_phi <- sapply(1 : datinitialandreversalsimulated$N_id, function(x) inv_logit( mean(drawsdataframe$logit_phi) + mean(drawsdataframe[,x+33])))
# Remove the stan command line file we created for this particular model from your computer
fn<-"myowntrial"
file.remove(fn)
# Reset your working directory to what it was before we ran the model
setwd(currentlocation)
simulatedphi<-initialandreversal_simulated %>% group_by(id) %>% summarise(mean(Phi_mean))
simulatedphi<-as.data.frame(simulatedphi)
simulatedphis<-simulatedphi[,2]
# OPEN QUESTIONS:
# How did you decide that the manipulation worked?
# two consecutive reversals passed in 50 or less trials
# Can check what phi/lambda has to be so that they pass in 50 or less trials in the simulation
# Is it easier to change phi or lambda to get at 50 or fewer trials?
# We might want to compare first 20 trials to last 20 trials, both for the simulated data, and for
# the observed data look at the first and last 20 trials for each the first and the last reversal
#
summarysimulateddata<-matrix(nrow=length(unique(simulatedreversaldata_attractionscores$id)),ncol=5)
summarysimulateddata<-as.data.frame(summarysimulateddata)
colnames(summarysimulateddata)<-c("id","ThisBirdsPhi","ThisBirdsLambda","TrialsInitial","TrialsReversal")
summarysimulateddata$id<-unique(simulatedreversaldata_attractionscores$id)
for (i in 1:nrow(summarysimulateddata)){
summarysimulateddata[i,]$TrialsInitial<-max(filter(simulatedreversaldata_attractionscores,id==unique(simulatedreversaldata_attractionscores$id)[i],Reversal=="initial")$Trial)
}
for (i in 1:nrow(summarysimulateddata)){
summarysimulateddata[i,]$TrialsReversal<-max(filter(simulatedreversaldata_attractionscores,id==unique(simulatedreversaldata_attractionscores$id)[i],Reversal=="reversal")$Trial)
}
for (i in 1:nrow(summarysimulateddata)){
summarysimulateddata[i,]$ThisBirdsPhi<-max(filter(simulatedreversaldata_attractionscores,id==unique(simulatedreversaldata_attractionscores$id)[i])$ThisBirdsPhi)
}
for (i in 1:nrow(summarysimulateddata)){
summarysimulateddata[i,]$ThisBirdsLambda<-max(filter(simulatedreversaldata_attractionscores,id==unique(simulatedreversaldata_attractionscores$id)[i])$ThisBirdsLambda)
}
plot(summarysimulateddata$TrialsReversal~summarysimulateddata$ThisBirdsPhi)
plot(summarysimulateddata$TrialsReversal~summarysimulateddata$ThisBirdsLambda)
dat_trialsphiandlambda<- list(
Trials = (summarysimulateddata$TrialsReversal),
bird = c(as.numeric(as.factor(summarysimulateddata$id))),
phi = standardize(c(summarysimulateddata$ThisBirdsPhi)),
lambda = standardize(c(summarysimulateddata$ThisBirdsLambda))
)
trials.phiandlambda <- ulam(
alist(
Trials ~ normal( mu , sigma ),
mu <- a + b*phi+c*lambda,
a ~ normal(70,40),
b ~ normal(0,20),
c ~ normal(0,20),
sigma ~ exponential(1)
) , data=dat_trialsphiandlambda , chains=4 , cores=4,iter=10000 )
precis(trials.phiandlambda,depth=2)
# mean sd 5.5% 94.5% n_eff Rhat4
# a 92.33 0.94 90.84 93.83 24367 1
# b -20.62 0.94 -22.12 -19.11 25492 1
# c -14.25 0.94 -15.74 -12.75 24876 1
# sigma 23.38 0.64 22.37 24.43 24251 1
summarysimulateddata_forplotting<-matrix(ncol=3,nrow=2*nrow(summarysimulateddata))
summarysimulateddata_forplotting<-as.data.frame(summarysimulateddata_forplotting)
colnames(summarysimulateddata_forplotting)<-c("TrialsReversal","Predictor","Value")
summarysimulateddata_forplotting$TrialsReversal<-c(summarysimulateddata$TrialsReversal,summarysimulateddata$TrialsReversal)
summarysimulateddata_forplotting$Predictor<-c(rep("phi",nrow(summarysimulateddata)),rep("lambda",nrow(summarysimulateddata)))
summarysimulateddata_forplotting$Value<-c(standardize(summarysimulateddata$ThisBirdsPhi),standardize(summarysimulateddata$ThisBirdsLambda))
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>181,]$TrialsReversal<-8
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>151,]$TrialsReversal<-7
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>131,]$TrialsReversal<-6
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>111,]$TrialsReversal<-5
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>91,]$TrialsReversal<-4
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>71,]$TrialsReversal<-3
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>51,]$TrialsReversal<-2
summarysimulateddata_forplotting[summarysimulateddata_forplotting$TrialsReversal>31,]$TrialsReversal<-1
summarysimulateddata_forplotting$TrialsReversal<-as.factor(summarysimulateddata_forplotting$TrialsReversal)
library(ggplot2)
ggplot(summarysimulateddata_forplotting, aes(x=TrialsReversal, y=Value, fill=Predictor)) +
geom_boxplot()+xlab("Trials simulated individuals needed in reversal")+scale_y_continuous(name = "Standardised phi/lambda of simulated individuals") + theme_classic()+scale_x_discrete(name="Trials simulated individuals needed in reversal",breaks=1:8, labels=c("31-50","51-70","71-90","91-110","111-130","131-150","151-180","181-220"))+theme(axis.text.x = element_text(size = 14, colour ="black",
hjust = 0.5,
angle = 0)) +
theme(axis.title.x = element_text(size = 18, colour ="black", face = "bold",
hjust = 0.5,
vjust = -0.5,
angle = 0))+
theme(axis.text.y = element_text(size = 14, colour ="black",
hjust = 0.5,
angle = 0)) +
theme(axis.title.y = element_text(size = 16, colour ="black", face = "bold",
hjust = 0.5,
angle = 90)) + theme(legend.title = element_text(size = 13))
```
With this Bayesian estimation of the parameters phi and lambda which underlie the performance in the reversal learning task, we wanted to address the following questions: 1) What did the manipulation change? Can we determine what mechanisms of flexibility the birds in the manipulated group who were already fast at reversing rely on? We predicted that birds that were already faster at reversing would have similar deviation rates from the learned attractions between the first and last reversals and lower learning rates than slower birds, which would allow them to change their preference more quickly because the attraction would be weaker and easier to reverse. 2) Do the manipulations shift birds beyond what is naturally observed and does it make them more similar? In the analyses above, it was unclear how it occurred that there was an effect on innovation and flexibility on the multiaccess boxes when, in some cases, there was no difference between the control and manipulated conditions. Therefore, for both the control and manipulated groups, we investigated whether the learning rate and rate of deviating from learned attractions differed between a bird's first 10 trials of the first and last reversals and whether what we observe among the manipulated birds at the end might already naturally present in some birds of the control group. In addition, we want to know if the manipulations affect all birds equally or if we can still detect variation. 3) Are phi or lambda, the two two components of behavioural flexibility in the reversal learning, associated with performance on the multiaccess boxes across control and manipulated birds? In the analyses above, we detected some associations between a bird's performance in the reversal learning task and on the multi-access boxes. Looking at the two parameters phi and lambda separately might offer a more detailed understanding of potential abilities that might influence performance in the different tasks.
*1) Observed effects of manipulation on performance, phi and lambda*
Manipulated birds improved from needing a median of 75 trials during their first reversal (nonmanipulated birds 70 trials) to needing a median of 40 trials during their last reversal. A pooled model estimates that birds can expect to improve by about 30 trials (89% PI 25-36) (Figure Y). While all birds improve, those birds that already needed fewer trials in the initial reversal improve less than the birds that initially needed many trials (posterior peak indicating a correlation between initial value and improvement of +0.22). However, the initially worse performing birds still do not catch up such that the bird who needed the fewest trials to reverse initially also needed the fewest trials to reverse in the last reversal, but the difference to birds performing worse is reduced in the last reversal. For these manipulated birds, the trials they needed in the last reversal equals roughly (trials first reversal)^2 / 200. This means that also among the manipulated birds, we still detect variation in their performance, and this variation in performance appears bird specific. Their performance in the last reversal and how much they improved is not linked to how many reversals they needed to reach criterion. Interestingly, most birds performed worse during the middle of the manipulation before improving and reaching the criterion.
![](gflexmanip_figure_manipulatedbirdstrialimprovement.png)
**Figure Y.** All eight manipulated birds need fewer trials to reverse in their last reversal than in their first. Their improvement depends on their starting value, with steeper slopes for those birds that needed more trials to reverse in the first reversal (blue colours for observed values and changes, black colours for model estimates). However, birds who needed more trials in the first reversal do not completely catch up, such that the birds that needed more trials in their first reversal also needed more trials in their last reversal than the other birds.
The findings from the simulated data indicated that lambda and phi can only be estimated accurately when calculating them across at least one switch (initial learning plus initial reversal or final two reversals). For the manipulated bird, the estimated phi more than doubled from 0.03 in their first reversal (nonmanipulated 0.03) to 0.07 in their last reversal (model estimate of expected average change +0.02 - +0.05), while their lambda went slightly down from 4.2 (nonmanipulated 4.3) to 3.2 (model estimate of average change -1.63 - -0.56). For phi, this pattern fits with the observations in the simulations, that larger phi values are associated with fewer trials to reverse. However, in the simulations we saw a negative correlation between lambda and the number of trials to reverse whereas for the birds here lambda in the last reversal when birds needed fewer trials is smaller than their lambda for the initial reversal. This does fit though with the observation that lambda seems to be a constraint rather than a direct linear influence on the number of trials, and that in combination with the correct phi value birds only need few trials to reverse even if they have a smaller lambda (see also below).
For the phi values, we also observe the same correlation between the initial value and the change toward the phi in the last reversal (-0.4; 50% HPDI all negative), while there is no such obvious relationship for lambda (-0.15; 50% HPDI spans zero). We can see that the manipulation changes both phi and lambda, so that across all birds there is a negative correlation between phi and lambda. In addition, for the manipulated birds, there is also a clear negative correlation among the phi and lambdas during their final reversal.
*2) Variation in performance, phi and lambda*
The values for the number of trials, phi, and lambda we observed after the manipulation in the last reversal fall within the variation we observe among the control birds in their first and only reversal. This means that the manipulation did not push birds to new levels but changed them within the boundaries that are induced in their natural environment. This fits with the observation that, for the correlations with performance on the multi-access box, there was generally no difference between the control and the manipulated birds and that instead their performance in the last trial before they attempted these other tasks (first reversal for control, final reversal for manipulated) showed associations with some of the other behaviors.
Across both manipulated and control birds, phi is more consistently associated with the number of trials individuals needed to reverse, and phi changes more than lambda for the manipulated birds (Figure Z). However, both changes in phi and lambda independently explain changes in the improvement in performance of the manipulated birds from the first to the last reversal (association of change in number of trials from first to last trial with standardized change in phi: 11, 89%PI:6-15 and with standardized lambda: 6, 89%PI:1-10).
As mentioned above, birds who needed fewer trials than other birds during the first reversal also needed fewer trials during the last reversal. Combining all these relationships into a single model suggests that the initial phi determines the number of trials individuals need during the first reversal, which in turn explains how many trials they need during their last reversal. The phi for the last reversal does not appear to provide any additional information about the number of trials during the last reversal, and lambda is not associated with the number of trials birds need to reverse.
![](g_flexmanip_figchangemanipulatedbirds.png)
**Figure Z.** Comparisons of the different measures of performance in the reversal task for each of the 19 birds. The figure shows the trials to reach the association criterion after the reversal (a, top) during the initial reversal (for all birds, yellow) and the final reversal (for manipulated birds, blue), the phi value reflecting their updating of their information about the two options during those trials and the trials before (initial = initial association plus first reversal; manipulated = last two reversals) (b, middle), and the lambda values reflecting their exploration of the two options during those trials (c, bottom). Individual birds have the same position along the x-axis in all three panels. Birds that only needed few trials to reverse their preference to the new association generally have higher phi values, whereas lambda appears to reflect more whether any choices of the unrewarded tube occurred throughout the trials or only at the beginning. The values do not align perfectly because the phi and lambda values are estimated across two rather than just the one reversal (e.g. bird 2, Mofongo, has a very high phi because he only needed 20 trials during the initial association learning). For the manipulated birds, their phi values change more consistently than their lambda values, and the phi values of the manipulated individuals are generally higher than those observed in the control group while their lambda values remain within the range also observed in the control group.
```{r posthoc, eval=FALSE, warning=FALSE, results='asis', echo=TRUE, include=TRUE}
### Code below copied from Blaisdell et al. 2021
# Using OBSERVED not simulated data
# We want to estimate lambda and phi differently. For the initial values, we combine the data from the first association learning with the first reversal.
dflex <- read.csv(url("https://raw.githubusercontent.com/corinalogan/grackles/master/Files/Preregistrations/g_flexmanip_data_reverseraw.csv"), header=T, sep=",", stringsAsFactors=F)
library(rstan)
library(rethinking)
library(cmdstanr)
library(posterior)
# If you have cmdstan installed, use the following:
# set_ulam_cmdstan(TRUE)
### Code below copied from Blaisdell et al. 2021
# PREPARE reversal learning data
#exclude yellow tube trials for control birds because we are only interested in reversal data
dflex <- subset(dflex, dflex$Reversal != "Control: Yellow Tube" & dflex$ID !="Memela")
#include only those trials where the bird made a choice (0 or 1)
dflex <- subset(dflex, dflex$CorrectChoice != -1)
#reverse number. 0=initial discrimination
dflex$Reversal <- as.integer(dflex$Reversal)
dflex$Correct <- as.integer(dflex$CorrectChoice)
dflex$Trial <- as.integer(dflex$Trial)
#exclude NAs from the CorrectChoice column
dflex <- subset(dflex, is.na(dflex$Correct) == FALSE)
# Want data ONLY from initial learning and first reversal to determine phi and lambda at the beginning. This is for all birds, including those that did not experience the reversal manipulation experiment
reduceddata <- matrix(ncol=ncol(dflex),nrow=0)
reduceddata <- data.frame(reduceddata)
for (i in 1:length(unique(dflex$ID))) {
thisbird <- unique(dflex$ID)[i]
thisbirddata <- dflex[dflex$ID==thisbird,]
thisbirdslastreversal <- thisbirddata[thisbirddata$Reversal %in% c(0,1),]
reduceddata <- rbind(reduceddata,thisbirdslastreversal)
}
dflex_beginning <- reduceddata
# We want to remove the birds who did not go through at least the first reversal trial
birdscompletedreversal<-unique(dflex_beginning[dflex_beginning$Reversal==1,]$ID)
dflex_beginning<-dflex_beginning[dflex_beginning$ID %in% birdscompletedreversal,]
length(unique(dflex_beginning$ID)) #21 birds
#Construct Choice variable
dflex_beginning$Choice <- NA
for (i in 1: nrow(dflex_beginning)) {
if (dflex_beginning$Reversal[i] %in% seq(0, max(unique(dflex_beginning$Reversal)), by = 2)){
if (dflex_beginning$Correct[i] == 1){
dflex_beginning$Choice[i] <- 1
} else {
dflex_beginning$Choice[i] <- 2
}
} else {
if (dflex_beginning$Correct[i] == 1){
dflex_beginning$Choice[i] <- 2
} else {
dflex_beginning$Choice[i] <- 1
}
}
}
dflex_beginning <- dflex_beginning[with(dflex_beginning, order(dflex_beginning$ID)), ]
colnames(dflex_beginning)[4]<-"id"
# Sort birds alphabetically
dflex_beginning <- dflex_beginning[with(dflex_beginning, order(dflex_beginning$id)), ]
birdnames<-unique(dflex_beginning$id)
# Convert bird names into numeric ids
dflex_beginning$id <- as.numeric(as.factor(dflex_beginning$id))
datinitialandfirstreversal <- as.list(dflex_beginning)
datinitialandfirstreversal$N <- nrow(dflex_beginning)
datinitialandfirstreversal$N_id <- length(unique(dflex_beginning$id))
reinforcement_model_nonzeroattraction <- "
data{
int N;
int N_id;
int id[N];
int Trial[N];
int Choice[N];
int Correct[N];
}
parameters{
real logit_phi;
real log_L;
// Varying effects clustered on individual
matrix[2,N_id] z_ID;
vector<lower=0>[2] sigma_ID; //SD of parameters among individuals
cholesky_factor_corr[2] Rho_ID;
}
transformed parameters{
matrix[N_id,2] v_ID; // varying effects on stuff
v_ID = ( diag_pre_multiply( sigma_ID , Rho_ID ) * z_ID )';
}
model{
matrix[N_id,2] A; // attraction matrix
logit_phi ~ normal(0,1);
log_L ~ normal(0,1);
// varying effects
to_vector(z_ID) ~ normal(0,1);
sigma_ID ~ exponential(1);
Rho_ID ~ lkj_corr_cholesky(4);
// initialize attraction scores
for ( i in 1:N_id ) {
A[i,1] = 0.1; A[i,2] = 0.1';
}
// loop over Choices
for ( i in 1:N ) {
vector[2] pay;
vector[2] p;
real L;
real phi;
// first, what is log-prob of observed choice
L = exp(log_L + v_ID[id[i],1]);
p = softmax(L*A[id[i],1:2]' );
Choice[i] ~ categorical( p );
// second, update attractions conditional on observed choice
phi = inv_logit(logit_phi + v_ID[id[i],2]);
pay[1:2] = rep_vector(0,2);
pay[ Choice[i] ] = Correct[i];
A[ id[i] , Choice[i] ] = ( (1-phi)*(A[ id[i] , Choice[i] ]) + phi*pay[Choice[i]])';
}//i
}
"
m_initialandreversal <- stan( model_code = reinforcement_model_nonzeroattraction, data=datinitialandfirstreversal ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
sinitialandreversal <- extract.samples(m_initialandreversal)
initialandreversal_lambda <- sapply(1 : datinitialandfirstreversal$N_id, function(x) exp( mean(sinitialandreversal$log_L) + mean(sinitialandreversal$v_ID[ ,x, 1])))
initialandreversal_phi <- sapply(1 : datinitialandfirstreversal$N_id, function(x) inv_logit( mean(sinitialandreversal$logit_phi) + mean(sinitialandreversal$v_ID[ ,x, 2])))
plot(initialandreversal_phi~initialandreversal_lambda)
# Next, for comparison, want data ONLY from last two reversal trials to determine phi and lambda at the end. This is for the manipulated birds only because the control group only went through a single reversal.
# Need to do the analysis for the last two reversals with the skewed priors for the attraction values for the manipulated birds.
# link manipulatedbirdids to birdnames
dflex_last_manipulated<- dflex[dflex$ID=="Chalupa" | dflex$ID=="Mole" | dflex$ID=="Habanero" | dflex$ID=="Diablo" | dflex$ID=="Burrito" | dflex$ID=="Adobo" | dflex$ID=="Chilaquile" | dflex$ID=="Pollito" | dflex$ID=="Memela",]
colnames(dflex_last_manipulated)[4]<-"id"
# Sort birds alphabetically
dflex_last_manipulated <- dflex_last_manipulated[with(dflex_last_manipulated, order(dflex_last_manipulated$id)), ]
birdnames_manipulated<-unique(dflex_last_manipulated$id)
# Convert bird names into numeric ids
dflex_last_manipulated$id <- as.numeric(as.factor(dflex_last_manipulated$id))
length(unique(dflex_last_manipulated$id)) #8 birds
#Construct Choice variable
dflex_last_manipulated$Choice <- NA
for (i in 1: nrow(dflex_last_manipulated)) {
if (dflex_last_manipulated$Reversal[i] %in% seq(0, max(unique(dflex_last_manipulated$Reversal)), by = 2)){
if (dflex_last_manipulated$Correct[i] == 1){
dflex_last_manipulated$Choice[i] <- 1
} else {
dflex_last_manipulated$Choice[i] <- 2
}
} else {
if (dflex_last_manipulated$Correct[i] == 1){
dflex_last_manipulated$Choice[i] <- 2
} else {
dflex_last_manipulated$Choice[i] <- 1
}
}
}
# Want data ONLY from last two reversals to determine phi and lambda at the beginning. This is for all birds, including those that did not experience the reversal manipulation experiment
reduceddata <- matrix(ncol=ncol(dflex),nrow=0)
reduceddata <- data.frame(reduceddata)
for (i in 1:length(unique(dflex_last_manipulated$id))) {
thisbird <- unique(dflex_last_manipulated$id)[i]
thisbirddata <- dflex_last_manipulated[dflex_last_manipulated$id==thisbird,]
thisbirdslastreversal <- thisbirddata[thisbirddata$Reversal %in% c(max(thisbirddata$Reversal)-1,max(thisbirddata$Reversal)),]
reduceddata <- rbind(reduceddata,thisbirdslastreversal)
}
dflex_last_manipulated <- reduceddata
datlasterversalsskewed <- as.list(dflex_last_manipulated)
datlasterversalsskewed$N <- nrow(dflex_last_manipulated)
datlasterversalsskewed$N_id <- length(unique(dflex_last_manipulated$id))
The STAN model is set up to have the inital attraction for each option set to 0.1, and that individuals only learn the reward of the option they chose in a given trial.
reinforcement_model_nonzeroattraction_skewedpriorattraction <- "
data{
int N;
int N_id;
int id[N];
int Trial[N];
int Choice[N];
int Correct[N];
}
parameters{
real logit_phi;
real log_L;
// Varying effects clustered on individual
matrix[2,N_id] z_ID;
vector<lower=0>[2] sigma_ID; //SD of parameters among individuals
cholesky_factor_corr[2] Rho_ID;
}
transformed parameters{
matrix[N_id,2] v_ID; // varying effects on stuff
v_ID = ( diag_pre_multiply( sigma_ID , Rho_ID ) * z_ID )';
}
model{
matrix[N_id,2] A; // attraction matrix
logit_phi ~ normal(0,1);
log_L ~ normal(0,1);
// varying effects
to_vector(z_ID) ~ normal(0,1);
sigma_ID ~ exponential(1);
Rho_ID ~ lkj_corr_cholesky(4);
// initialize attraction scores
for ( i in 1:N_id ) {
A[i,1] = 0.7; A[i,2] = 0.1';
}
// loop over Choices
for ( i in 1:N ) {
vector[2] pay;
vector[2] p;
real L;
real phi;
// first, what is log-prob of observed choice
L = exp(log_L + v_ID[id[i],1]);
p = softmax(L*A[id[i],1:2]' );
Choice[i] ~ categorical( p );
// second, update attractions conditional on observed choice
phi = inv_logit(logit_phi + v_ID[id[i],2]);
pay[1:2] = rep_vector(0,2);
pay[ Choice[i] ] = Correct[i];
A[ id[i] , Choice[i] ] = ( (1-phi)*(A[ id[i] , Choice[i] ]) + phi*pay[Choice[i]])';
}//i
}
"
m_lastreversals_skewed <- stan( model_code = reinforcement_model_nonzeroattraction_skewedpriorattraction, data=datlasterversalsskewed ,iter = 5000, cores = 4, chains=4, control = list(adapt_delta=0.9, max_treedepth = 12))
slastreversals_skewed <- extract.samples(m_lastreversals_skewed)
lastreversals_lambda_skewed <- sapply(1 : datlasterversalsskewed$N_id, function(x) exp( mean(slastreversals_skewed$log_L) + mean(slastreversals_skewed$v_ID[ ,x, 1])))
lastreversals_phi_skewed <- sapply(1 : datlasterversalsskewed$N_id, function(x) inv_logit( mean(slastreversals_skewed$logit_phi) + mean(slastreversals_skewed$v_ID[ ,x, 2])))
# We can now combine the information on the estimated phis and lambdas for the initial performance of all birds and the last performance of the manipulated birds into a single table
eachbirdslearningparameters<-matrix(nrow=datinitialandfirstreversal$N_id,ncol=8)
eachbirdslearningparameters<-data.frame(eachbirdslearningparameters)
colnames(eachbirdslearningparameters)<-c("Bird","Number","beginningphi","beginninglambda","manipulatedphi","manipulatedlambda","lastphi","lastlambda")
eachbirdslearningparameters[,1]<-birdnames
eachbirdslearningparameters[,2]<-unique(dflex_beginning$id)
eachbirdslearningparameters[,3]<-initialandreversal_phi
eachbirdslearningparameters[,4]<-initialandreversal_lambda
eachbirdslearningparameters[eachbirdslearningparameters$Bird %in% birdnames_manipulated,5]<-lastreversals_phi_skewed
eachbirdslearningparameters[eachbirdslearningparameters$Bird %in% birdnames_manipulated,6]<-lastreversals_lambda_skewed
for(i in 1:nrow(eachbirdslearningparameters)){
if(is.na(eachbirdslearningparameters[i,]$manipulatedphi)==T) {
eachbirdslearningparameters[i,]$lastphi<- eachbirdslearningparameters[i,]$beginningphi
eachbirdslearningparameters[i,]$lastlambda<- eachbirdslearningparameters[i,]$beginninglambda}
if(is.na(eachbirdslearningparameters[i,]$manipulatedphi)==F) {
eachbirdslearningparameters[i,]$lastphi<- eachbirdslearningparameters[i,]$manipulatedphi
eachbirdslearningparameters[i,]$lastlambda<- eachbirdslearningparameters[i,]$manipulatedlambda}
}
write.csv(eachbirdslearningparameters,file="g_flexmanip_ArizonaBirds_EstimatedPhiLambdaReversalLearning.csv")
#################
# Is there a linear improvement?
performanceimprovement <- matrix(ncol=10,nrow=length(unique(dflex$ID)))
performanceimprovement<-as.data.frame(performanceimprovement)
colnames(performanceimprovement)<-c("Bird","initialassociation", "reversal1","reversal2","reversal3","reversal4","reversal5","reversal6","reversal7","reversal8")