Permalink
Cannot retrieve contributors at this time
Fetching contributors…
Cannot retrieve contributors at this time
| <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.1d1 20130915//EN" "JATS-archivearticle1.dtd"><article article-type="research-article" dtd-version="1.1d1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><front><journal-meta><journal-id journal-id-type="nlm-ta">elife</journal-id><journal-id journal-id-type="hwp">eLife</journal-id><journal-id journal-id-type="publisher-id">eLife</journal-id><journal-title-group><journal-title>eLife</journal-title></journal-title-group><issn publication-format="electronic">2050-084X</issn><publisher><publisher-name>eLife Sciences Publications, Ltd</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="publisher-id">02670</article-id><article-id pub-id-type="doi">10.7554/eLife.02670</article-id><article-categories><subj-group subj-group-type="display-channel"><subject>Research article</subject></subj-group><subj-group subj-group-type="heading"><subject>Neuroscience</subject></subj-group></article-categories><title-group><article-title>Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex</article-title></title-group><contrib-group><contrib contrib-type="author" id="author-12056"><name><surname>Gu</surname><given-names>Yong</given-names></name><xref ref-type="aff" rid="aff1"/><xref ref-type="other" rid="par-3"/><xref ref-type="other" rid="par-4"/><xref ref-type="fn" rid="con1"/><xref ref-type="fn" rid="conf2"/></contrib><contrib contrib-type="author" equal-contrib="yes" id="author-1039"><name><surname>Angelaki</surname><given-names>Dora E</given-names></name><xref ref-type="aff" rid="aff2"/><xref ref-type="fn" rid="equal-contrib">†</xref><xref ref-type="other" rid="par-1"/><xref ref-type="fn" rid="con2"/><xref ref-type="fn" rid="conf1"/></contrib><contrib contrib-type="author" corresp="yes" equal-contrib="yes" id="author-11799"><name><surname>DeAngelis</surname><given-names>Gregory C</given-names></name><xref ref-type="aff" rid="aff3"/><xref ref-type="corresp" rid="cor1">*</xref><xref ref-type="fn" rid="equal-contrib">†</xref><xref ref-type="other" rid="par-2"/><xref ref-type="fn" rid="con3"/><xref ref-type="fn" rid="conf2"/></contrib><aff id="aff1"><institution content-type="dept">Institute of Neuroscience and Key Laboratory of Primate Neurobiology</institution>, <institution>Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences</institution>, <addr-line><named-content content-type="city">Shanghai</named-content></addr-line>, <country>China</country></aff><aff id="aff2"><institution content-type="dept">Department of Neuroscience</institution>, <institution>Baylor College of Medicine</institution>, <addr-line><named-content content-type="city">Houston</named-content></addr-line>, <country>United States</country></aff><aff id="aff3"><institution content-type="dept">Department of Brain and Cognitive Sciences</institution>, <institution>University of Rochester</institution>, <addr-line><named-content content-type="city">New York</named-content></addr-line>, <country>United States</country></aff></contrib-group><contrib-group content-type="section"><contrib contrib-type="editor"><name><surname>Romo</surname><given-names>Ranulfo</given-names></name><role>Reviewing editor</role><aff><institution>Universidad Nacional Autonoma de Mexico</institution>, <country>Mexico</country></aff></contrib></contrib-group><author-notes><corresp id="cor1"><label>*</label>For correspondence: <email>gdeangelis@cvs.rochester.edu</email></corresp><fn fn-type="con" id="equal-contrib"><label>†</label><p>These authors contributed equally to this work</p></fn></author-notes><pub-date date-type="pub" publication-format="electronic"><day>01</day><month>07</month><year>2014</year></pub-date><pub-date pub-type="collection"><year>2014</year></pub-date><volume>3</volume><elocation-id>e02670</elocation-id><history><date date-type="received"><day>27</day><month>02</month><year>2014</year></date><date date-type="accepted"><day>01</day><month>07</month><year>2014</year></date></history><permissions><copyright-statement>© 2014, Gu et al</copyright-statement><copyright-year>2014</copyright-year><copyright-holder>Gu et al</copyright-holder><license xlink:href="http://creativecommons.org/licenses/by/4.0/"><license-p>This article is distributed under the terms of the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License</ext-link>, which permits unrestricted use and redistribution provided that the original author and source are credited.</license-p></license></permissions><self-uri content-type="pdf" xlink:href="elife02670.pdf"/><abstract><object-id pub-id-type="doi">10.7554/eLife.02670.001</object-id><p>Trial by trial covariations between neural activity and perceptual decisions (quantified by choice Probability, CP) have been used to probe the contribution of sensory neurons to perceptual decisions. CPs are thought to be determined by both selective decoding of neural activity and by the structure of correlated noise among neurons, but the respective roles of these factors in creating CPs have been controversial. We used biologically-constrained simulations to explore this issue, taking advantage of a peculiar pattern of CPs exhibited by multisensory neurons in area MSTd that represent self-motion. Although models that relied on correlated noise or selective decoding could both account for the peculiar pattern of CPs, predictions of the selective decoding model were substantially more consistent with various features of the neural and behavioral data. While correlated noise is essential to observe CPs, our findings suggest that selective decoding of neuronal signals also plays important roles.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.001">http://dx.doi.org/10.7554/eLife.02670.001</ext-link></p></abstract><abstract abstract-type="executive-summary"><object-id pub-id-type="doi">10.7554/eLife.02670.002</object-id><title>eLife digest</title><p>Even the simplest tasks require the brain to process vast amounts of information. To take a step forward, for example, the brain must process information about the orientation of the animal's body and what the animal is seeing, hearing and feeling in order to determine whether any obstacles stand in the way. The brain must integrate all this information to make decisions about how to proceed. And once a decision is made, the brain must send signals via the nervous system to the muscles to physically move the foot forward.</p><p>Specialized brain cells called sensory neurons help to process this sensory information. For example, visual neurons process information about what the animal sees, while auditory neurons process information about what it hears. Other sensory neurons—called multisensory neurons—can process information coming from more than one of an animal's senses.</p><p>For more than two decades, researchers have known that the firing of an individual sensory neuron can be linked to the decision that an animal makes about the meaning of the sensory information it has received. The ability to predict whether an animal will make a given decision based on the firing of individual sensory neurons is often referred to as a ‘choice probability’. Measurements of single neurons have often been used to try to work out how the brain decodes the sensory information that is needed to carry out a specific task. However, it remains unclear whether choice probabilities really reflect how sensory information is decoded in the brain, or whether these measurements are just reflecting coordinated patterns of background ‘noise’ among the neurons as the decisions are being made.</p><p>Gu et al. set out to help resolve this debate by examining choice probabilities in the multisensory neurons in one area of the brain. A series of experiments was conducted to see how these neurons process information, both from the eyes and the part of the inner ear that helps control balance, to work out the direction in which an animal was moving. By performing computer simulations of the activity of groups of neurons, Gu et al. found that choice probability measurements are better explained by the models whereby these measurements did reflect the strategy that is used to decode the sensory information. Models based solely on patterns of correlated noise did not explain the data as well, though Gu et al. suggest that this noise is likely to also contribute to the observed effects.</p><p>Following on from the work of Gu et al., a major challenge will be to see if it is possible to infer how the brain extracts the relevant information from the different sensory neurons. This may require recordings from large groups of neurons, but it might help us to decipher how patterns of activity in the brain lead to decisions about the world around us.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.002">http://dx.doi.org/10.7554/eLife.02670.002</ext-link></p></abstract><kwd-group kwd-group-type="author-keywords"><title>Author keywords</title><kwd>macaque</kwd><kwd>neural coding</kwd><kwd>decision</kwd><kwd>computational</kwd></kwd-group><kwd-group kwd-group-type="research-organism"><title>Research organism</title><kwd>other</kwd></kwd-group><funding-group><award-group id="par-1"><funding-source><institution-wrap><institution-id institution-id-type="FundRef">http://dx.doi.org/10.13039/100000002</institution-id><institution>National Institutes of Health</institution></institution-wrap></funding-source><award-id>EY017866</award-id><principal-award-recipient><name><surname>Angelaki</surname><given-names>Dora E</given-names></name></principal-award-recipient></award-group><award-group id="par-2"><funding-source><institution-wrap><institution-id institution-id-type="FundRef">http://dx.doi.org/10.13039/100000002</institution-id><institution>National Institutes of Health</institution></institution-wrap></funding-source><award-id>EY016178</award-id><principal-award-recipient><name><surname>DeAngelis</surname><given-names>Gregory C</given-names></name></principal-award-recipient></award-group><award-group id="par-3"><funding-source><institution-wrap><institution>Recruitment Program of Global Youth Experts</institution></institution-wrap></funding-source><principal-award-recipient><name><surname>Gu</surname><given-names>Yong</given-names></name></principal-award-recipient></award-group><award-group id="par-4"><funding-source><institution-wrap><institution>Shanghai Pujiang Program</institution></institution-wrap></funding-source><award-id>13PJ1409400</award-id><principal-award-recipient><name><surname>Gu</surname><given-names>Yong</given-names></name></principal-award-recipient></award-group><funding-statement>The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.</funding-statement></funding-group><custom-meta-group><custom-meta><meta-name>elife-xml-version</meta-name><meta-value>2</meta-value></custom-meta><custom-meta specific-use="meta-only"><meta-name>Author impact statement</meta-name><meta-value>Addressing the controversy regarding the interpretation of choice probabilities demonstrates that both readout strategy and noise correlations contribute to the link between neural activity and perceptual decisions.</meta-value></custom-meta></custom-meta-group></article-meta></front><body><sec id="s1" sec-type="intro"><title>Introduction</title><p>In most sensory systems, neurons encode sensory stimuli by responding selectively to a particular range of stimulus parameters, as typically characterized by tuning curves (<xref ref-type="bibr" rid="bib12">Dayan and Abbott, 2001</xref>). In turn, the pattern of activation across a population of such neurons provides information about the most likely stimulus that may have occurred (<xref ref-type="bibr" rid="bib12">Dayan and Abbott, 2001</xref>). Whether or not a sensory neuron contributes to perceptual decisions generally depends on whether that neuron is selective to the stimulus dimensions relevant to the task at hand, and how much weight is given to the activity of that neuron in population decoding. One method for assessing the potential contribution of a sensory neuron to perception involves measuring the trial-by-trial covariation between neural activity and perceptual decisions, as typically quantified by computing the choice probability (CP) (<xref ref-type="bibr" rid="bib6">Britten et al., 1996</xref>; <xref ref-type="bibr" rid="bib13">Dodd et al., 2001</xref>; <xref ref-type="bibr" rid="bib36">Uka and DeAngelis, 2004</xref>; <xref ref-type="bibr" rid="bib31">Purushothaman and Bradley, 2005</xref>; <xref ref-type="bibr" rid="bib17">Gu et al., 2007</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>; <xref ref-type="bibr" rid="bib26">Nienborg and Cumming, 2009</xref>, <xref ref-type="bibr" rid="bib27">2010</xref>; <xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>).</p><p>When a sensory neuron shows a significant CP, there is a stereotypical relationship between response, tuning, and choice: neurons tend to respond more strongly when the subject reports perceiving the stimulus as having a value that is more preferred by the neuron. Tested properly, such effects are typically found to be independent of the stimulus value itself (<xref ref-type="bibr" rid="bib6">Britten et al., 1996</xref>; <xref ref-type="bibr" rid="bib36">Uka and DeAngelis, 2004</xref>). While the phenomenology is rather consistent across many studies, the interpretation of CPs has remained controversial (<xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>; <xref ref-type="bibr" rid="bib11">Cohen and Kohn, 2011</xref>). Some studies have suggested that the pattern of CPs across a population of neurons can provide insight into how responses of neurons with different tuning properties are selectively weighted in the decision process, that is selective decoding (<xref ref-type="bibr" rid="bib6">Britten et al., 1996</xref>; <xref ref-type="bibr" rid="bib36">Uka and DeAngelis, 2004</xref>; <xref ref-type="bibr" rid="bib31">Purushothaman and Bradley, 2005</xref>; <xref ref-type="bibr" rid="bib17">Gu et al., 2007</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>). Other studies have pointed out that correlated noise among neurons is necessary to observe significant CPs in large populations, suggesting that CPs are dominated by correlated noise and may not carry any useful information about decoding strategy (<xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>; <xref ref-type="bibr" rid="bib11">Cohen and Kohn, 2011</xref>; <xref ref-type="bibr" rid="bib28">Nienborg et al., 2012</xref>). As an extreme example, neurons that are not involved in the decision process can exhibit significant CPs solely through correlations with other neurons that do contribute (<xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>). Thus, a critical issue is whether CPs can reflect selective decoding of sensory neurons.</p><p>A recent theoretical study potentially unifies these divergent perspectives (<xref ref-type="bibr" rid="bib22">Haefner et al., 2013</xref>), demonstrating mathematically that CPs could reflect both the structure of correlated noise and selective decoding of neurons. However, experimental evidence that can dissociate these causes has been lacking. Here, we take advantage of a peculiar pattern of CPs exhibited by multisensory neurons that represent translational self-motion (i.e., heading). Some neurons in areas MSTd (<xref ref-type="bibr" rid="bib19">Gu et al., 2006</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>) and VIP (<xref ref-type="bibr" rid="bib9">Chen et al., 2013</xref>) have matched heading preferences in response to visual and vestibular stimuli (‘congruent’ cells), whereas others prefer widely disparate headings (‘opposite’ cells). Opposite cells could be decoded such that they provide evidence in favor of either their visual or their vestibular heading preference. We showed previously that congruent and opposite cells have CPs with opposite polarities in a visual heading discrimination task (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>; <xref ref-type="bibr" rid="bib9">Chen et al., 2013</xref>), and we suggested that this may result from selectively decoding both congruent and opposite cells according to their vestibular heading preferences (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>). This system provides a valuable test bed for exploring the roles of noise correlations and selective decoding in producing CPs.</p><p>Using simulations, we explore whether the peculiar pattern of CPs exhibited by multisensory neurons can be explained solely by correlated noise or whether selective decoding is also involved. Our results suggest that selective decoding can play important roles in shaping the pattern of CPs across a population of sensory neurons.</p></sec><sec id="s2" sec-type="results"><title>Results</title><p>We explore how selective decoding and correlated noise contribute to choice probabilities (CPs) in three stages. First, we consider a population of hypothetical neurons that represent a stimulus feature, such as heading, based on a single sensory modality (e.g. visual motion). Second, we extend this simplified population model to the multisensory case, in which a second sensory cue (e.g., vestibular) also provides information about the stimulus feature, and we consider the predictions that arise for choice probabilities in neurons with mismatched tuning for the two sensory modalities (opposite cells). Finally, we apply our analyses to models that are based more closely on data from neurons in area MSTd, and we compare the predictions of selective decoding and pure correlation models with the animals' behavior, as well as the structure of correlated noise among neurons.</p><sec id="s2-1"><title>Noise correlations and readout for the single modality case</title><p>To study how correlated noise and selective decoding affect the CPs of sensory neurons, we first considered a simple model in which only one sensory cue (e.g., visual) was involved. We generated a population of 1000 hypothetical neurons with cosine tuning for heading. All neurons in this population had tuning curves with the same amplitude and width, but differed in their heading preferences. For simplicity, half of the neurons preferred leftward heading (−90°) while the other half preferred rightward heading (+90°) so that tuning curves for any pair of neurons had either identical slopes or opposite slopes around a straight-forward heading reference. Other distributions of heading preferences (e.g., uniform or bimodal) did not substantially alter our conclusions. We used a maximum likelihood decoder (<xref ref-type="bibr" rid="bib33">Sanger, 1996</xref>; <xref ref-type="bibr" rid="bib12">Dayan and Abbott, 2001</xref>; <xref ref-type="bibr" rid="bib23">Jazayeri and Movshon, 2006</xref>; <xref ref-type="bibr" rid="bib20">Gu et al., 2010</xref>; <xref ref-type="bibr" rid="bib16">Fetsch et al., 2011</xref>) to estimate heading from simulated population activity, and we required the population activity to discriminate between headings that were slightly leftward or rightward relative to a straight-forward reference heading. Specifically, a likelihood function over heading was computed from the population activity on each trial. The decoder then made a ‘leftward’ choice if the area under the likelihood function for leftward headings exceeded the area under the curve for rightward headings, and vice versa for a ‘rightward’ choice. We then computed each model neuron's CP for the ambiguous stimulus condition (i.e., straight forward motion, 0°, ‘Materials and methods’), and we explored how correlated noise and selective decoding affected CPs.</p><p>As a prelude to considering the multisensory situation, we consider two extreme cases in which CPs of a group of neurons are driven mainly by correlated noise or by selective decoding. In both schemes, structured noise correlations are necessary to observe significant CPs, but the models differ in terms of which pools of neurons are correlated and how they are decoded. For the ‘pure-correlation’ model (<xref ref-type="fig" rid="fig1">Figure 1A,B</xref>), only correlated noise is needed to produce CPs that are significantly different from the chance level of 0.5. In this model, we divided the population of neurons into two groups, each of which contained an equal number of neurons preferring leftward and rightward headings. The first group of neurons (pool 1 in <xref ref-type="fig" rid="fig1">Figure 1A</xref>) contributed to the decoder's heading report (decoding weight = 1), while responses from the other group (pool 2) were ignored by the decoder (decoding weight = 0). We then examined the CPs of pool 2 neurons as a function of their correlations with pool 1. Although the signals from pool 2 neurons did not contribute to the decoder output, they still exhibited significant CPs as long as their noise was correlated with that of pool 1 neurons (<xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>).<fig id="fig1" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.003</object-id><label>Figure 1.</label><caption><title>Comparison of models in which choice probabilities (CPs) arise through either correlated noise or selective decoding.</title><p>Each model consists of two pools of neurons (500 neurons each) with equal numbers of neurons that prefer leftward and rightward headings. In the 'pure-correlation' model (<bold>A</bold> and <bold>B</bold>), neurons in pool 2 make no contribution to the decision and activity within or across pools is correlated according to the relationship illustrated in panel <bold>B</bold>. In the 'selective decoding' model (<bold>C</bold> and <bold>D</bold>), neurons shared correlated noise within each pool but not across pools. Neurons in pool 1 were always given a decoding weight of 1, while neurons in pool 2 were given weights ranging from 0 to 1. Solid curves in <bold>D</bold>: responses of pool 2 were decoded according to each neuron's preferred stimulus; dashed curves: pool 2 responses were decoded relative to each neuron's anti-preferred stimulus. Dashed black horizontal line: CP = 0.5.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.003">http://dx.doi.org/10.7554/eLife.02670.003</ext-link></p></caption><graphic xlink:href="elife02670f001"/></fig></p><p>We introduced correlated noise having a structure that is based on experimental observations from heading-selective neurons in both cortical (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>; <xref ref-type="bibr" rid="bib9">Chen et al., 2013</xref>) and subcortical (<xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>) areas. Specifically we assumed that noise correlations among pairs of model neurons are a linear function of their tuning similarity, or signal correlation (<italic>r</italic><sub><italic>signal</italic></sub>, <xref ref-type="fig" rid="fig1">Figure 1B</xref>, insets). In this correlation structure, neurons with similar heading tuning (positive <italic>r</italic><sub><italic>signal</italic></sub>) generally have positive noise correlations, whereas neurons with dissimilar tuning (negative <italic>r</italic><sub><italic>signal</italic></sub>) tend to have negative noise correlations. This correlation structure was applied to all pairs of neurons, regardless of whether they were from pool 1 or pool 2. For the simulations, the average noise correlation across all neurons was close to zero, consistent with some previous findings (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>), but the results are not sensitive to this mean value (<xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>). Rather, the critical factor is the slope of the relationship between r<sub>noise</sub> and <italic>r</italic><sub><italic>signal</italic></sub> (<xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>): larger slopes lead to greater CPs among pool 2 neurons (<xref ref-type="fig" rid="fig1">Figure 1B</xref>). Hence, in this pure-correlation model, CPs of pool 2 neurons are driven exclusively through correlations with neurons in pool 1 that contribute to the decision process (<xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>; <xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>).</p><p>For the ‘selective decoding’ model, noise among neurons within each pool was correlated in the same manner as described above, but there were no correlations between neurons in different pools (<xref ref-type="fig" rid="fig1">Figure 1C</xref>). In this case, significant CPs for pool 2 neurons require that these neurons make a contribution to the decision (<xref ref-type="fig" rid="fig1">Figure 1D</xref>). We manipulated two aspects of the contribution of each pool 2 neuron to the decoder output: magnitude and polarity. The magnitude reflects how strongly each neuron's activity influenced the decoder and was implemented mathematically by multiplying each neuron's contribution by a value in the range [0–1]. If the weight value is 1, then pool 2 neurons contribute equally as pool 1 neurons. As the weight is reduced toward zero, the contribution of pool 2 neurons diminishes and eventually is eliminated. The polarity (or sign) of the weight determines whether each neuron provides the decoder with evidence for (positive polarity) or against (negative polarity) its heading preference. It may appear counterintuitive to consider that neurons might provide evidence against their stimulus preference, but the need to consider this case will arise later in the multisensory version of the model due to the presence of opposite cells. These neurons have different heading preferences for the two sensory modalities, so they can provide evidence in favor of their stimulus preference for one modality or the other.</p><p>In this selective decoding model, the magnitude of CP increases with the weight applied to pool 2 neurons. In addition, whether the CP value is greater or less than 0.5 depends on the polarity of the contribution of pool 2 neurons (<xref ref-type="fig" rid="fig1">Figure 1D</xref>). Interpreting responses as evidence in favor of the preferred heading produces CP >0.5 (solid curves) while decoding responses as evidence against the preferred heading leads to CP <0.5 (dashed curves). Hence, the two models generate CPs for pool 2 neurons through different mechanisms. In the pure-correlation model, CPs of pool 2 neurons are produced through correlations with pool 1 neurons that are involved in the decision process (<xref ref-type="fig" rid="fig1">Figure 1B</xref>). In the selective decoding model, pool 2 neurons have CPs that depend on how strongly they contribute to the decision, as well as the polarity of the contribution of each neuron to the decision (<xref ref-type="fig" rid="fig1">Figure 1D</xref>).</p></sec><sec id="s2-2"><title>Noise correlations and readout for the multiple modality case</title><p>We next consider a more complicated case in which two different sensory cues are involved in a perceptual decision. For example, both visual (optic flow) and vestibular signals provide information about the direction of self-motion, or heading (<xref ref-type="bibr" rid="bib1">Angelaki and Cullen, 2008</xref>; <xref ref-type="bibr" rid="bib5">Britten, 2008</xref>). Previous studies have reported that neurons in multiple cortical areas (e.g., MSTd, VIP, VPS) are tuned for heading, and tend to prefer either the same or opposite headings defined by optic flow and vestibular cues (<xref ref-type="bibr" rid="bib29">Page and Duffy, 2003</xref>; <xref ref-type="bibr" rid="bib19">Gu et al., 2006</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>; <xref ref-type="bibr" rid="bib7">Chen et al., 2011a</xref>, <xref ref-type="bibr" rid="bib8">2011b</xref>). We refer to these as congruent cells and opposite cells, respectively (<xref ref-type="fig" rid="fig2">Figure 2A</xref>). For opposite cells, the preferred heading is different for the two sensory modalities, thus raising the fundamental question of how these cells may be decoded. In a multimodal heading discrimination task (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>), we showed previously that CPs of MSTd neurons have a peculiar dependence on the congruency of visual/vestibular heading tuning (<xref ref-type="fig" rid="fig2">Figure 2B</xref>). For congruent cells (cyan symbols in <xref ref-type="fig" rid="fig2">Figure 2B</xref>), CPs were consistently >0.5 when heading judgments were based on either vestibular or visual cues. In contrast, CPs for opposite cells tended to be >0.5 in the vestibular task condition but <0.5 in the visual condition (magenta symbols in <xref ref-type="fig" rid="fig2">Figure 2B</xref>). We suggested previously (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>) that this peculiar pattern of CPs might result from decoding the responses of MSTd neurons according to their vestibular heading preferences, a form of selective decoding. In what follows, we evaluate whether this pattern of CPs is compatible with a multisensory version of either the pure correlation or selective decoding models described above.<fig-group><fig id="fig2" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.004</object-id><label>Figure 2.</label><caption><title>Responses of multisensory neurons and multisensory versions of the pure-correlation and selective-decoding models.</title><p>(<bold>A</bold>) Heading tuning curves from two example MSTd neurons measured during a fine heading discrimination task (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>): one congruent cell (left) and one opposite cell (right). Red and blue data show responses measured for the visual and vestibular conditions, respectively. (<bold>B</bold>) Choice probability as a function of congruency for MSTd neurons tested in the vestibular (left) and visual (right) conditions (adapted, with permission, from Supplement Figure 8A and Figure 6C of <xref ref-type="bibr" rid="bib18">Gu et al.(2008)</xref>; respectively). Cyan and magenta symbols denote data for congruent and opposite cells, respectively (unfilled symbols: intermediate cells). (<bold>C</bold> and <bold>D</bold>) Multisensory version of the pure-correlation model (500 model neurons in each pool). Pool 1 consists of all congruent cells (same slope tuning curves for the two cues), whereas pool 2 contains all opposite neurons. Correlated noise within or across pools depends only on the similarity of tuning for cue 1. (<bold>E</bold> and <bold>F</bold>) In the selective decoding model, neurons were correlated according to the similarity of tuning for both cues (‘Materials and methods’). This rule generated correlated noise within each pool but not between pools. Neurons in pool 1 were always given a full weight of 1 in the decoding, whereas the decoding weights of neurons in pool 2 ranged from 0 to 1 (different colors in <bold>F</bold>). Dashed black horizontal line: CP = 0.5.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.004">http://dx.doi.org/10.7554/eLife.02670.004</ext-link></p></caption><graphic xlink:href="elife02670f002"/></fig><fig id="fig2s1" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.005</object-id><label>Figure 2—figure supplement 1.</label><caption><title>Predictions from a variant of the pure-correlation model in which correlated noise depends only on signal correlations from the visual tuning curves.</title><p>This modification reverses the patterns of CPs across stimulus conditions, as compared to <xref ref-type="fig" rid="fig2">Figure 2C,D</xref>. This pattern of results is not consistent with experimental data.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.005">http://dx.doi.org/10.7554/eLife.02670.005</ext-link></p></caption><graphic xlink:href="elife02670fs001"/></fig><fig id="fig2s2" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.006</object-id><label>Figure 2—figure supplement 2.</label><caption><title>Predictions from a variant of the selective decoding model in which responses are decoded according to the visual heading tuning of each neuron, instead of the vestibular tuning.</title><p>This modification reverses the CP patterns across stimulus conditions, as compared to <xref ref-type="fig" rid="fig2">Figure 2E,F</xref>, and is inconsistent with experimental data.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.006">http://dx.doi.org/10.7554/eLife.02670.006</ext-link></p></caption><graphic xlink:href="elife02670fs002"/></fig><fig id="fig2s3" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.007</object-id><label>Figure 2—figure supplement 3.</label><caption><title>Predictions from a “hybrid” model (see text for details) in which correlated noise was assigned according to vestibular signal correlations, and heading was decoded relative to the vestibular heading tuning of each neuron.</title><p>CP patterns were roughly similar to those seen in the pure-correlation model (<xref ref-type="fig" rid="fig2">Figure 2C,D</xref>).</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.007">http://dx.doi.org/10.7554/eLife.02670.007</ext-link></p></caption><graphic xlink:href="elife02670fs003"/></fig></fig-group></p><p>We again divided 1000 model neurons into two pools of equal size, with one pool consisting of congruent cells and the other consisting of opposite cells (<xref ref-type="fig" rid="fig2">Figure 2C,E</xref>). For simplicity, these neurons again preferred either leftward or rightward headings. In the pure-correlation model, only pool 1 neurons (congruent cells) provided inputs to the decoder’s decision process, whereas pool 2 neurons (opposite cells) were given no weight. However, opposite cells can still exhibit CPs as long as they are correlated with congruent cells (<xref ref-type="fig" rid="fig2">Figure 2D</xref>). For a pair of neurons that includes an opposite cell, correlated noise could depend on the similarity of vestibular tuning between the members of the pair or on the similarity of visual tuning (or on both, as discussed further below). If correlated noise is dependent on the similarity of vestibular tuning (i.e., vestibular signal correlation), opposite cells show CPs that are >0.5 in the vestibular condition and <0.5 in the visual condition (<xref ref-type="fig" rid="fig2">Figure 2D</xref>), roughly similar to experimental data from MSTd neurons (<xref ref-type="fig" rid="fig2">Figure 2A,B</xref>). The intuition for this result is straightforward. When a congruent cell fires more spikes than average, an opposite cell that shares the same vestibular tuning will also fire more spikes than average and will have a CP >0.5 in the vestibular condition. In the visual condition, this correlation structure will lead to an opposite cell responding more when the animal chooses its non-preferred visual heading, thus producing a CP <0.5. Note that this prediction of the pure correlation model only mimics real data if opposite cells are correlated with congruent cells having matched vestibular heading preferences.</p><p>In contrast, if correlated noise depends on the similarity of visual heading preferences, then the pattern of results will be reversed for the vestibular and visual conditions (<xref ref-type="fig" rid="fig2s1">Figure 2—figure supplement 1</xref>), which is clearly inconsistent with the experimental data (<xref ref-type="fig" rid="fig2">Figure 2B</xref>). Finally, correlations between the two pools of the pure-correlation model could depend on the similarity of heading preferences for both the visual and vestibular modalities (‘Materials and methods’; <xref ref-type="disp-formula" rid="equ2">Equation 2</xref>). If correlated noise depends equally on both visual and vestibular signal correlations (i.e., <xref ref-type="disp-formula" rid="equ2">Equation 2</xref> with <italic>a</italic><sub><italic>vestibular</italic></sub> = <italic>a</italic><sub><italic>visual</italic></sub>), then correlations between opposite cells and congruent cells become effectively zero because the two terms of <xref ref-type="disp-formula" rid="equ2">Equation 2</xref> cancel. In this case, the pure-correlation model becomes equivalent to the selective decoding model with zero weight placed on opposite cells, as considered below.</p><p>In the selective decoding model, we assume that correlated noise among a pair of neurons depends on the similarity of both vestibular and visual tuning (‘Materials and methods’), as demonstrated previously for pairs of MSTd neurons (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>). Because we assume <italic>a</italic><sub><italic>vestibular</italic></sub> = <italic>a</italic><sub><italic>visual</italic></sub> in <xref ref-type="disp-formula" rid="equ2">Equation 2</xref>, the resulting effective noise correlation between mixed pairs of congruent and opposite cells will be zero (denoted by ‘ × ’ between the two pools in <xref ref-type="fig" rid="fig2">Figure 2E</xref>). Thus, use of the selective-decoding model with this particular correlation structure effectively eliminates correlated noise between the two pools while allowing the decoding weights of pool 2 to vary. Under these conditions, non-zero decoding weights must be applied to pool 2 in order to observe CPs for opposite cells (<xref ref-type="fig" rid="fig2">Figure 2F</xref>). Note that, for the selective decoding model, we do not consider situations in which correlated noise between the two pools depends only on vestibular or visual heading preferences. In such cases, there would be correlated noise between pairs of neurons drawn from the two pools, and the CPs of opposite cells would depend on both readout weights and noise correlations. Thus, the two models would no longer be conceptually distinct. We shall evaluate these assumptions in the following sections.</p><p>To examine the predictions of the selective decoding model, we again manipulated two aspects of the readout. With regard to magnitude of the decoding weights, a decoding weight different from zero was essential to produce CPs for opposite cells that were different from the chance level (green symbols/lines vs other colors in <xref ref-type="fig" rid="fig2">Figure 2F</xref>). With regard to polarity (or sign) of the decoding weights, decoding responses of opposite cells with respect to their vestibular heading preference led to CP >0.5 in the vestibular condition and <0.5 in the visual condition (<xref ref-type="fig" rid="fig2">Figure 2F</xref>), which was similar to that seen in the real data. On the other hand, if responses of model neurons were decoded with respect to their visual heading preferences, the pattern of CPs across stimulus conditions would reverse and would be incompatible with that observed for MSTd neurons (<xref ref-type="fig" rid="fig2s2">Figure 2—figure supplement 2</xref>).</p><p>Hence, both models can produce CP patterns for congruent and opposite cells that are similar to those seen in the real data, but through critically different mechanisms. In the pure-correlation model, CPs of opposite cells arise solely through correlations with congruent cells having matched vestibular tuning, even though opposite cells do not contribute directly to the decision. In the selective decoding model, there is no effective correlation between mixed pairs of congruent/opposite cells (i.e., no correlated noise between pools). In this case, to qualitatively match the pattern of CP results from real neurons, the activity of opposite cells must be given weight in the decoding and these cells must be decoded selectively according to their vestibular preferences. Another way to summarize the distinction between models is that the pure correlation model mimics experimental CP results by virtue of modality specificity in the structure of correlated noise, whereas the selective decoding model achieves this by modality specificity in the decoding weights.</p><p>For completeness, we also considered a hybrid model that combines features from both of the above models. Noise correlations were linearly dependent only on the similarity of vestibular tuning, as in the pure-correlation model. This produced correlations between the two pools (congruent and opposite cells), unlike in the selective decoding model. In addition, a readout weight was assigned to the opposite cells, as in the selective decoding model. Under these conditions, we found that the predicted patterns of CPs largely resembled those from the pure-correlation model, as if the decoding weights on pool 2 played little role (<xref ref-type="fig" rid="fig2s3">Figure 2—figure supplement 3</xref>). In the following analyses, we only considered comparisons between the pure-correlation model and the selective decoding model, as these are conceptually distinct. Although this distinction is useful for exploring the relative roles of correlated noise and selective decoding in producing CPs, we recognize that both factors may contribute.</p></sec><sec id="s2-3"><title>Comparison of model predictions with data: noise correlations</title><p>Which model best matches experimental data on correlated noise? Our previous study (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>) showed that r<sub>noise</sub> measured for pairs of MSTd neurons depended approximately equally on <italic>r</italic><sub><italic>signal</italic></sub> computed from both visual and vestibular tuning curves (<xref ref-type="fig" rid="fig3">Figure 3A</xref>). This dependence on <italic>r</italic><sub><italic>signal</italic></sub> for both modalities is not due to strong covariance between the two signal correlations because <italic>r</italic><sub><italic>signal</italic></sub> values for visual and vestibular tuning are only weakly correlated (<xref ref-type="fig" rid="fig3s1">Figure 3—figure supplement 1</xref>). Note also that noise correlations are generally negative for neurons with opposite tuning (negative signal correlations) in our heading discrimination data sets (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>; <xref ref-type="bibr" rid="bib9">Chen et al., 2013</xref>; <xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>), such that subtracting responses of oppositely tuned neurons is not expected to have the benefit seen in other systems (<xref ref-type="bibr" rid="bib32">Romo et al., 2003</xref>).<fig-group><fig id="fig3" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.008</object-id><label>Figure 3.</label><caption><title>Comparison of the structure of correlated noise between models and data.</title><p>(<bold>A</bold>) Data from pairs of neurons recorded from area MSTd (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>). Noise correlation is plotted against signal correlations obtained from the vestibular (left column) or visual (right column) tuning curves. Black and gray symbols denote pairs with matched (black) or mismatched (gray) congruency. Open symbols represent undefined pairs. (<bold>B</bold>) Predicted noise correlations as a function of signal correlation based on fits of the selective decoding model. Format as in panel <bold>A</bold>. (<bold>C</bold>) Predicted noise correlations as a function of signal correlation by for the pure-correlation model fit, for which noise correlations depend only on vestibular signal correlation.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.008">http://dx.doi.org/10.7554/eLife.02670.008</ext-link></p></caption><graphic xlink:href="elife02670f003"/></fig><fig id="fig3s1" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.009</object-id><label>Figure 3—figure supplement 1.</label><caption><title>Comparison of vestibular and visual signal correlations for 127 pairs of neurons simultaneously recorded from area MSTd by <xref ref-type="bibr" rid="bib21">Gu et al. (2011)</xref>.</title><p>There is a fairly weak correlation between the two variables (R = 0.29, p=0.001, Spearman correlation). Histograms along the top and right side show marginal distributions of the two signal correlations.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.009">http://dx.doi.org/10.7554/eLife.02670.009</ext-link></p></caption><graphic xlink:href="elife02670fs004"/></fig><fig id="fig3s2" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.010</object-id><label>Figure 3—figure supplement 2.</label><caption><title>Noise correlation structure of the pure correlation model computed from the signal correlations of all distinct pairings of 129 neurons that were recorded previously by <xref ref-type="bibr" rid="bib21">Gu et al. (2011)</xref>.</title><p>For the pure-correlation model, correlated noise is determined by the signal correlation in the vestibular condition, such that pairs of cells with matched (black) and mismatched (gray) congruency have opposite relationships between r<sub>noise</sub> and <italic>r</italic><sub><italic>signal</italic></sub> in the visual condition.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.010">http://dx.doi.org/10.7554/eLife.02670.010</ext-link></p></caption><graphic xlink:href="elife02670fs005"/></fig><fig id="fig3s3" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.011</object-id><label>Figure 3—figure supplement 3.</label><caption><title>Noise correlation structure of the selective decoding model computed from the signal correlations of all distinct pairings of 129 neurons that were recorded previously by <xref ref-type="bibr" rid="bib21">Gu et al. (2011)</xref>.</title><p>For the selective decoding model, correlated noise depends on <italic>r</italic><sub><italic>signal</italic></sub> in both stimulus conditions. As a result, the relationship between r<sub>noise</sub> and <italic>r</italic><sub><italic>signal</italic></sub> is strong for pairs with matched congruency in both stimulus conditions (black), and this relationship is weak for pairs with mismatched congruency (gray).</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.011">http://dx.doi.org/10.7554/eLife.02670.011</ext-link></p></caption><graphic xlink:href="elife02670fs006"/></fig></fig-group></p><p>Here, we have sorted these previous data (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>) into two groups according to whether the congruency of visual/vestibular tuning is matched or mismatched for the two members of each pair of neurons: ‘matched congruency’ pairs consist of either two congruent cells or two opposite cells, whereas ‘mismatched congruency’ pairs consist of one congruent cell and one opposite cell (see ‘Materials and methods’ for classification procedure). Pairs that could not be classified into either group were labeled as ‘undefined’ (open symbols, <xref ref-type="fig" rid="fig3">Figure 3A</xref>). We found that the dependence of r<sub>noise</sub> on both visual and vestibular signal correlations was weak for the mismatched congruency pairs (vestibular <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.033, CI = [−0.5–0.27], R = 0.08, p=0.8; visual <italic>r</italic><sub><italic>signal</italic></sub>: slope = −0.057, CI = [−0.424–0.258], R = −0.12, p=0.7, type II linear regression), whereas this dependence was quite robust for the matched congruency pairs (vestibular <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.192, CI = [0.082–0.288], R = 0.61, p=0.001; visual <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.186, CI = [0.069–0.28], R = 0.62, p<0.001, type II linear regression). This pattern of results is consistent with the correlation structure assumed in the selective decoding model.</p><p>To evaluate whether the experimental data significantly favor the selective decoding model over the pure correlation model, we fit the data from these 127 pairs of neurons with two correlation structures: (1) r<sub>noise</sub> depended only on vestibular <italic>r</italic><sub><italic>signal</italic></sub>, r<sub>noise</sub> = <italic>a</italic><sub><italic>vestibular</italic></sub> * <italic>r</italic><sub><italic>signal_vestibular</italic></sub>, as in the pure correlation model; (2) r<sub>noise</sub> depended on both vestibular and visual signal correlations, r<sub>noise</sub> = <italic>a</italic><sub><italic>vestibular</italic></sub> * <italic>r</italic><sub><italic>signal_vestibular</italic></sub> + <italic>a</italic><sub><italic>visual</italic></sub> *<italic>r</italic><sub><italic>signal_visual</italic></sub>, as assumed in the selective decoding model. We then used linear regression to fit the data with both correlation structures. Importantly, we found that the model with coefficients for both vestibular and visual signal correlations provided a significantly better fit to the data, after accounting for the difference in the number of parameters (p=0.0003, sequential F-test). The coefficients of the best-fitting model were <italic>a</italic><sub><italic>vestibular</italic></sub> = 0.12 and <italic>b</italic><sub><italic>visual</italic></sub> = 0.09, respectively. Thus, the empirical correlation data are significantly better fit with a correlation structure in which r<sub>noise</sub> depends on <italic>r</italic><sub><italic>signal</italic></sub> for both vestibular and visual tuning curves.</p><p>To help visualize why the second structure above better fits the data, we used the measured signal correlations for each pair of neurons, along with the fitted coefficients, to predict r<sub>noise</sub> for each pair of neurons. When r<sub>noise</sub> is predicted based on both visual and vestibular signal correlations (<xref ref-type="fig" rid="fig3">Figure 3B</xref>), the dependence of r<sub>noise</sub> on <italic>r</italic><sub><italic>signal</italic></sub> was much weaker for mismatched congruency pairs (vestibular <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.074, 95% CI = [0.044–0.109], R = 0.82, p=0.004; visual <italic>r</italic><sub><italic>signal</italic></sub>: slope = −0.01, 95% CI = [−0.08–0.066], R = −0.09, p=0.8, type II linear regression) than for matched congruency pairs (vestibular <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.19, 95% CI = [0.177–0.203], R = 0.99, p<<0.001; visual <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.18, 95% CI = [0.157–0.195], R = 0.96, p<<0.001, type II linear regression), similar to the MSTd data (<xref ref-type="fig" rid="fig3">Figure 3A</xref>). In contrast, when r<sub>noise</sub> is only dependent on vestibular <italic>r</italic><sub><italic>signal</italic></sub>, the predicted correlation structure is quite different (<xref ref-type="fig" rid="fig3">Figure 3C</xref>). While r<sub>noise</sub> is perfectly correlated with vestibular <italic>r</italic><sub><italic>signal</italic></sub> for all neurons (by assumption), the mismatched congruency pairs reveal roughly equal but opposite dependencies on vestibular and visual signal correlations (vestibular <italic>r</italic><sub><italic>signal</italic></sub>: slope = 0.16, 95% CI = [0.161–0.161], R = 1, p<<0.001; visual <italic>r</italic><sub><italic>signal</italic></sub>: slope = −0.12, 95% CI = [−0.220–0.018], R = −0.65, p=0.04, type II linear regression). Thus, the available data on noise and signal correlations compare more favorably with the assumptions of the selective decoding model than the pure correlation model.</p></sec><sec id="s2-4"><title>Comparison of model predictions with data: choice probabilities</title><p>We now evaluate whether predictions of the pure correlation and selective decoding models are compatible with choice probability data obtained from area MSTd neurons during a fine heading discrimination task (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>). The Gu et al. dataset consisted of 129 single neurons that were not recorded simultaneously. Thus, to generate model population responses for decoding analyses, we generated responses for a model population of 1000 neurons, each of which had visual and vestibular tuning curves that were obtained by drawing data (with replacement) from the sample of 129 real MSTd neurons. Responses of the 1000 model neurons on each simulated trial were generated from a covariance matrix that was based on the two different correlation structures described in the previous section, with parameters that were obtained by fits to the MSTd data. This yielded noise and signal correlations similar to those described in <xref ref-type="fig" rid="fig3">Figure 3</xref> (<xref ref-type="fig" rid="fig3s2 fig3s3">Figure 3—figure supplements 2 and 3</xref>). With these constraints, we can decode the simulated population responses (‘Materials and methods’) and make predictions of CPs and neuronal thresholds.</p><p>With our biologically-constrained versions of the pure correlation and selective decoding models, we now consider the patterns of choice probabilities predicted by each model and how they compare to data from MSTd neurons. In the pure-correlation model (<xref ref-type="fig" rid="fig4">Figure 4A</xref>), the average CP for congruent cells is significantly greater than 0.5 for both the vestibular (0.65 ± 0.06 SD) and visual (0.65 ± 0.04 SD) conditions (p<0.001, <italic>t</italic> test). For opposite cells, the average CP is significantly >0.5 in the vestibular condition (0.623 ± 0.039 SD, p<0.001, <italic>t</italic> test) and significantly <0.5 in the visual condition (0.372 ± 0.047 SD, p<0.001). This pattern of CPs is qualitatively similar to that observed for MSTd neurons (<xref ref-type="fig" rid="fig2">Figure 2B</xref>).<fig id="fig4" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.012</object-id><label>Figure 4.</label><caption><title>Predictions of choice probabilities from the two models.</title><p>(<bold>A</bold>) The pattern of CPs predicted by the pure-correlation model. Format as in <xref ref-type="fig" rid="fig2">Figure 2B</xref>. (<bold>B</bold>) A family of weighting profiles used to consider various degrees of contribution of opposite cells to the selective decoding model. Each curve shows the decoding weight as a function of congruency between visual and vestibular heading tuning. Each curve corresponds to a specific value of the Readout Index (RI). (<bold>C</bold>) Predicted average CPs from the selective decoding model for a subset of the RI values illustrated in (<bold>B</bold>). (<bold>D</bold>) The pattern of CPs across neurons in the selective decoding model for an RI value of 0.5. Cyan symbols: congruent cells; Magenta symbols: opposite cells; Unfilled symbols: intermediate cells. Solid squares: mean CP. Dashed horizontal line: CP = 0.5.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.012">http://dx.doi.org/10.7554/eLife.02670.012</ext-link></p></caption><graphic xlink:href="elife02670f004"/></fig></p><p>In the selective decoding model, we need to consider how the weight applied to opposite cells influences the pattern of CPs. However, unlike the simple hypothetical model of <xref ref-type="fig" rid="fig2">Figure 2</xref>, real neurons exhibit a range of congruencies between visual and vestibular tuning. To explore a range of possible relative weightings of opposite cells, we used a sigmoidal function (having a single parameter called the Readout Index, RI) to selectively weight the contributions of congruent and opposite cells (<xref ref-type="fig" rid="fig4">Figure 4B</xref>, ‘Materials and methods’). For RI values near 1.0, all neurons contribute equally to the decoder output, regardless of congruency. As RI declines, opposite cells are given gradually less weight in the decoding (<xref ref-type="fig" rid="fig4">Figure 4B</xref>). For small values of RI, opposite neurons in the selective decoding model show CPs that approach 0.5, as expected since these neurons are given little weight in the decision process. However, as RI increases, the average CP for opposite cells increases in the vestibular condition and decreases in the visual condition (<xref ref-type="fig" rid="fig4">Figure 4C</xref>). For RI values near 0.5 (<xref ref-type="fig" rid="fig4">Figure 4D</xref>), the pattern of CPs across the population of simulated neurons resembles that exhibited by MSTd neurons (<xref ref-type="fig" rid="fig2">Figure 2B</xref>).</p><p>Together, these results (<xref ref-type="fig" rid="fig4">Figure 4</xref>) demonstrate that both the pure correlation and selective decoding models are capable of mimicking the patterns of CPs shown by MSTd neurons in the visual and vestibular conditions (although the pure correlation model assumes a correlation structure different from that seen experimentally). We now consider whether the models make distinct predictions regarding CPs for the combined condition, in which visual and vestibular cues are presented together. As described previously (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>), when animals judge heading based on congruent combinations of visual and vestibular cues, congruent cells tend to have CPs >0.5, whereas opposite cells have CPs that cluster around the chance level of 0.5 (<xref ref-type="fig" rid="fig5">Figure 5A</xref>). Interestingly, although the average CP of opposite cells was approximately 0.5 for both models in the combined condition (pure-correlation model: 0.486 ± 0.13 SD; selective decoding model: 0.491 ± 0.06 SD), the SD of CP across the population of opposite cells was much greater for the pure-correlation model than for the selective decoding model (<xref ref-type="fig" rid="fig5">Figure 5B,C</xref>). Indeed, a non-parametric test showed that the dispersion of the CPs was significantly greater for the pure correlation model (p<<0.001, Ansari–Bradley test). The reason for this difference is apparent: CPs of opposite cells in the pure correlation model clearly have a bimodal distribution (p<sub>uni</sub><<0.001, p<sub>bi</sub>>0.05, modality test, middle column of <xref ref-type="fig" rid="fig5">Figure 5D</xref>). In contrast, CPs of opposite cells in the selective readout model have a unimodal distribution centered around 0.5 (p<sub>uni</sub>>0.05, modality test, right column in <xref ref-type="fig" rid="fig5">Figure 5D</xref>), which is similar to that seen for MSTd neurons (p<sub>uni</sub>>0.05, modality test, left column in <xref ref-type="fig" rid="fig5">Figure 5D</xref>). Thus, the pattern of CPs in the combined condition favors the selective decoding model.<fig-group><fig id="fig5" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.013</object-id><label>Figure 5.</label><caption><title>Analysis of choice probabilities for the combined condition in which both visual and vestibular heading cues are present.</title><p>(<bold>A</bold>) CP values plotted as a function of congruency index for neurons from area MSTd tested in the Combined condition (adapted, with permission, from <xref ref-type="fig" rid="fig6">Figure 6A</xref> of <xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>). (<bold>B</bold> and <bold>C</bold>) CP as a function of congruency for model neurons from the pure-correlation model, and model cells from the selective decoding model, respectively. (<bold>D</bold>) Distributions of CPs for opposite cells from area MSTd, the pure-correlation model and the selective decoding model. (<bold>E</bold>–<bold>G</bold>) CP values plotted as a function of neuronal discrimination thresholds for real MSTd neurons (adapted, with permission, from <xref ref-type="fig" rid="fig6">Figure 6B</xref> of <xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>), units from the pure-correlation model and units from the selective decoding model, respectively. (<bold>H</bold>) Correlation coefficient of the best linear fit to the relationship between CP and neuronal threshold for MSTd data (filled black circles), pure-correlation model (open black circles) and selective decoding model (open red circles). Error bars represent 95% confidence intervals.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.013">http://dx.doi.org/10.7554/eLife.02670.013</ext-link></p></caption><graphic xlink:href="elife02670f005"/></fig><fig id="fig5s1" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.014</object-id><label>Figure 5—figure supplement 1.</label><caption><title>Bimodality of CP for opposite cells in the cue combined condition.</title><p>Top panels: four examples of simulations of the pure-correlation model having different values of <italic>a</italic><sub><italic>vestibular</italic></sub> ranging from 0.01 to 0.12. Both the magnitude and bimodality of the CP distribution increases with <italic>a</italic><sub><italic>vestibular</italic></sub>. To quantify bimodality, we chose cells with Congruency Index in the range [−1–0.5], and applied K-means clustering to generate two clusters. The vertical distance, d, between the centroids of the two clusters was taken as an index of the separation of the two clusters. Bottom row: (left) Summary of d values as a function of <italic>a</italic><sub><italic>vestibular</italic></sub>. Asterisks represent significant bimodality, as assessed by a modality test (‘Materials and methods’). (right) Relationship between the CP of congruent cells (congruency index from 0.5 to 1) and <italic>a</italic><sub><italic>vestibular</italic></sub>.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.014">http://dx.doi.org/10.7554/eLife.02670.014</ext-link></p></caption><graphic xlink:href="elife02670fs007"/></fig><fig id="fig5s2" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.015</object-id><label>Figure 5—figure supplement 2.</label><caption><title>Same format as in <xref ref-type="fig" rid="fig5s1">Figure 5—figure supplement 1</xref>, but results are shown for the selective decoding model.</title><p>Top row: simulations for four different values of the Readout Index (RI). Bottom row: A bimodal distribution of CP is only seen for RI values >= 0.8, and CP of congruent cells is independent of RI.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.015">http://dx.doi.org/10.7554/eLife.02670.015</ext-link></p></caption><graphic xlink:href="elife02670fs008"/></fig></fig-group></p><p>Additional analyses suggest that this difference in the shape of the CP distribution between models is fairly robust to variations in the key model parameters. For the pure correlation model, a bimodal distribution of CPs in the combined condition is a robust result over a range of values of <italic>a</italic><sub><italic>vestibular</italic></sub>, including values much smaller than needed to fit our noise correlation data (<xref ref-type="fig" rid="fig5s1">Figure 5—figure supplement 1</xref>). For the selective decoding model, a unimodal distribution of CPs is predicted for a wide range of RI values, and significant bimodality only occurs for RI values ≥0.8, which are not consistent with behavioral thresholds as described below (<xref ref-type="fig" rid="fig5s2">Figure 5—figure supplement 2</xref>).</p><p>A possible explanation for the near-chance CPs of opposite cells in the combined condition is that the sensitivity these cells is low relative to that in the single cue conditions and thus opposite cells contribute less to the decision process (see insets in <xref ref-type="fig" rid="fig5">Figure 5A</xref>, left panel). Indeed, for real MSTd neurons, congruent cells tend to have low thresholds and high CPs whereas opposite cells tend to have high thresholds and low CPs (<xref ref-type="fig" rid="fig5">Figure 5E</xref>; R = −0.65, 95% CI: [−0.47 to −0.78], p<<0.001, linear regression). Although both models also show a similar dependency of CP on neuronal sensitivity in the combined condition (p<<0.001, linear regression, <xref ref-type="fig" rid="fig5">Figure 5F,G</xref>), the strength of the correlation is significantly weaker for the pure correlation model (R = −0.41, 95% CI: [−0.32 to −0.48]) than for the selective readout model (R = −0.67, 95% CI: [−0.61 to −0.72], <xref ref-type="fig" rid="fig5">Figure 5H</xref>). This also appears to be largely due to the bimodal distribution of CPs for opposite cells in the pure correlation model, which is not observed for the real MSTd data. Thus, while both models produce similar patterns of CPs in the vestibular and visual conditions (<xref ref-type="fig" rid="fig4">Figure 4</xref>), the pattern of CPs from the selective decoding model was more analogous to the measured data in the combined condition.</p></sec><sec id="s2-5"><title>Comparison of task performance between models and monkeys</title><p>Finally, we compute the predicted psychophysical performance by decoding data from real MSTd neurons using each readout model, and we compare the results to animals’ behavioral performance. In a fine heading discrimination task, we have previously shown that heading sensitivity in the combined condition increases when the two cues have comparable sensitivity, and that the effects are close to that predicted from optimal cue integration theory (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>; <xref ref-type="bibr" rid="bib15">Fetsch et al., 2009</xref>, <xref ref-type="bibr" rid="bib16">2011</xref>). In <xref ref-type="fig" rid="fig6">Figure 6A</xref>, we have replotted the animals' behavioral data (solid circles and dashed curve). The average psychophysical thresholds were 2.03°±0.09° (mean ± SEM) and 2.12° ± 0.1° for the vestibular and visual conditions, respectively. In the combined condition, the average threshold was reduced to 1.44° ± 0.06°, a 29% improvement compared to the best single cue, and was very close to the prediction (1.43° ± 0.06°) from optimal cue integration theory.<fig-group><fig id="fig6" position="float"><object-id pub-id-type="doi">10.7554/eLife.02670.016</object-id><label>Figure 6.</label><caption><title>Comparisons between model population threshold and psychophysical performance of the animals.</title><p>(<bold>A</bold>) Average thresholds are shown for the vestibular, visual, and combined conditions, along with the prediction from optimal cue integration theory. Data are shown for the average of two monkeys (filled symbols, dashed curve), for predictions of the pure-correlation model (black open symbols and solid curve), and for predictions of the selective decoding model with RI = 0.5 (red symbols and curve). (<bold>B</bold>) Predicted thresholds from the selective decoding model are plotted as a function of Readout Index for vestibular, visual, and combined conditions, as well as the optimal prediction from the single-cue thresholds. (<bold>C</bold>) Thresholds from the selective-decoding model (with RI = 0.5) as a function of population size. Solid curves: model predictions; dashed horizontal lines: average performance of two animals.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.016">http://dx.doi.org/10.7554/eLife.02670.016</ext-link></p></caption><graphic xlink:href="elife02670f006"/></fig><fig id="fig6s1" position="float" specific-use="child-fig"><object-id pub-id-type="doi">10.7554/eLife.02670.017</object-id><label>Figure 6—figure supplement 1.</label><caption><title>Comparison of neuronal sensitivity between the visual (ordinate) and vestibular (abscissa) stimulus conditions for 30 congruent cells.</title><p>Arrows and numbers indicate geometric mean values. Overall, visual responses were somewhat more sensitive than vestibular responses, although the animals’ behavioral performance was similar between conditions.</p><p><bold>DOI:</bold> <ext-link ext-link-type="doi" xlink:href="10.7554/eLife.02670.017">http://dx.doi.org/10.7554/eLife.02670.017</ext-link></p></caption><graphic xlink:href="elife02670fs009"/></fig></fig-group></p><p>Unlike the animals' behavior, decoder performance based on the pure-correlation model predicted mismatched thresholds for the visual and vestibular conditions (<xref ref-type="fig" rid="fig6">Figure 6A</xref>, open symbols and solid black curve). The average threshold predicted for the visual condition (1.24° ± 0.02°, mean ± SEM) was 42% lower than that for the vestibular condition (2.16° ± 0.14°,mean ± SEM). This may be due to the fact that the average neuronal threshold of congruent MSTd neurons tends to be lower for the visual condition (5.5°) than the vestibular condition (7.1°), although this difference did not reach significance (p=0.106, <italic>t</italic> test, N = 30, <xref ref-type="fig" rid="fig6s1">Figure 6—figure supplement 1</xref>). Despite the performance mismatch between the two single-cue conditions, sensitivity was still improved during the combined condition (1.10° ± 0.05°), as expected by optimal cue integration predictions (1.07° ± 0.02°).</p><p>For the selective decoding model, we examined how performance changes as a function of the weighting applied to opposite cells (Readout Index, <xref ref-type="fig" rid="fig6">Figure 6B</xref>, ‘Materials and methods’). In the vestibular condition, the decoder's discrimination threshold decreased modestly as more weight was applied to opposite cells because more cells contribute to the likelihood function (<xref ref-type="fig" rid="fig6">Figure 6B</xref>, blue curve). In the visual condition, because heading was decoded with respect to the vestibular preference of each neuron, a greater contribution of opposite cells tends to drive the decoder to make choices that are opposite to that being signaled by congruent cells. Consequently, predicted thresholds in the visual condition rise precipitously as opposite cells are given more weight (<xref ref-type="fig" rid="fig6">Figure 6B</xref>, red curve). From these data, we can see that a Readout Index value of ∼0.5 produces roughly matched visual and vestibular heading thresholds, similar to the animals’ behavior (arrow in <xref ref-type="fig" rid="fig6">Figure 6B</xref>). With this weighting of opposite cells, the selective decoding model predicts similar heading thresholds for vestibular (2.33° ± 0.04°, mean ± SEM) and visual (2.40° ± 0.1°) conditions, as well as thresholds in the combined condition (1.58° ± 0.01°) that are very close to the optimal prediction (1.57° ± 0.02°, <xref ref-type="fig" rid="fig6">Figure 6B</xref>, green and brown curves). Critically, we note that the value of the readout index (0.5) that allows the selective-decoding model to mimic behavioral performance is the same value that we separately found to produce a pattern of CPs that approximates the experimentally observed data (<xref ref-type="fig" rid="fig5">Figure 5C,G</xref>). Thus, converging lines of evidence suggest a readout in which opposite cells contribute, but substantially less than congruent cells.</p><p>The above simulation results were based on an arbitrarily sized population of neurons (n = 1000). Not surprisingly, as the population size is varied in the simulations, predicted psychophysical thresholds decline as a function of population size in all stimulus conditions (<xref ref-type="fig" rid="fig6">Figure 6C</xref>). In our simulations, performance reaches a plateau at a population size of a few hundred neurons, and is roughly comparable to the animals' behavioral performance over a broad range of pool sizes. It must be noted that the extent to which performance asymptotes with population size is likely to depend on the exact structure of correlated noise, the extent to which the true decoder has full and accurate knowledge of the correlation structure, the extent to which correlated noise mimics stimulus variations and thus can be removed by decoding, and the degree of heterogeneity of the tuning curves in the population (<xref ref-type="bibr" rid="bib14">Ecker et al., 2011</xref>; <xref ref-type="bibr" rid="bib3">Beck et al., 2012</xref>). Importantly, however, the key experimental features that we have sought to understand here are related to the sign of CP for opposite cells and not simply the magnitude of CPs; thus, the basic qualitative nature of our findings is not likely to be altered by the considerations above.</p></sec></sec><sec id="s3" sec-type="discussion"><title>Discussion</title><p>We explored the relative roles of correlated noise and selective decoding in generating the pattern of CPs observed across a population of multisensory neurons. While it is well accepted that correlated noise is necessary to observe CPs in large populations of cortical neurons (<xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>; <xref ref-type="bibr" rid="bib11">Cohen and Kohn, 2011</xref>; <xref ref-type="bibr" rid="bib28">Nienborg et al., 2012</xref>), a critical question is whether CPs can also reflect selective decoding of neural responses. Both the pure-correlation model and the selective decoding model could account for the peculiar finding that opposite cells show CPs that are systematically >0.5 in the vestibular condition and <0.5 in the visual condition (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>; <xref ref-type="bibr" rid="bib9">Chen et al., 2013</xref>). However, three main features of our findings favor the selective decoding model over the pure correlation model. First, the pure correlation model predicts a pattern of correlated noise that is inconsistent with experimental data from area MSTd, whereas correlated noise in the selective decoding model depends on congruency of tuning in a manner that is similar to that exhibited by pairs of MSTd neurons. Second, the pure correlation model predicts that CPs for opposite cells in the combined condition should be bimodally distributed, which is inconsistent with data from MSTd and with a broad range of predictions of the selective decoding model. Third, with modest weight given to opposite cells, the selective decoding model predicts a pattern of psychophysical sensitivity across stimulus conditions that closely matches the behavioral data from monkeys. Together our findings indicate that, although correlated noise is essential to observe CPs in large neural populations, selective decoding can play important roles in shaping patterns of CPs and behavioral performance.</p><sec id="s3-1"><title>Origins of choice probability</title><p>Choice probability measures the trial-by-trial correlation between the activity of a single neuron and perceptual decisions. While the measurement itself is straightforward, the interpretation of CPs has been varied and somewhat controversial (<xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>; <xref ref-type="bibr" rid="bib28">Nienborg et al., 2012</xref>). One possible interpretation of a significant CP is that variability in the response of a sensory neuron drives variability in perceptual decisions across trials—this is the so-called ‘bottom-up’ interpretation (<xref ref-type="bibr" rid="bib30">Parker and Newsome, 1998</xref>). If this were true, then CPs would at least partially reflect the contribution of each neuron to the decision, and would be shaped by selective decoding of sensory signals. Along these lines, some studies have suggested that the pattern of CPs observed reflects the strategy by which sensory signals are decoded to perform specific tasks (<xref ref-type="bibr" rid="bib36">Uka and DeAngelis, 2004</xref>; <xref ref-type="bibr" rid="bib31">Purushothaman and Bradley, 2005</xref>; <xref ref-type="bibr" rid="bib17">Gu et al., 2007</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>). Somewhat analogous conclusions were drawn in a previous study which showed that ‘detect probabilities’ depend on tuning preferences in a change-detection task, which may also be compatible with the notion of selective decoding (<xref ref-type="bibr" rid="bib4">Bosking and Maunsell, 2011</xref>).</p><p>An alternative (but not mutually exclusive) possibility is that CPs are mainly driven by top-down feedback signals from parts of the brain involved in making decisions (<xref ref-type="bibr" rid="bib26">Nienborg and Cumming, 2009</xref>, <xref ref-type="bibr" rid="bib27">2010</xref>). If this is the case, then the pattern of CPs need not be directly related to the way that sensory signals are decoded to perform a task. Regardless of the relative roles of bottom-up and top-down signals in generating CPs, it is broadly recognized that CPs should not be observable in large neural populations unless noise is correlated among neurons (<xref ref-type="bibr" rid="bib34">Shadlen et al., 1996</xref>; <xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>; <xref ref-type="bibr" rid="bib27">Nienborg and Cumming, 2010</xref>; <xref ref-type="bibr" rid="bib11">Cohen and Kohn, 2011</xref>; <xref ref-type="bibr" rid="bib28">Nienborg et al., 2012</xref>). Indeed, a recent study (<xref ref-type="bibr" rid="bib25">Liu et al., 2013</xref>) provided the first experimental evidence that the difference in magnitude of CPs between two brain areas coincides with a difference in the structure of correlated noise between areas.</p><p>The controversy regarding whether CPs can reflect decoding strategy or just correlated noise was recently resolved by an important theoretical study (<xref ref-type="bibr" rid="bib22">Haefner et al., 2013</xref>), which shows that CPs are determined by both factors. However, whether the pattern of CPs in a population reflects decoding strategy or not will depend on the specific details of the decoding weights, correlation structure, population size, etc. This theory shows that the decoding weights could be inferred from CPs if the full structure of correlated noise is known with sufficient accuracy and precision.</p><p>Using multisensory heading perception as a model system, we show that the pattern of CPs exhibited by neurons in area MSTd is more compatible with a model in which both selective decoding and correlated noise contribute to the generation of CPs than a model in which CPs are determined solely from correlations. Multisensory representations may have advantages for studying CPs because of the presence of neurons that show opposite tuning for the two cues. If such neurons are decoded as providing evidence in favor of either their visual or vestibular heading preference, then the sign of their CP (whether it is > or <0.5) may reverse depending on the decoding strategy. We suspect that this feature of our model system has provided us with additional leverage to dissociate models that emphasize correlations vs selective decoding.</p><p>In our selective decoding model, we assume that responses of all MSTd neurons, both congruent and opposite, are decoded relative to their vestibular heading preference. This accounts for opposite cells having CPs <0.5 in the visual condition, as seen in the real data. Why might responses be decoded according to the vestibular heading preference? One possibility is that this allows the system to estimate heading in a manner that is robust to the presence of moving objects in a scene. Indeed, we have recently shown that a strategy of decoding both congruent and opposite cells according to their vestibular preferences can provide a near-optimal solution to the problem of marginalizing over object motion in order to extract heading in a robust manner (<xref ref-type="bibr" rid="bib24">Kim et al., 2014</xref>). Moreover, adjusting the relative weighting of opposite to congruent cells can allow the population code to tradeoff robustness to object motion against increased sensitivity during cue integration (<xref ref-type="bibr" rid="bib24">Kim et al., 2014</xref>). Thus, the selective decoding strategy that we employ here may provide a flexible way to decode self-motion signals efficiently under conditions in which moving objects may or may not distort optic flow.</p></sec></sec><sec id="s4" sec-type="materials|methods"><title>Materials and methods</title><sec id="s4-1"><title>Tuning curves for hypothetical and real neurons</title><sec id="s4-1-1"><title>Hypothetical neurons</title><p>Two populations of MSTd-like neurons were simulated with cosine heading tuning:<disp-formula id="equ1"><label>(1)</label><mml:math id="m1"><mml:mrow><mml:msub><mml:mrow><mml:mi>F</mml:mi><mml:mi>i</mml:mi><mml:mi>r</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi><mml:mi>g</mml:mi><mml:mo>_</mml:mo><mml:mi>m</mml:mi><mml:mi>e</mml:mi><mml:mi>a</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mi>A</mml:mi><mml:mo>×</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>cos</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mi>θ</mml:mi><mml:mo>+</mml:mo><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>×</mml:mo><mml:mi>π</mml:mi><mml:mo>/</mml:mo><mml:mn>180</mml:mn></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>where <italic>k</italic> is the stimulus condition (visual, vestibular), <italic>i</italic> indexes a particular neuron, and <italic>θ</italic> denotes heading direction within the horizontal plane [−180° +180°]. To simulate the heading discrimination task (<xref ref-type="bibr" rid="bib17">Gu et al., 2007</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>), we used a small range of headings [<italic>θ =</italic> ±8°, ±4°, ±2°, ±1°, ±0.5°, ±0.25°, ±0.1° and 0°] around straight ahead. <italic>P<sub>k,i</sub></italic> denotes the heading preference of each neuron, which is either +90° (rightward) or −90° (leftward) for the purposes of simulating heading discrimination, given that most neurons have monotonic tuning over this range (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>). Each model population contains equal numbers of units with leftward and rightward heading preferences. <italic>A</italic> is a scaling factor to adjust the peak response amplitude, and was arbitrarily set to be 100 such that the peak response of each neuron is 200 spikes/s and the minimum response is 0 spike/s.</p></sec><sec id="s4-1-2"><title>Real neurons</title><p>Data were acquired from 129 single MSTd neurons that were recorded previously while animals performed a fine heading discrimination task based on either vestibular or visual (optic flow) cues, as well as the congruent combination of these cues (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>). In each trial of the discrimination task, the monkey experienced forward motion with a small leftward or rightward component. At the end of each trial, the monkey made a saccadic eye movement to report his perceived motion as leftward or rightward relative to straight ahead. Across trials, heading was varied in fine steps around straight ahead. Nine logarithmically spaced heading angles were tested for each monkey, including an ambiguous straight-forward direction (monkey A: ±9°, ±3.5°, ±1.3°, ±0.5° and 0°; monkey C: ±16°, ±6.4°, ±2.5°, ±1°, and 0°). Because the tested headings were different between the two animals, these measured tuning curves were linearly interpolated to 0.1° resolution, and a set of new local tuning curves was constructed that had a common set of stimulus headings for the two animals [±8°, ±4°, ±2°, ±1°, ±0.5°, ±0.2°, ±0.1° and 0°]. This allowed data to be pooled across animals and decoded as a single population.</p></sec></sec><sec id="s4-2"><title>Correlated noise</title><p>Correlation matrices that describe interneuronal noise correlations for a simulated population of neurons were constructed by assigning correlated noise (r<sub>noise</sub>) to each pair of neurons (either hypothetical or real) according to the measured relationship between r<sub>noise</sub> and the signal correlation between the pair of tuning curves, that is <italic>r</italic><sub><italic>signal</italic></sub> (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>). Signal correlation, <italic>r</italic><sub><italic>signal</italic></sub>, was computed as the Pearson correlation coefficient between the tuning curves (mean firing rates) for a pair of neurons. We previously showed that the relationship between r<sub>noise</sub> and <italic>r</italic><sub><italic>signal</italic></sub> for MSTd neurons could be well described by <xref ref-type="bibr" rid="bib21">Gu et al. (2011)</xref>:<disp-formula id="equ2"><label>(2)</label><mml:math id="m2"><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi mathvariant="italic">noise</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>b</mml:mi><mml:mi>u</mml:mi><mml:mi>l</mml:mi><mml:mi>a</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>×</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi>s</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>n</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mo>,</mml:mo><mml:mi>v</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>t</mml:mi><mml:mi>i</mml:mi><mml:mi>b</mml:mi><mml:mi>u</mml:mi><mml:mi>l</mml:mi><mml:mi>a</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>+</mml:mo><mml:msub><mml:mi>a</mml:mi><mml:mrow><mml:mi>v</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>u</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi></mml:mrow></mml:msub><mml:mo>×</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi>s</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>n</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mo>,</mml:mo><mml:mi>v</mml:mi><mml:mi>i</mml:mi><mml:mi>s</mml:mi><mml:mi>u</mml:mi><mml:mi>a</mml:mi><mml:mi>l</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>where <italic>a</italic><sub><italic>vestibular</italic></sub> was 0.12 and <italic>a</italic><sub><italic>visual</italic></sub> was 0.09 (<xref ref-type="bibr" rid="bib21">Gu et al., 2011</xref>). For population decoding simulations based on real neurons (<xref ref-type="fig" rid="fig4 fig5 fig6">Figures 4–6</xref>), correlated noise between pairs of neurons was assigned according to <xref ref-type="disp-formula" rid="equ2">Equation 2</xref> with the parameters above. For simulations based on hypothetical neurons, both coefficients in <xref ref-type="disp-formula" rid="equ2">Equation 2</xref> were set to 0.1 for simplicity.</p><p>Single-trial responses of model neurons to each heading stimulus were generated according to the assumption of proportional Gaussian noise, with response variance set to be 1.5 times the mean firing rate to approximate the general behavior of cortical neurons (<xref ref-type="bibr" rid="bib34">Shadlen et al., 1996</xref>; <xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>; <xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>). To generate simulated population responses that incorporated correlated noise among neurons, we incorporated the estimated correlation matrix and generated population activity for each stimulus modality (<italic>k</italic>), trial (<italic>tr</italic>), and heading (<italic>θ</italic>) according to the following equation (<xref ref-type="bibr" rid="bib34">Shadlen et al., 1996</xref>; <xref ref-type="bibr" rid="bib10">Cohen and Newsome, 2009</xref>):<disp-formula id="equ3"><label>(3)</label><mml:math id="m3"><mml:mrow><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>=</mml:mo><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo><mml:mo>+</mml:mo><mml:mi>Q</mml:mi><mml:mo>×</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi>r</mml:mi><mml:mi>a</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>×</mml:mo><mml:msqrt><mml:mrow><mml:mn>1.5</mml:mn><mml:mo>×</mml:mo><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo></mml:mrow></mml:msqrt><mml:mo>,</mml:mo></mml:mrow></mml:math></disp-formula>where <italic>Q</italic> is the square root of the correlation matrix, <italic>r</italic><sub><italic>rand</italic></sub> is a random vector of standard normal deviates with the same length as the number of neurons (Matlab function ‘normrnd’, zero mean, unit variance), and ‘<>’ represents the mean value. We typically produced 200 trials of responses for each heading (<italic>θ</italic>).</p></sec><sec id="s4-3"><title>Analysis of visual-vestibular tuning congruency</title><p>As shown previously, neurons in area MSTd have vestibular heading tuning that can be either congruent with or opposite to their visual heading tuning (<xref ref-type="bibr" rid="bib19">Gu et al., 2006</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>). Over the narrow range of headings used in the discrimination task, congruent cells generally have monotonic visual and vestibular tuning curves with matched slopes, whereas opposite cells generally have oppositely signed slopes (<xref ref-type="fig" rid="fig2">Figure 2A</xref>). To quantify tuning congruency, we compute a Pearson correlation coefficient between firing rate and heading for each tuning curve. From these, we compute a Congruency Index for each neuron, which is the product of the correlation coefficients for the visual and vestibular tuning curves. Thus, congruent cells will have a positive Congruency Index, whereas opposite cells with have a negative Congruency Index.</p><p>A pair of MSTd neurons can therefore have matched congruency (congruent–congruent pairs or opposite–opposite pairs), or mismatched congruency (congruent–opposite pairs). We categorized each pair of neurons as matched or mismatched by computing the product of their two congruency indices. Specifically, matched congruency pairs were classified as those having a product >0.2 (black symbols, <xref ref-type="fig" rid="fig3">Figure 3A</xref>), and mismatched pairs are those having a product <−0.2 (gray symbols, <xref ref-type="fig" rid="fig3">Figure 3A</xref>). The remaining cell pairs with products of Congruency Indices that fall in the range from −0.2 to +0.2 were classified as ‘undefined’ (open symbols, <xref ref-type="fig" rid="fig3">Figure 3A</xref>). These criteria are rather stringent, but we found that they reliably classify cells pairs.</p><p>To test whether a distribution of CPs contains a single mode (unimodal) or two modes (bimodal), we used a multimodality test based on the kernel density estimate method (<xref ref-type="bibr" rid="bib19">Gu et al., 2006</xref>; <xref ref-type="bibr" rid="bib35">Takahashi et al., 2007</xref>). Watson's U2 statistic, corrected for grouping, was computed as a goodness-of-fit test statistic to obtain a p value through a bootstrapping procedure. This test generates two p values, with the first one (p<sub>uni</sub>) for the test of unimodality and the second one (p<sub>bi</sub>) for the test of bimodality. If p<sub>uni</sub> >0.05, the distribution is defined as unimodal. If p<sub>uni</sub> <0.05, the hypothesis of unimodality is rejected. If p<sub>bi</sub> >0.05 as well, the distribution is considered bimodal.</p></sec><sec id="s4-4"><title>Population decoding</title><p>To transform responses from a population of neurons into quantitative predictions of behavioral sensitivity and choice probabilities, we decoded simulated population activity by computing a likelihood function as in previous studies (<xref ref-type="bibr" rid="bib33">Sanger, 1996</xref>; <xref ref-type="bibr" rid="bib12">Dayan and Abbott, 2001</xref>; <xref ref-type="bibr" rid="bib23">Jazayeri and Movshon, 2006</xref>; <xref ref-type="bibr" rid="bib20">Gu et al., 2010</xref>; <xref ref-type="bibr" rid="bib16">Fetsch et al., 2011</xref>). For each stimulus modality (<italic>k</italic>), the likelihood over heading (<italic>θ</italic>) given the observed population activity on a particular trial (<italic>tr</italic>) was given by:<disp-formula id="equ4"><label>(4)</label><mml:math id="m4"><mml:mrow><mml:mi>L</mml:mi><mml:mi>o</mml:mi><mml:mi>g</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>L</mml:mi><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>×</mml:mo><mml:mi>L</mml:mi><mml:mi>o</mml:mi><mml:mi>g</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>−</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo></mml:mrow></mml:mstyle></mml:mrow></mml:mstyle><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p><p>The first term describes the summation of each cell’s contribution to the log likelihood function, which corresponds to the response on each trial weighted by the logarithm of each cell’s tuning curve. The second term is the sum of all tuning curves, to counter biases associated with a non-uniform distribution of stimulus preferences.</p><p>This formulation embodies two assumptions. First, it assumes Poisson firing statistics. We have also tried alternative decoders that are based on Gaussian noise, such as the Fisher linear discriminant (<xref ref-type="bibr" rid="bib12">Dayan and Abbott, 2001</xref>), and the results are almost identical. Thus, the details of the spiking statistics have little effect on our conclusions. Second, this formulation does not assume that the decoder has knowledge of the structure of correlated noise among neurons in the population, that is, a factorized decoder (<xref ref-type="bibr" rid="bib2">Averbeck et al., 2006</xref>). While this assumption very likely affects the absolute sensitivity of the decoder, it is unlikely to alter the basic pattern of predicted results regarding CPs (<xref ref-type="bibr" rid="bib20">Gu et al., 2010</xref>). However, further investigation is needed to systematically examine the detailed differences between decoders with and without knowledge of correlations, as well as the effects of different forms of correlated activity. Given these assumptions, our decoder is optimal in the maximum-likelihood sense, and does not otherwise assume a specific (and perhaps substantially suboptimal) ‘pooling model’, as was done in some previous simulations of CPs (<xref ref-type="bibr" rid="bib34">Shadlen et al., 1996</xref>).</p><p>The decoder determined whether a tested heading was leftward or rightward relative to straight ahead by comparing the area under the computed likelihood function for leftward headings and rightward headings. If the summed likelihood for rightward headings was greater than that for leftward headings, the decoder would report ‘right’, and vice versa. Choice probability for each neuron in the simulated population was consequently computed for the ambiguous straight-forward heading (0°) (<xref ref-type="bibr" rid="bib6">Britten et al., 1996</xref>; <xref ref-type="bibr" rid="bib34">Shadlen et al., 1996</xref>; <xref ref-type="bibr" rid="bib17">Gu et al., 2007</xref>, <xref ref-type="bibr" rid="bib18">2008</xref>). The precision (threshold) of the decoder was also computed from each simulated psychometric function, which was analyzed using methods identical to those applied to the human and monkey behavior (<xref ref-type="bibr" rid="bib20">Gu et al., 2010</xref>).</p><p>Heading information was decoded in two ways. According to our hypothesis that both congruent and opposite cells are decoded according to their vestibular heading preference (<xref ref-type="bibr" rid="bib18">Gu et al., 2008</xref>), our main method of decoding involved using the vestibular heading tuning curve for each neuron in the formulation of <xref ref-type="disp-formula" rid="equ4">Equation 4</xref>. In addition, for some analyses, heading was also decoded relative to the heading preference of each neuron in each stimulus condition. In this case, responses from the visual condition were decoded based on visual tuning curves, and so on.</p><p>For the selective decoding model, we implemented a function that controlled the contribution of each model neuron to the decoder output based on each neuron’s congruency value, as given below:<disp-formula id="equ5"><label>(5)</label><mml:math id="m5"><mml:mrow><mml:msub><mml:mrow><mml:mi>w</mml:mi><mml:mi>e</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>h</mml:mi><mml:mi>t</mml:mi></mml:mrow><mml:mi>i</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mn>1</mml:mn><mml:mo>−</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>−</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mrow><mml:mrow><mml:msub><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>g</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mi>n</mml:mi><mml:mi>c</mml:mi><mml:mi>y</mml:mi></mml:mrow><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>/</mml:mo><mml:mn>2</mml:mn></mml:mrow></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>×</mml:mo><mml:mi>R</mml:mi><mml:mi>I</mml:mi></mml:mrow></mml:msup></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>−</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>−</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:mrow></mml:mfrac><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p><p>Here, <italic>weight</italic><sub><italic>i</italic></sub> denotes the decoding weight of the <italic>i</italic><sup>th</sup> neuron, <italic>congruency</italic><sub><italic>i</italic></sub> denotes the Congruency Index (described above) for the <italic>i</italic><sup>th</sup> neuron, and <italic>RI</italic> represents a Readout Index that ranges from 0 to 1 in steps of 0.1 (<xref ref-type="fig" rid="fig4">Figure 4B</xref>).</p><p>In the selective decoding model, the computed decoding weight of each neuron was then multiplied by that neuron's contribution to the likelihood function (<xref ref-type="disp-formula" rid="equ4">Equation 4</xref>) before the likelihood contributions are summed across neurons. Thus, the computation of the likelihood under the selective decoding model was given by:<disp-formula id="equ6"><label>(6)</label><mml:math id="m6"><mml:mrow><mml:mi>L</mml:mi><mml:mi>o</mml:mi><mml:mi>g</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:msub><mml:mi>L</mml:mi><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:msub><mml:mrow><mml:mi>w</mml:mi><mml:mi>e</mml:mi><mml:mi>i</mml:mi><mml:mi>g</mml:mi><mml:mi>h</mml:mi><mml:mi>t</mml:mi></mml:mrow><mml:mi>i</mml:mi></mml:msub><mml:mo>×</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>×</mml:mo><mml:mi>L</mml:mi><mml:mi>o</mml:mi><mml:mi>g</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>−</mml:mo><mml:mstyle displaystyle="true"><mml:munderover><mml:mo>∑</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mrow><mml:mo><</mml:mo><mml:msub><mml:mrow><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>s</mml:mi><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>,</mml:mo><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mo>,</mml:mo><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>(</mml:mo><mml:mi>θ</mml:mi><mml:mo>)</mml:mo><mml:mo>></mml:mo></mml:mrow></mml:mstyle></mml:mrow></mml:mstyle><mml:mo>.</mml:mo></mml:mrow></mml:math></disp-formula></p></sec></sec></body><back><sec sec-type="additional-information"><title>Additional information</title><fn-group content-type="competing-interest"><title>Competing interests</title><fn fn-type="conflict" id="conf1"><p>DEA: At the time of submission, DEA was a Reviewing editor for <italic>eLife</italic>.</p></fn><fn fn-type="conflict" id="conf2"><p>The other authors declare that no competing interests exist.</p></fn></fn-group><fn-group content-type="author-contribution"><title>Author contributions</title><fn fn-type="con" id="con1"><p>YG, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article</p></fn><fn fn-type="con" id="con2"><p>DEA, Conception and design, Analysis and interpretation of data, Drafting or revising the article</p></fn><fn fn-type="con" id="con3"><p>GCDA, Conception and design, Analysis and interpretation of data, Drafting or revising the article</p></fn></fn-group></sec><ref-list><title>References</title><ref id="bib1"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Angelaki</surname><given-names>DE</given-names></name><name><surname>Cullen</surname><given-names>KE</given-names></name></person-group><year>2008</year><article-title>Vestibular system: the many facets of a multimodal sense</article-title><source>Annual Review of Neuroscience</source><volume>31</volume><fpage>125</fpage><lpage>150</lpage><pub-id pub-id-type="doi">10.1146/annurev.neuro.31.060407.125555</pub-id></element-citation></ref><ref id="bib2"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Averbeck</surname><given-names>BB</given-names></name><name><surname>Latham</surname><given-names>PE</given-names></name><name><surname>Pouget</surname><given-names>A</given-names></name></person-group><year>2006</year><article-title>Neural correlations, population coding and computation</article-title><source>Nature Reviews Neuroscience</source><volume>7</volume><fpage>358</fpage><lpage>366</lpage><pub-id pub-id-type="doi">10.1038/nrn1888</pub-id></element-citation></ref><ref id="bib3"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Beck</surname><given-names>JMR</given-names></name><name><surname>Latham</surname><given-names>P</given-names></name><name><surname>Pouget</surname><given-names>A</given-names></name></person-group><year>2012</year><article-title>Noise correlations in population codes with finite information. Cosyne 2012</article-title></element-citation></ref><ref id="bib4"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bosking</surname><given-names>WH</given-names></name><name><surname>Maunsell</surname><given-names>JH</given-names></name></person-group><year>2011</year><article-title>Effects of stimulus direction on the correlation between behavior and single units in area MT during a motion detection task</article-title><source>The Journal of Neuroscience</source><volume>31</volume><fpage>8230</fpage><lpage>8238</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.0126-11.2011</pub-id></element-citation></ref><ref id="bib5"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Britten</surname><given-names>KH</given-names></name></person-group><year>2008</year><article-title>Mechanisms of self-motion perception</article-title><source>Annual Review of Neuroscience</source><volume>31</volume><fpage>389</fpage><lpage>410</lpage><pub-id pub-id-type="doi">10.1146/annurev.neuro.29.051605.112953</pub-id></element-citation></ref><ref id="bib6"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Britten</surname><given-names>KH</given-names></name><name><surname>Newsome</surname><given-names>WT</given-names></name><name><surname>Shadlen</surname><given-names>MN</given-names></name><name><surname>Celebrini</surname><given-names>S</given-names></name><name><surname>Movshon</surname><given-names>JA</given-names></name></person-group><year>1996</year><article-title>A relationship between behavioral choice and the visual responses of neurons in macaque MT</article-title><source>Visual Neuroscience</source><volume>13</volume><fpage>87</fpage><lpage>100</lpage><pub-id pub-id-type="doi">10.1017/S095252380000715X</pub-id></element-citation></ref><ref id="bib7"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Chen</surname><given-names>A</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2011a</year><article-title>Convergence of vestibular and visual self-motion signals in an area of the posterior sylvian fissure</article-title><source>The Journal of Neuroscience</source><volume>31</volume><fpage>11617</fpage><lpage>11627</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.1266-11.2011</pub-id></element-citation></ref><ref id="bib8"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Chen</surname><given-names>A</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2011b</year><article-title>Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex</article-title><source>The Journal of Neuroscience</source><volume>31</volume><fpage>12036</fpage><lpage>12052</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.0395-11.2011</pub-id></element-citation></ref><ref id="bib9"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Chen</surname><given-names>A</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2013</year><article-title>Functional specializations of the ventral intraparietal area for multisensory heading discrimination</article-title><source>The Journal of Neuroscience</source><volume>33</volume><fpage>3567</fpage><lpage>3581</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.4522-12.2013</pub-id></element-citation></ref><ref id="bib10"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cohen</surname><given-names>MR</given-names></name><name><surname>Newsome</surname><given-names>WT</given-names></name></person-group><year>2009</year><article-title>Estimates of the contribution of single neurons to perception depend on timescale and noise correlation</article-title><source>The Journal of Neuroscience</source><volume>29</volume><fpage>6635</fpage><lpage>6648</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.5179-08.2009</pub-id></element-citation></ref><ref id="bib11"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cohen</surname><given-names>MR</given-names></name><name><surname>Kohn</surname><given-names>A</given-names></name></person-group><year>2011</year><article-title>Measuring and interpreting neuronal correlations</article-title><source>Nature Neuroscience</source><volume>14</volume><fpage>811</fpage><lpage>819</lpage><pub-id pub-id-type="doi">10.1038/nn.2842</pub-id></element-citation></ref><ref id="bib12"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Dayan</surname><given-names>P</given-names></name><name><surname>Abbott</surname><given-names>LF</given-names></name></person-group><year>2001</year><source>Theoretical neuroscience</source><publisher-name>MIT Press</publisher-name></element-citation></ref><ref id="bib13"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Dodd</surname><given-names>JV</given-names></name><name><surname>Krug</surname><given-names>K</given-names></name><name><surname>Cumming</surname><given-names>BG</given-names></name><name><surname>Parker</surname><given-names>AJ</given-names></name></person-group><year>2001</year><article-title>Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT</article-title><source>The Journal of Neuroscience</source><volume>21</volume><fpage>4809</fpage><lpage>4821</lpage></element-citation></ref><ref id="bib14"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ecker</surname><given-names>AS</given-names></name><name><surname>Berens</surname><given-names>P</given-names></name><name><surname>Tolias</surname><given-names>AS</given-names></name><name><surname>Bethge</surname><given-names>M</given-names></name></person-group><year>2011</year><article-title>The effect of noise correlations in populations of diversely tuned neurons</article-title><source>The Journal of Neuroscience</source><volume>31</volume><fpage>14272</fpage><lpage>14283</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.2539-11.2011</pub-id></element-citation></ref><ref id="bib15"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Fetsch</surname><given-names>CR</given-names></name><name><surname>Turner</surname><given-names>AH</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2009</year><article-title>Dynamic reweighting of visual and vestibular cues during self-motion perception</article-title><source>The Journal of Neuroscience</source><volume>29</volume><fpage>15601</fpage><lpage>15612</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.2574-09.2009</pub-id></element-citation></ref><ref id="bib16"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Fetsch</surname><given-names>CR</given-names></name><name><surname>Pouget</surname><given-names>A</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2011</year><article-title>Neural correlates of reliability-based cue weighting during multisensory integration</article-title><source>Nature Neuroscience</source><volume>15</volume><fpage>146</fpage><lpage>154</lpage><pub-id pub-id-type="doi">10.1038/nn.2983</pub-id></element-citation></ref><ref id="bib17"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2007</year><article-title>A functional link between area MSTd and heading perception based on vestibular signals</article-title><source>Nature Neuroscience</source><volume>10</volume><fpage>1038</fpage><lpage>1047</lpage><pub-id pub-id-type="doi">10.1038/nn1935</pub-id></element-citation></ref><ref id="bib18"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name></person-group><year>2008</year><article-title>Neural correlates of multisensory cue integration in macaque MSTd</article-title><source>Nature Neuroscience</source><volume>11</volume><fpage>1201</fpage><lpage>1210</lpage><pub-id pub-id-type="doi">10.1038/nn.2191</pub-id></element-citation></ref><ref id="bib19"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>Watkins</surname><given-names>PV</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name></person-group><year>2006</year><article-title>Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area</article-title><source>The Journal of Neuroscience</source><volume>26</volume><fpage>73</fpage><lpage>85</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.2356-05.2006</pub-id></element-citation></ref><ref id="bib20"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>Fetsch</surname><given-names>CR</given-names></name><name><surname>Adeyemo</surname><given-names>B</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2010</year><article-title>Decoding of MSTd population activity accounts for variations in the precision of heading perception</article-title><source>Neuron</source><volume>66</volume><fpage>596</fpage><lpage>609</lpage><pub-id pub-id-type="doi">10.1016/j.neuron.2010.04.026</pub-id></element-citation></ref><ref id="bib21"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>Liu</surname><given-names>S</given-names></name><name><surname>Fetsch</surname><given-names>CR</given-names></name><name><surname>Yang</surname><given-names>Y</given-names></name><name><surname>Fok</surname><given-names>S</given-names></name><name><surname>Sunkara</surname><given-names>A</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2011</year><article-title>Perceptual learning reduces interneuronal correlations in macaque visual cortex</article-title><source>Neuron</source><volume>71</volume><fpage>750</fpage><lpage>761</lpage><pub-id pub-id-type="doi">10.1016/j.neuron.2011.06.015</pub-id></element-citation></ref><ref id="bib22"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Haefner</surname><given-names>RM</given-names></name><name><surname>Gerwinn</surname><given-names>S</given-names></name><name><surname>Macke</surname><given-names>JH</given-names></name><name><surname>Bethge</surname><given-names>M</given-names></name></person-group><year>2013</year><article-title>Inferring decoding strategies from choice probabilities in the presence of correlated variability</article-title><source>Nature Neuroscience</source><volume>16</volume><fpage>235</fpage><lpage>242</lpage><pub-id pub-id-type="doi">10.1038/nn.3309</pub-id></element-citation></ref><ref id="bib23"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Jazayeri</surname><given-names>M</given-names></name><name><surname>Movshon</surname><given-names>JA</given-names></name></person-group><year>2006</year><article-title>Optimal representation of sensory information by neural populations</article-title><source>Nature Neuroscience</source><volume>9</volume><fpage>690</fpage><lpage>696</lpage><pub-id pub-id-type="doi">10.1038/nn1691</pub-id></element-citation></ref><ref id="bib24"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kim</surname><given-names>HRPX</given-names></name><name><surname>Sasaki</surname><given-names>R</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name></person-group><year>2014</year><article-title>Near-optimal linear decoding for marginalization: application to heading estimation. Cosyne 2014</article-title></element-citation></ref><ref id="bib25"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Liu</surname><given-names>S</given-names></name><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2013</year><article-title>Choice-related activity and correlated noise in subcortical vestibular neurons</article-title><source>Nature Neuroscience</source><volume>16</volume><fpage>89</fpage><lpage>97</lpage><pub-id pub-id-type="doi">10.1038/nn.3267</pub-id></element-citation></ref><ref id="bib26"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nienborg</surname><given-names>H</given-names></name><name><surname>Cumming</surname><given-names>BG</given-names></name></person-group><year>2009</year><article-title>Decision-related activity in sensory neurons reflects more than a neuron's causal effect</article-title><source>Nature</source><volume>459</volume><fpage>89</fpage><lpage>92</lpage><pub-id pub-id-type="doi">10.1038/nature07821</pub-id></element-citation></ref><ref id="bib27"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nienborg</surname><given-names>H</given-names></name><name><surname>Cumming</surname><given-names>B</given-names></name></person-group><year>2010</year><article-title>Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron's causality?</article-title><source>Current Opinion in Neurobiology</source><volume>20</volume><fpage>376</fpage><lpage>381</lpage><pub-id pub-id-type="doi">10.1016/j.conb.2010.05.002</pub-id></element-citation></ref><ref id="bib28"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nienborg</surname><given-names>H</given-names></name><name><surname>Cohen</surname><given-names>MR</given-names></name><name><surname>Cumming</surname><given-names>BG</given-names></name></person-group><year>2012</year><article-title>Decision-related activity in sensory neurons: correlations among neurons and with behavior</article-title><source>Annual Review of Neuroscience</source><volume>35</volume><fpage>463</fpage><lpage>483</lpage><pub-id pub-id-type="doi">10.1146/annurev-neuro-062111-150403</pub-id></element-citation></ref><ref id="bib29"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Page</surname><given-names>WK</given-names></name><name><surname>Duffy</surname><given-names>CJ</given-names></name></person-group><year>2003</year><article-title>Heading representation in MST: sensory interactions and population encoding</article-title><source>Journal of Neurophysiology</source><volume>89</volume><fpage>1994</fpage><lpage>2013</lpage><pub-id pub-id-type="doi">10.1152/jn.00493.2002</pub-id></element-citation></ref><ref id="bib30"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Parker</surname><given-names>AJ</given-names></name><name><surname>Newsome</surname><given-names>WT</given-names></name></person-group><year>1998</year><article-title>Sense and the single neuron: probing the physiology of perception</article-title><source>Annual Review of Neuroscience</source><volume>21</volume><fpage>227</fpage><lpage>277</lpage><pub-id pub-id-type="doi">10.1146/annurev.neuro.21.1.227</pub-id></element-citation></ref><ref id="bib31"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Purushothaman</surname><given-names>G</given-names></name><name><surname>Bradley</surname><given-names>DC</given-names></name></person-group><year>2005</year><article-title>Neural population code for fine perceptual decisions in area MT</article-title><source>Nature Neuroscience</source><volume>8</volume><fpage>99</fpage><lpage>106</lpage><pub-id pub-id-type="doi">10.1038/nn1373</pub-id></element-citation></ref><ref id="bib32"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Romo</surname><given-names>R</given-names></name><name><surname>Hernandez</surname><given-names>A</given-names></name><name><surname>Zainos</surname><given-names>A</given-names></name><name><surname>Salinas</surname><given-names>E</given-names></name></person-group><year>2003</year><article-title>Correlated neuronal discharges that increase coding efficiency during perceptual discrimination</article-title><source>Neuron</source><volume>38</volume><fpage>649</fpage><lpage>657</lpage><pub-id pub-id-type="doi">10.1016/S0896-6273(03)00287-3</pub-id></element-citation></ref><ref id="bib33"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Sanger</surname><given-names>TD</given-names></name></person-group><year>1996</year><article-title>Probability density estimation for the interpretation of neural population codes</article-title><source>Journal of Neurophysiology</source><volume>76</volume><fpage>2790</fpage><lpage>2793</lpage></element-citation></ref><ref id="bib34"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Shadlen</surname><given-names>MN</given-names></name><name><surname>Britten</surname><given-names>KH</given-names></name><name><surname>Newsome</surname><given-names>WT</given-names></name><name><surname>Movshon</surname><given-names>JA</given-names></name></person-group><year>1996</year><article-title>A computational analysis of the relationship between neuronal and behavioral responses to visual motion</article-title><source>The Journal of Neuroscience</source><volume>16</volume><fpage>1486</fpage><lpage>1510</lpage></element-citation></ref><ref id="bib35"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Takahashi</surname><given-names>K</given-names></name><name><surname>Gu</surname><given-names>Y</given-names></name><name><surname>May</surname><given-names>PJ</given-names></name><name><surname>Newlands</surname><given-names>SD</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name><name><surname>Angelaki</surname><given-names>DE</given-names></name></person-group><year>2007</year><article-title>Multimodal coding of three-dimensional rotation and translation in area MSTd: comparison of visual and vestibular selectivity</article-title><source>The Journal of Neuroscience</source><volume>27</volume><fpage>9742</fpage><lpage>9756</lpage><pub-id pub-id-type="doi">10.1523/JNEUROSCI.0817-07.2007</pub-id></element-citation></ref><ref id="bib36"><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Uka</surname><given-names>T</given-names></name><name><surname>DeAngelis</surname><given-names>GC</given-names></name></person-group><year>2004</year><article-title>Contribution of area MT to stereoscopic depth perception: choice-related response modulations reflect task strategy</article-title><source>Neuron</source><volume>42</volume><fpage>297</fpage><lpage>310</lpage><pub-id pub-id-type="doi">10.1016/S0896-6273(04)00186-2</pub-id></element-citation></ref></ref-list></back><sub-article article-type="article-commentary" id="SA1"><front-stub><article-id pub-id-type="doi">10.7554/eLife.02670.018</article-id><title-group><article-title>Decision letter</article-title></title-group><contrib-group content-type="section"><contrib contrib-type="editor"><name><surname>Romo</surname><given-names>Ranulfo</given-names></name><role>Reviewing editor</role><aff><institution>Universidad Nacional Autonoma de Mexico</institution>, <country>Mexico</country></aff></contrib></contrib-group></front-stub><body><boxed-text><p>eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see <ext-link ext-link-type="uri" xlink:href="http://elifesciences.org/review-process">review process</ext-link>). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.</p></boxed-text><p>Thank you for sending your work entitled “Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex” for consideration at <italic>eLife</italic>. Your article has been favorably evaluated by Eve Marder (Senior editor) and 3 reviewers, one of whom is a member of our Board of Reviewing Editors.</p><p>The Reviewing editor and the other reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission. Your manuscript occasioned a significant discussion among the reviewers, largely around the issue of whether your paper makes a general contribution or whether it deals with a relatively narrow set of issues. Therefore, in preparing your revision, please think about any generalizations of the work that might make this more salient to people working on associated problems. Minimally, these issues should be confronted squarely in the revised Discussion.</p><p><italic>General:</italic></p><p>This paper focuses on the nature of choice probabilities (CPs). CPs is a measure of the covariation, in trial-by-trial-basis, between neural activity and behavior, typically in a decision task. As is well explained by the authors, the origins of CPs could originate from pure correlated noise of pairs of neurons as estimates of a neuronal population or by selective decoding, the neural response to a stimulus parameter(s) and its relation to behavior. According to the authors there is an historical debate of whether CPs, originate from pure correlated noise from neurons or by the selective decoding in a neuronal population. Here the authors use both simulations in artificial neurons biologically constrained and then tested the simulated results from real data of MSTd neurons, which are known by the authors; these neurons are associated with heading responses to visual flow and vestibular perturbations. The results of both simulations and real data are that MSTs neurons best conform to a selective decoding computation, although pure correlated noise contributes to selectivity.</p><p><italic>Reviewer #1:</italic></p><p>Found that the conclusions do not appear quite categorical since the authors claim that correlated noise is important but also cannot be excluded the selectivity decoding hypothesis. Curiously, cells in second somatosensory cortex have opposite tuning properties to the same stimulus, but that shared correlated noise has beneficial effect for improving the neurometric function and correlates with the psychometric performance (Romo et al., Neuron 2003). The beneficial effect came from a subtraction operation between the opposite tuning and correlated noise. Is this the case for congruent cells? In other words, for pool 2 neurons that carry the decision, she/he suspects that another group of neurons doing the opposite and very likely by subtracting noise and not a simple cancellation.</p><p><italic>Reviewer #2:</italic></p><p>As the authors acknowledge, the idea that all MSTd neurons are decoded according to their vestibular tuning preferences is a bit counterintuitive. He/she wonders whether the same simulations could be used to test the implications of this sort of decoding on the amount of information extracted about the stimulus. That is, does decoding only according to vestibular preference account for any sub-optimalties in the monkey's behavior? Such a result would provide more evidence for this decoding algorithm.</p><p><italic>Reviewer #3:</italic></p><p>This reviewer considers that the clear cut difference between the two models can only be formulated within the context of the Shadlen et al 96 model (recently revised by Haeffner et al), which the authors seem to take a ground truth, but which is itself a useful but fairly crude and simplistic picture of how the decision is being made), i.e. totally feedforward, no dynamics, etc).</p><p>Secondly even if one accepted that the decision making model is valid, the analysis does not allow the authors to draw general enough conclusions. Conceivably, one would set oneself to answer the question of whether it is possible to rule out the possibility than incongruent neurons do not contribute at all to the decision, regardless of all other modeling choices. The reviewer wasn't convinced by the analysis that this is the case. There are 3 main pieces of evidence: the patterns of CCs in <xref ref-type="fig" rid="fig3">Figure 3</xref>, the patterns of CPs in <xref ref-type="fig" rid="fig5">Figure 5</xref> and the psychophysical data in <xref ref-type="fig" rid="fig6">Figure 6</xref>. In <xref ref-type="fig" rid="fig3">Figure 3</xref>, using an all-or-none approach where r_noise is ‘only’ a function of vestibular r signal, the data seems to favor the selective decoding model. However, (a) there are only 10 mixed pairs and (b) it is not at all clear that having r_noise be ‘mainly’ due to vestibular r_signal but with some contribution from visual r_signal, the two models would not look more similar to themselves and to data. This is a general concern: Even assuming the framework is valid, a simplified situation (in term of model parameters) is used to make categorical distinctions, and so it's not clear whether the conclusions are generic or if they only applied to the simplified situation.</p><p>The evidence from <xref ref-type="fig" rid="fig5">Figure 5</xref> is also not shown to be conclusive/generic. The bi-modality in <xref ref-type="fig" rid="fig5">Figure 5b</xref> is not explained clearly and so it's not clear what exactly is necessary to obtain bi-modality. Is there no model with zero weights from incongruent neurons able to show unimodal CPs? Are there no parameters from the selective decoding model which would result in bi-modality? To make things worse, the selective decoding model seems to have one more degree of freedom (Readout Index). Similar concerns of generality apply to the analysis in <xref ref-type="fig" rid="fig6">Figure 6</xref>.</p><p>The authors conclude that both noise correlations and selective weighting probably contribute to the measured CPs. But this is hardly surprising (again, even assuming the whole framework is valid)!</p><p>In summary, the paper attempts to make a relatively minor distinction (i.e., to refute a model that represents a small region of parameter space (CPs are a result of only CCs) compared to the alternatives (CPs are a result of both CCs and weights)) and it does so it not in general terms, which limits the potential impact of the work.</p></body></sub-article><sub-article article-type="reply" id="SA2"><front-stub><article-id pub-id-type="doi">10.7554/eLife.02670.019</article-id><title-group><article-title>Author response</article-title></title-group></front-stub><body><p>Reviewer #1:</p><p><italic>Found that the conclusions do not appear quite categorical since the authors claim that correlated noise is important but also cannot be excluded the selectivity decoding hypothesis</italic>.</p><p>We thank the reviewer for this comment as it is important to make these issues clear. Importantly, correlated noise is necessary to produce substantial choice probabilities in both models. So the key issue is whether the data suggest that there is selective decoding of responses of congruent and opposite cells. Another way to describe the key difference between the models is that the ‘pure correlation’ model relies on modality specificity of the noise correlation structure, whereas the ‘selective decoding’ model relies on modality specificity of the decoding weights. We have made a number of changes to the manuscript to clarify these issues, including additions and revisions.</p><p><italic>Curiously, cells in second somatosensory cortex have opposite tuning properties to the same stimulus, but that shared correlated noise has beneficial effect for improving the neurometric function and correlates with the psychometric performance (Romo et al., Neuron 2003). The beneficial effect came from a subtraction operation between the opposite tuning and correlated noise. Is this the case for congruent cells? In other words, for pool 2 neurons that carry the decision, she/he suspects that another group of neurons doing the opposite and very likely by subtracting noise and not a simple cancellation</italic>.</p><p>In the case of <xref ref-type="bibr" rid="bib32">Romo et al. (2003)</xref>, subtracting responses of oppositely-tuned neurons would be beneficial because these neurons generally showed positive noise correlations. In our studies, we consistently find that pairs of neurons with opposite tuning have negative noise correlations (<xref ref-type="bibr" rid="bib21">Gu et al, 2011</xref>; <xref ref-type="bibr" rid="bib9">Chen et al. 2013</xref>; <xref ref-type="fig" rid="fig3">Figure 3</xref>). This is the case for pairs of neurons with matched congruency (two congruent cells or two opposite cells), whereas mismatched pairs have weak noise correlation. Thus, it appears that the benefit of subtracting oppositely-tuned cells discovered by <xref ref-type="bibr" rid="bib32">Romo et al (2003)</xref> would not be helpful in our case. We now mention this in the text where we introduce our correlation structure, and we cite the study of <xref ref-type="bibr" rid="bib32">Romo et al. (2003)</xref>.</p><p>Reviewer #2:</p><p><italic>As the authors acknowledge, the idea that all MSTd neurons are decoded according to their vestibular tuning preferences is a bit counterintuitive</italic>.</p><p>It is indeed counterintuitive when one thinks about the problem from the standpoint of cue integration. As we have shown previously (<xref ref-type="bibr" rid="bib18">Gu et al. 2008</xref>; <xref ref-type="bibr" rid="bib16">Fetsch et al. 2011</xref>), decoding both congruent and opposite cells is suboptimal when the goal is simply to perform heading discrimination. However, we believe that other computational goals come into play, specifically the problem of dissociating self-motion from object motion. This is now discussed in more detail; decoding a mixed population of congruent and opposite cells may provide a near optimal solution to the problem of estimating heading in the presence of objects, which can be framed as a marginalization problem. In this light, such a decoding scheme becomes less surprising. We are currently working on another manuscript that describes these issues in much greater detail.</p><p><italic>He/she wonders whether the same simulations could be used to test the implications of this sort of decoding on the amount of information extracted about the stimulus. That is, does decoding only according to vestibular preference account for any sub-optimalties in the monkey's behavior? Such a result would provide more evidence for this decoding algorithm</italic>.</p><p>We are unsure as to exactly what sort of sub-optimality the reviewer has in mind here, but we can say something about optimality of cue integration. As shown previously (<xref ref-type="bibr" rid="bib16">Fetsch et al. 2011</xref>), incorporating opposite cells into the decoding impairs cue integration performance. On the other hand, including opposite cells adds robustness to object motion, and we have shown (in preliminary form) that titrating the proportion of opposite cells (i.e., the Readout Index) allows one to tradeoff cue integration against robustness to object motion (<xref ref-type="bibr" rid="bib24">Kim et al., 2014</xref>). We now make this point in the Discussion and this will be explored in detail in a manuscript that is currently in preparation. Whether such a tradeoff in optimality of cue-integration would be observed behaviorally in the presence of moving objects is something that we cannot address at this time, but is something that we would like to study in the future.</p><p>Reviewer #3:</p><p><italic>This reviewer considers that the clear cut difference between the two models can only be formulated within the context of the Shadlen et al 96 model (recently revised by Haeffner et al), which the authors seem to take a ground truth, but which is itself a useful but fairly crude and simplistic picture of how the decision is being made) i.e. totally feedforward, no dynamics, etc)</italic>.</p><p>It is true that our simulations do not specifically embody dynamics or feedback. However, our models are also not incompatible with such features. We simply compute a likelihood function over heading from the population responses of our model neurons. The correlated noise in our population responses might arise from either feedforward or feedback sources, and we are agnostic about that. Under the assumptions we make (see Methods), our decoder is optimal in the maximum likelihood sense. In this regard, we do not necessarily see our simulations as “formulated within the context of the Shadlen et al. 96 model”. That model adopts a specific pooling strategy for decoding responses, which is substantially suboptimal. We do not assume a specific pooling strategy but rather simply compute the likelihood over heading. We have modified the text to clarify this issue.</p><p><italic>Secondly even if one accepted that the decision making model is valid, the analysis does not allow the authors to draw general enough conclusions. Conceivably, one would set oneself to answer the question of whether it is possible to rule out the possibility than incongruent neurons do not contribute at all to the decision, regardless of all other modeling choices. The reviewer wasn't convinced by the analysis that this is the case. There are 3 main pieces of evidence: the patterns of CCs in</italic> <xref ref-type="fig" rid="fig3"><italic>Figure 3</italic></xref><italic>, the patterns of CPs in</italic> <xref ref-type="fig" rid="fig5"><italic>Figure 5</italic></xref> <italic>and the psychophysical data in</italic> <xref ref-type="fig" rid="fig6"><italic>Figure 6</italic></xref>.</p><p>We agree that it is important to address the robustness of the conclusions. We have implemented new analyses to address this issue as detailed below.</p><p><italic>In</italic> <xref ref-type="fig" rid="fig3"><italic>Figure 3</italic></xref><italic>, using an all-or-none approach where r_noise is ‘only’ a function of vestibular r signal, the data seems to favor the selective decoding model. However, (a) there are only 10 mixed pairs and (b)</italic> it is not at all clear that having <italic>r_noise be ‘mainly’ due to vestibular r_signal but with some contribution from visual r_signal, the two models would not look more similar to themselves and to data. This is a general concern: Even assuming the framework is valid, a simplified situation (in term of model parameters) is used to make categorical distinctions, and so it's not clear whether the conclusions are generic or if they only applied to the simplified situation</italic>.</p><p>We have substantially improved this part of the analysis. In addition to just comparing the small samples of neurons that were classified as matched vs mismatched congruency, we now perform a model comparison using the data from all 127 pairs of neurons, which gives us substantially greater power. Specifically, we fit the data with two models: r_noise = a*r_signal_vestibular, and r_noise = a*r_signal_vestibular + b*r_signal_visual. Accounting for the extra free parameter, we find that the data highly favor the latter model which is consistent with the assumption of the selective decoding model (p=0.0003, sequential F-test). This section of the text has been extensively revamped, and the new results now appear in a modified <xref ref-type="fig" rid="fig3">Figure 3</xref>, while the original <xref ref-type="fig" rid="fig3">Figure 3a, b</xref> have become figure supplements.</p><p><italic>The evidence from</italic> <xref ref-type="fig" rid="fig5"><italic>Figure 5</italic></xref> <italic>is also not shown to be conclusive/generic. The bi-modality in</italic> <xref ref-type="fig" rid="fig5"><italic>Figure 5b</italic></xref> <italic>is not explained clearly and so it's not clear what exactly is necessary to obtain bi-modality. Is there no model with zero weights from incongruent neurons able to show unimodal CPs? Are there no parameters from the selective decoding model which would result in bi-modality? To make things worse, the selective decoding model seems to have one more degree of freedom (Readout Index)</italic>.</p><p>This is a fair point and we agree that it is important for the robustness of the different predictions to be addressed. We have now examined the bimodality of the predicted CP distributions for each model while we vary the key parameter of each model. The results show that these predictions are rather robust over a range of parameters of the two models that are consistent with the constraints implied by our other analyses as well. This is now addressed in a new paragraph as well as two new figure supplements (<xref ref-type="fig" rid="fig5s1 fig5s2">Figure 5–figure supplements 1 and 2</xref>).</p><p><italic>Similar concerns of generality apply to the analysis in</italic> <xref ref-type="fig" rid="fig6"><italic>Figure 6</italic></xref>.</p><p><italic>The authors conclude that both noise correlations and selective weighting probably contribute to the measured CPs. But this is hardly surprising (again, even assuming the whole framework is valid)!</italic></p><p>The parametric analysis of <xref ref-type="fig" rid="fig6">Figure 6</xref>, combined with the new parametric analyses of <xref ref-type="fig" rid="fig5s1 fig5s2">Figure 5–figure supplements 1 and 2</xref> suggests that only the selective decoding model accounts for all of the data reasonably well. Moreover, the range of Readout Index values that account well for the behavioral thresholds matches very well with the range (values around 0.5) that accounts well for the distributions of choice probabilities. Thus, we feel that the overall pattern of results provides considerable evidence that selective decoding makes a contribution to the choice probabilities that we observe experimentally.</p><p><italic>In summary, the paper attempts to make a relatively minor distinction (i.e., to refute a model that represents a small region of parameter space (CPs are a result of only CCs) compared to the alternatives (CPs are a result of both CCs and weights)) and it does so it not in general terms, which limits the potential impact of the work</italic>.</p><p>We think that the additional analyses described above, as well as the clarified text in a number of places, have improved the manuscript substantially. Moreover, we do not agree that this represents a “minor distinction”. It has been argued in the literature that CPs may be determined solely by how neurons are correlated with each other, and that CPs may not at all reflect how neurons are decoded (we have colleagues who believe this very strongly). So the key issue at stake here, as we have now clarified in the manuscript (see other replies above), is whether there is evidence that the pattern of CPs in influenced by the manner in which neurons are decoded, and we think that the balance of evidence we present makes a reasonably strong argument in favor of this. As also discussed in the text, we think that our particular model system gives us additional power to test these predictions, due to the fact that opposite cells can be decoded either in favor of their visual or vestibular heading preference.</p></body></sub-article></article> |