review: Starting Point of Research

[mewmew]

Identify a rough starting point for the meta-study research.

Which papers are key within this field?

• ROBIN: Kohonen, T., et al. "A principle of neural associative memory." Neuroscience 2.6 (1977): 1065-1076. (issue review: Kohonen, "A principle of neural associative memory" #15)
• Kohonen, Teuvo. Self-organization and associative memory. Vol. 8. Springer Science & Business Media, 2012.
• Kohonen, Teuvo. Associative memory: A system-theoretical approach. Vol. 17. Springer Science & Business Media, 2012.
• ROBIN: Hinton, Geoffrey E., and James A. Anderson. Parallel Models of Associative Memory: Updated Edition. Psychology press, 2014. (issue review: Hinton et al., "Parallel Models of Associative Memory" #16)
• LUCAS: McEliece, Robert J., et al. "The capacity of the Hopfield associative memory." Information Theory, IEEE Transactions on 33.4 (1987): 461-482.
• ROBIN: Bohland, Jason W., and Ali A. Minai. "Efficient associative memory using small-world architecture." Neurocomputing 38 (2001): 489-496. (issue review: Bohland et al., "Efficient associative memory using small-world architecture" #17)
• ROBIN: Nugent, Michael Alexander, and Timothy Wesley Molter. "AHaH Computing–From Metastable Switches to Attractors to Machine Learning." PloS one 9.2 (2014): e85175.
• LUCAS: Giles, C. Lee, and Tom Maxwell. "Learning, invariance, and generalization in high-order neural networks." Applied optics 26.23 (1987): 4972-4978.
• ROBIN: Nakano, Kaoru. "Associatron-a model of associative memory." Systems, Man and Cybernetics, IEEE Transactions on 3 (1972): 380-388. (issue review: Nakano et al., "Associatron - a model of associative memory" #18)
• Cao, Jinde, and Qiankun Song. "Stability in Cohen? Grossberg-type bidirectional associative memory neural networks with time-varying delays." Nonlinearity 19.7 (2006): 1601.
• WENTING: Hopfield, John J. "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the national academy of sciences 79.8 (1982): 2554-2558.
• Brown, Martin, and Christopher John Harris. "Neurofuzzy adaptive modelling and control." (1994).
• WENTING: Psaltis, Demetri, and Nabil Farhat. "Optical information processing based on an associative-memory model of neural nets with thresholding and feedback." Optics Letters 10.2 (1985): 98-100.
• LUCAS: Carpenter, Gail A. "Neural network models for pattern recognition and associative memory." Neural networks 2.4 (1989): 243-257.
• ROBIN: Barto, Andrew G., Richard S. Sutton, and Peter S. Brouwer. "Associative search network: A reinforcement learning associative memory." Biological cybernetics 40.3 (1981): 201-211. (issue review: Barto et al., "Associative search network: A reinforcement learning associative memory" #19)
• Raaijmakers, Jeroen GW, and Richard M. Shiffrin. "SAM: A theory of probabilistic search of associative memory." The psychology of learning and motivation: Advances in research and theory 14 (1981): 207-262.
• Liao, Xiaofeng, and Juebang Yu. "Qualitative analysis of Bi‐directional Associative Memory with time delay." International Journal of Circuit Theory and Applications 26.3 (1998): 219-229.
• WENTING: Yoshizawa, Shuji, Masahiko Morita, and Shun-Ichi Amari. "Capacity of associative memory using a nonmonotonic neuron model." Neural Networks 6.2 (1993): 167-176.
• Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, Kilian Weinberger. "Deep Networks with Stochastic Depth", 2016
• ...

Have other meta-studies been done within this field? Which papers did they refer to?

• ROBIN: Arel, Itamar, Derek C. Rose, and Thomas P. Karnowski. "Deep machine learning-a new frontier in artificial intelligence research [research frontier]." Computational Intelligence Magazine, IEEE 5.4 (2010): 13-18. (issue review: Arel et al., "Deep machine learning-a new frontier in artificial intelligence research [research frontier]" #20)
• ROBIN: Palm, Günther. "On associative memory." Biological cybernetics 36.1 (1980): 19-31. (issue review: Palm et al., "On associative memory" #21)
• WENTING: Kan, Irene P., et al. "Implicit memory for novel associations between pictures: effects of stimulus unitization and aging." Memory & cognition 39.5 (2011): 778-790.
• ...

Interesting papers from other disciplines. What does neuroscience have to say about associative memories?

• Fanselow, Michael S., and Andrew M. Poulos. "The neuroscience of mammalian associative learning." Annu. Rev. Psychol. 56 (2005): 207-234.
• Gabrieli, John DE. "Cognitive neuroscience of human memory." Annual review of psychology 49.1 (1998): 87-115.
• Reijmers, Leon G., et al. "Localization of a stable neural correlate of associative memory." Science 317.5842 (2007): 1230-1233.
• Hasselmo, Michael E., et al. "A model of the hippocampus combining self-organization and associative memory function." Advances in neural information processing systems (1995): 77-84.
• Lytton, William W., and Peter Lipton. "Can the hippocampus tell time? The temporo‐septal engram shift model." Neuroreport 10.11 (1999): 2301-2306.
• Biological Aspects of learning, memory formation and ontogeny of the CNS. 1977.
• Rahmann, Hinrich, and Mathilde Rahmann. The neurobiological basis of memory and behavior. Springer Science & Business Media, 2012.
• Amari, Shun-Ichi, and Kenjiro Maginu. "Statistical neurodynamics of associative memory." Neural Networks 1.1 (1988): 63-73.
• ROBIN: Pershin, Yuriy V., and Massimiliano Di Ventra. "Experimental demonstration of associative memory with memristive neural networks." Neural Networks 23.7 (2010): 881-886. (issue review: [3h] Pershin et al., "Experimental demonstration of associative memory with memristive neural networks" #22)
• Wang, DeLiang, Joachim Buhmann, and Christoph von der Malsburg. "Pattern segmentation in associative memory." Neural Computation 2.1 (1990): 94-106.
• Nakazawa, Kazu, et al. "Requirement for hippocampal CA3 NMDA receptors in associative memory recall." Science 297.5579 (2002): 211-218.
• Ranganath, Charan, et al. "Inferior temporal, prefrontal, and hippocampal contributions to visual working memory maintenance and associative memory retrieval." The Journal of Neuroscience 24.16 (2004): 3917-3925.
• Levy, William B., and Oswald Steward. "Synapses as associative memory elements in the hippocampal formation." Brain research 175.2 (1979): 233-245.
• ROBIN: Lansner, Anders. "Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations." Trends in neurosciences 32.3 (2009): 178-186. (issue review: Lansner et al., "Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations" #23)
• Doyère, Valérie, and Serge Laroche. "Linear relationship between the maintenance of hippocampal long‐term potentiation and retention of an associative memory." Hippocampus 2.1 (1992): 39-48.
• Staresina, Bernhard P., and Lila Davachi. "Object unitization and associative memory formation are supported by distinct brain regions." The Journal of Neuroscience 30.29 (2010): 9890-9897.
• LUCAS: Amari, Shun-Ichi. "Characteristics of sparsely encoded associative memory." Neural Networks 2.6 (1989): 451-457.
• Gibson, William G., and John Robinson. "Statistical analysis of the dynamics of a sparse associative memory." Neural Networks 5.4 (1992): 645-661.
• Maren, Stephen. "Synaptic mechanisms of associative memory in the amygdala." Neuron 47.6 (2005): 783-786.
• ...

What about psychology (cognitive psychology, biological psychology, ...)?

• McClelland, James L., Bruce L. McNaughton, and Randall C. O'Reilly. "Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory." Psychological review 102.3 (1995): 419.
• Anderson, John R., and Gordon H. Bower. Human associative memory. Psychology press, 2014.
• Srull, Thomas K., Meryl Lichtenstein, and Myron Rothbart. "Associative storage and retrieval processes in person memory." Journal of Experimental Psychology: Learning, Memory, and Cognition 11.2 (1985): 316.
• ...

Other fields

• Paek, Eung G., and Demetri Psaltis. "Optical associative memory using Fourier transform holograms." Optical Engineering 26.5 (1987): 265428-265428.
• ...

Please add to this list as you find more resources that may be a good starting point for the meta-study research.

[mewmew]

@Chilinot, @chrisking2020 and @slax32, please select the research literature you wish to review, and comment in this issue to tell the others which ones you took, so that we don't do too much redundant work. (I've marked the papers which I've printed and started skimming through with the subtle ROBIN: prefix above)

We should try to conduct a preliminary literature review (basically skimming the main research sources) before handing in the project plan on Monday next week. The Project Proposal Milestone tracks each task we should complete before Monday. Feel free to open new issues, tracking sub-tasks and assign them to the relevant milestone. This way I hope we may keep better track of time.

[Chilinot]

I have now selected the papers to read until monday in the original comment :)

[mewmew]

@Chilinot, perfect! Just finished reading the first paper ("Experimental demonstration of associative memory with memristive neural networks."), and it was very interesting! They managed to create artificial synapses using resistors with microprocessors to store and update the resistive properties over time. Using these they managed to show experimentally the formation of associative memory, as the firing of two previously uncorrelated synapses became correlated after firing together. They also go on the mention the underlying rule behind how associative memory is believed to be formed, namely the Hebbian rule stating

"Neurons that fire together, wire together"

[mewmew]

Lets try to find papers through other search engines than Google scholar. For instance the library may be a good place to start. Looking for other meta-study papers may help us get a good overview of the field, needed to identify the key models of associative memory (see issue #25).

[mewmew]

Closing this issue for now, as the literature review has been completed. Some research papers were skimmed, others read with great care. Even the papers which were not included in the final literature helped give an high-level picture and understanding of what's out there, and to get a grasp of the interdisciplinary field of models for associative memory.

In the future, it may be interesting to refer back to these research papers to gain a deeper understanding across the different fields of research.

[mewmew]

Retrospectively, our starting point of research was heavily based on Google scholar search results, in combination with Google and Wikipedia search results. It would be interesting to find what research literature that was censored by our filter bubbles.

Personally, I was sad to find out that the research search capabilities of the University of Portsmouth's library were not open to non-faculty members or students of UoP. This was a real shame, as those capabilities are astounding in their quality and manage to provide a good complementary service to be used along side of Google scholar. I asked the staff members working at the library of UoP to elaborate on why this was the case, and if it was intentional. Sadly, for the time being, it does not seem like they will update their policy in this regard, thus limiting access to an otherwise wonderful resource.

I have yet to fully discover the capabilities provided in this regard by UU. Will take a deeper look for future research projects.