Skip to content

Latest commit

 

History

27 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 

Repository files navigation

🧠 Metacognition in LLMs:
Foundations, Progress, and Opportunities

arXiv

License: MIT GitHub stars

This repository provides an organized collection of papers related to metacognition in LLMs, reviewed in our paper Metacognition in LLMs: Foundations, Progress, and Opportunities. We comprehensively and systematically taxonomize the landscape of this emerging field, summarizing existing methods and benchmarks to measure and elicit LLMs' metacognitive abilities, techniques to develop, improve, and apply models' metacognitive skills, and findings and implications of ongoing research. We also discuss applications, open challenges, and promising directions for future work. We hope this can serve as a detailed, up-to-date reference and stimulate meaningful research and discussion.

🌟 Citation

If you find this paper or repository useful for your research, please cite:

@misc{liu2026metacognition,
      title={Metacognition in LLMs: Foundations, Progress, and Opportunities}, 
      author={Gabrielle Kaili-May Liu and Areeb Gani and Jacqueline Lu and Jordan Thomas and Mark Steyvers and Arman Cohan},
      journal={arXiv preprint arXiv:2607.11881},
      year={2026},
      url={https://arxiv.org/abs/2607.11881}, 
}

📧 Contact

Feel free to open an issue or contact us if you have any feedback or want to include your work in this list!

Corresponding Author: Kaili Liu (kaili.liu@yale.edu)

📖 Table of Contents

🗺️ Overview

Taxonomy of research on metacognition in LLMs

Taxonomy of current research on metacognition in LLMs, as organized in our paper.
Metacognition describes the capacity for a system to assess and regulate its own cognition. The metacognitive loop consists of two interacting processes: monitoring (e.g., forming judgments of uncertainty, task performance, and progress, among other internal states) and control (e.g., engaging in planning, strategy selection, or effort re-allocation based on monitoring).

📄 Paper List

🧠 0. Metacognition Background

Metacognition is a foundational component of intelligence that is critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems.

Fundamentals

  1. Metacognition and Cognitive Monitoring: A New Area of Cognitive–Developmental Inquiry, Flavell, American Psychologist 1979. [Paper]
  2. Feeling of Knowing in Memory and Problem Solving, Metcalfe, Journal of Experimental Psychology: Learning, Memory, and Cognition 1986. [Paper]
  3. Metamemory: A Theoretical Framework and New Findings, Nelson, Psychology of Learning and Motivation 1990. [Paper]
  4. Metacognition and Awareness, Kentridge, Consciousness and Cognition 2000. [Paper]
  5. Metacognition and Learning, McCormick, Handbook of Psychology 2003. [Paper]
  6. Metacognition and Learning: Conceptual and Methodological Considerations, Veenman et al., Metacognition and Learning 2006. [Paper]
  7. Metacognitive Aspects of Memory, Koriat and Helstrup, Everyday Memory (Book Chapter) 2007. [Paper]
  8. Metacognition, Dunlosky and Metcalfe, Sage Publications (Book) 2008. [Paper]
  9. Metacognition: Computation, Biology and Function, Fleming et al., Philosophical Transactions of the Royal Society B 2012. [Paper]
  10. Handbook of Metamemory and Memory, Dunlosky and Bjork, Psychology Press (Book) 2013. [Paper]
  11. Metacognition: Decision Making Processes in Self-Monitoring and Self-Regulation, Koriat, Wiley Blackwell Handbook of Judgment and Decision Making 2015. [Paper]
  12. Metacognitive Theories Revisited, Moshman, Educational Psychology Review 2018. [Paper]
  13. Metacognition and Confidence: A Review and Synthesis, Fleming, Annual Review of Psychology 2024. [Paper]

Mechanisms

  1. The Neural Basis of Metacognitive Ability, Fleming and Dolan, Philosophical Transactions of the Royal Society B 2012. [Paper]
  2. Domain-Specific and Domain-General Processes Underlying Metacognitive Judgments, Fitzgerald et al., Consciousness and Cognition 2017. [Paper]
  3. Domain-General and Domain-Specific Patterns of Activity Supporting Metacognition in Human Prefrontal Cortex, Morales et al., Journal of Neuroscience 2018. [Paper]
  4. Towards a Common Conceptual Space for Metacognition in Perception and Memory, Mazancieux et al., Nature Reviews Psychology 2023. [Paper]

Measuring Metacognition

  1. Confidence and Accuracy of Near-Threshold Discrimination Responses, Kunimoto et al., Consciousness and Cognition 2001. [Paper]
  2. A Conceptual Analysis of Five Measures of Metacognitive Monitoring, Schraw, Metacognition and Learning 2009. [Paper]
  3. Literature Review on Metacognition and Its Measurement, Akturk and Sahin, Procedia-Social and Behavioral Sciences 2011. [Paper]
  4. A Signal Detection Theoretic Approach for Estimating Metacognitive Sensitivity from Confidence Ratings, Maniscalco and Lau, Consciousness and Cognition 2012. [Paper]
  5. Measures of Metacognition on Signal-Detection Theoretic Models, Barrett et al., Psychological Methods 2013. [Paper]
  6. How to Measure Metacognition, Fleming and Lau, Frontiers in Human Neuroscience 2014. [Paper]
  7. Metacognitive Information Theory, Dayan, Open Mind 2023. [Paper]
  8. Common Computations for Metacognition and Meta-Metacognition, Zheng et al., Neuroscience of Consciousness 2023. [Paper]
  9. Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance, Meyen et al., Open Mind 2025. [Paper]

Connections to Consciousness

  1. Metacognition and Consciousness, Newton, Pragmatics & Cognition 1995. [Paper]
  2. Conscious and Unconscious Metacognition: A Rejoinder, Koriat and Levy-Sadot, Consciousness and Cognition 2000. [Paper]
  3. Consciousness and Metacognition, Rosenthal, Metarepresentation (Book Chapter) 2000. [Paper]
  4. Metacognition in Strategy Selection: Giving Consciousness Too Much Credit, Cary and Reder, Metacognition: Process, Function and Use (Book Chapter) 2002. [Paper]
  5. Consciousness, Metacognition, and the Unconscious, Winkielman and Schooler, The Sage Handbook of Social Cognition 2012. [Paper]

Confidence, Decisions, & Other Findings

  1. A Revised Methodology for Research on Metamemory: Pre-Judgment Recall and Monitoring (PRAM), Nelson et al., Psychological Methods 2004. [Paper]
  2. Logic, Self-Awareness and Self-Improvement: The Metacognitive Loop and the Problem of Brittleness, Anderson and Perlis, Journal of Logic and Computation 2005. [Paper]
  3. Competence and Control Beliefs: Distinguishing the Means and Ends, Schunk and Zimmerman, Handbook of Educational Psychology 2006. [Paper]
  4. The Dunning–Kruger Effect: On Being Ignorant of One's Own Ignorance, Dunning, Advances in Experimental Social Psychology 2011. [Paper]
  5. Thinking, Fast and Slow, Kahneman, Macmillan (Book) 2011. [Paper]
  6. Higher Order Thoughts in Action: Consciousness as an Unconscious Re-Description Process, Timmermans et al., Philosophical Transactions of the Royal Society B 2012. [Paper]
  7. Meta-Reasoning: Monitoring and Control of Thinking and Reasoning, Ackerman and Thompson, Trends in Cognitive Sciences 2017. [Paper]
  8. But I Was So Sure! Metacognitive Judgments Are Less Accurate Given Prospectively Than Retrospectively, Siedlecka et al., Frontiers in Psychology 2016. [Paper]
  9. Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation, Fleming and Daw, Psychological Review 2017. [Paper]
  10. Closed-Loop Brain Training: The Science of Neurofeedback, Sitaram et al., Nature Reviews Neuroscience 2017. [Paper]
  11. How Multiple Levels of Metacognitive Awareness Operate in Collaborative Problem Solving, Cini et al., Metacognition and Learning 2023. [Paper]
  12. Bits of Confidence: Metacognition as Uncertainty Reduction, Fitousi, Psychonomic Bulletin & Review 2025. [Paper]

Education & Learning

  1. A Metacognitive View of Individual Differences in Self-Regulated Learning, Winne, Learning and Individual Differences 1996. [Paper]
  2. Motivational Skills Training: Combining Metacognitive, Cognitive, and Affective Learning Strategies, McCombs, Learning and Study Strategies (Book Chapter) 1988. [Paper]
  3. Promoting General Metacognitive Awareness, Schraw, Instructional Science 1998. [Paper]
  4. Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments, Kruger and Dunning, Journal of Personality and Social Psychology 1999. [Paper]
  5. Enhancing Mathematical Reasoning in the Classroom: The Effects of Cooperative Learning and Metacognitive Training, Kramarski and Mevarech, American Educational Research Journal 2003. [Paper]
  6. A Theoretical Framework and Approach for Fostering Metacognitive Development, White and Frederiksen, Educational Psychologist 2005. [Paper]
  7. Metacognitive Knowledge Monitoring and Self-Regulated Learning, Isaacson and Fujita, Journal of the Scholarship of Teaching and Learning 2006. [Paper]
  8. Metacognitive Awareness and Academic Achievement in College Students, Young and Fry, Journal of the Scholarship of Teaching and Learning 2008. [Paper]
  9. The Importance of Self-Regulation for College Student Learning, Cohen, APA PsycNet 2012. [Paper]
  10. The Role of Metacognition in Human Social Interactions, Frith, Philosophical Transactions of the Royal Society B 2012. [Paper]
  11. Development and Evaluation of Metacognition in Early Childhood Education, Chatzipanteli et al., Early Child Development and Care 2014. [Paper]
  12. Can the Use of Cognitive and Metacognitive Self-Regulated Learning Strategies Be Predicted by Learners' Levels of Prior Knowledge in Hypermedia-Learning Environments?, Taub et al., Computers in Human Behavior 2014. [Paper]
  13. Strategic Resource Use for Learning: A Self-Administered Intervention That Guides Self-Reflection on Effective Resource Use Enhances Academic Performance, Chen et al., Psychological Science 2017. [Paper]
  14. Domain-general enhancements of metacognitive ability through adaptive training, Carpente et al., Journal of Experimental Psychology: General 2019. [Paper]
  15. Metacognition: Ideas and Insights from Neuro- and Educational Sciences, Fleur et al., npj Science of Learning 2021. [Paper]
  16. Metacognitive Feelings as a Source of Information for the Creative Process: A Conceptual Exploration, Puente-Diaz, Journal of Intelligence 2023. [Paper]
  17. Experiencing Hallucinations in Daily Life: The Role of Metacognition, Wright et al., Schizophrenia Research 2024. [Paper]

Connections to AI

  1. Modeling Metacognition for Learning in Artificial Systems, Josyula et al., NaBIC 2009. [Paper]
  2. From Internal Models Toward Metacognitive AI, Kawato and Cortese, Biological Cybernetics 2021. [Paper]
  3. Fast, Slow, and Metacognitive Thinking in AI, Ganapini et al., npj Artificial Intelligence 2025. [Paper]

🔎 1. Measuring Metacognition in LLMs

Psychologically-Grounded Methods

  1. Measuring the Metacognition of AI, Servajean and Servajean, arXiv 2026. [Paper]
  2. Metacognitive Sensitivity for Test-Time Dynamic Model Selection, Trinh et al., CogInterp Workshop 2026. [Paper]
  3. Judgments of Learning Distinguish Humans from Large Language Models in Predicting Memory, Huff and Ulakci, Scientific Reports 2025. [Paper]
  4. Do LLMs Dream of Electric Emotions? Towards Quantifying Metacognition and Generalizing the Teacher-Student Model Using Ensembles of LLMs, Sethi et al., CIKM 2025. [Paper]
  5. Decoupling Metacognition from Cognition: A Framework for Quantifying Metacognitive Ability in LLMs, Wang et al., AAAI 2025. [Paper]
  6. Metacognitive Monitoring: A Human Ability Beyond Generative Artificial Intelligence, Huff and Ulakci, arXiv 2024. [Paper]

Neurofeedback-Based Methods

  1. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations, Li et al., NeurIPS 2026. [Paper]
  2. Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models, Yalon et al., arXiv 2026. [Paper]

Confidence-Based Methods

  1. Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models, Lu et al., NeurIPS 2026. [Paper]
  2. Large Language Models Have Intrinsic Meta-Cognition, but Need a Good Lens, Ma et al., EMNLP 2025. [Paper]

Interpretability-Based Methods

  1. Towards Understanding Metacognition in Large Reasoning Models, Li et al., OpenReview 2026. [Paper]
  2. Emergent Introspective Awareness in Large Language Models, Lindsey, arXiv 2026. [Paper]
  3. Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger, Li et al., ACL 2025. [Paper]

Task-Specific Methods

  1. Evidence for Limited Metacognition in LLMs, Ackerman, arXiv 2025. [Paper]
  2. Large Linguistic Models: Investigating LLMs' Metalinguistic Abilities, Begus et al., IEEE Transactions on Artificial Intelligence 2025. [Paper]
  3. Does It Make Sense to Speak of Introspection in Large Language Models?, Comsa and Shanahan, arXiv 2025. [Paper]
  4. When Two LLMs Debate, Both Think They'll Win, Prasad and Nguyen, arXiv 2025. [Paper]
  5. Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving, Didolkar et al., NeurIPS 2024. [Paper]
  6. Beyond Traditional AI IQ Metrics: Metacognition and Benchmarking for LLMs, AGI, and ASI, Uzwyshyn, ResearchGate 2024. [Paper]

Benchmarks

  1. The metacognitive monitoring battery: A cross-domain benchmark for LLM self-monitoring, Cacioli et al., arXiv 2026. [Paper]
  2. Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs, Fan et al., The Web Conference 2026. [Paper]
  3. Me, Myself, and π : Evaluating and Explaining LLM Introspection, Naphade et al., arXiv 2026. [Paper]
  4. Large Language Models Lack Essential Metacognition for Reliable Medical Reasoning, Griot et al., Nature Communications 2025. [Paper]
  5. ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge Under Multi-Turn Jailbreaks, Kim et al., arXiv 2025. [Paper]
  6. Awareness in LLMs Improves Through Collaboration, Passaro et al., EurIPS Workshop on Metacognition in Generative AI 2025. [Paper]
  7. From Remembering to Metacognition: Do Existing Benchmarks Accurately Evaluate LLMs?, Zhang et al., EMNLP Findings 2025. [Paper]

📊 2. Current Findings on Metacognition in LLMs

  1. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations, Li et al., NeurIPS 2026. [Paper]
  2. Emergent Introspective Awareness in Large Language Models, Lindsey, arXiv 2026. [Paper]
  3. Emergent Mechanisms of Self-Awareness in LLMs, Bozoukov et al., XAI4Science Workshop (NeurIPS) 2025. [Paper]
  4. Looking Inward: Language Models Can Learn About Themselves by Introspection, Binder et al., ICLR 2025. [Paper]
  5. Do Large Language Models Know How Much They Know?, Prato et al., EMNLP 2024. [Paper]
  6. Feeling the Strength but Not the Source: Partial Introspection in LLMs, Hahami et al., arXiv 2025. [Paper]
  7. Judgments of Learning Distinguish Humans from Large Language Models in Predicting Memory, Huff and Ulakci, Scientific Reports 2025. [Paper]
  8. Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models, Li et al., LREC-COLING 2024. [Paper]
  9. Meta-Memory for Large Language Models, Liang et al., IEEE Transactions on Audio, Speech and Language Processing 2026. [Paper]
  10. Decoupling Metacognition from Cognition: A Framework for Quantifying Metacognitive Ability in LLMs, Wang et al., AAAI 2025. [Paper]
  11. MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs, Liu et al., EMNLP 2025. [Paper]
  12. Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence? Liu and Cohan, arXiv 2026. [Paper]
  13. Quantifying Faithful Confidence Expression in Large Reasoning Models, Liu et al., arXiv 2026. [Paper]
  14. Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs, Liu et al., arXiv 2026. [Paper]
  15. Metacognition and Uncertainty Communication in Humans and Large Language Models, Steyvers and Peters, Current Directions in Psychological Science 2025. [Paper]
  16. What Large Language Models Know and What People Think They Know, Steyvers et al., Nature Machine Intelligence 2025. [Paper]
  17. Metacognitive Monitoring: A Human Ability Beyond Generative Artificial Intelligence, Huff and Ulakci, arXiv 2024. [Paper]
  18. Large Language Models Lack Essential Metacognition for Reliable Medical Reasoning, Griot et al., Nature Communications 2025. [Paper]
  19. Quantifying Uncert-AI-nty: Testing the Accuracy of LLMs' Confidence Judgments, Cash et al., Memory & Cognition 2026. [Paper]
  20. Can LLMs Estimate Cognitive Complexity of Reading Comprehension Items?, Hwang et al., arXiv 2025. [Paper]
  21. When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents, Advani, arXiv 2026. [Paper]
  22. Do Large Language Models Know What They Are Capable Of?, Barkan et al., arXiv 2025. [Paper]
  23. Tell Me About Yourself: LLMs Are Aware of Their Learned Behaviors, Betley et al., arXiv 2025. [Paper]
  24. Do AI Know What They Know? Exploring Metacognition in LLMs, Iqbal, Intelligent Data Analysis 2023. [Paper]
  25. Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries, Kale and Nadadur, TrustNLP Workshop (ACL) 2025. [Paper]
  26. From Human to Model Overconfidence: Evaluating Confidence Dynamics in Large Language Models, Wen et al., NeurIPS Workshop on Behavioral Machine Learning 2024. [Paper]
  27. Causal Evidence that Language Models use Confidence to Drive Behavior, Kumaran et al., arXiv 2026. [Paper]
  28. Cognitive Foundations for Reasoning and Their Manifestation in LLMs, Karapugta et al., HCAIR Workshop (ICLR) 2026. [Paper]
  29. Beyond Confidence: Rethinking Self-Assessments for Performance Prediction in LLMs, Bhattacharyya et al., arXiv 2026. [Paper]
  30. MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Models, Wang et al., arXiv 2026. [Paper]
  31. LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling, Cao et al., arXiv 2026. [Paper]
  32. Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions, Huang et al., arXiv 2026. [Paper]
  33. LLMs as Signal Detectors: Sensitivity, Bias, and the Temperature-Criterion Analogy, Cacioli, arXiv 2026. [Paper]
  34. Rescaling Confidence: What Scale Design Reveals About LLM Metacognition, Dai, arXiv 2026. [Paper]
  35. Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models, Chhikara, arXiv 2025. [Paper]
  36. Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI, Kaur et al., NeurIPS Safe Generative AI Workshop 2024. [Paper]
  37. Worth the Weight: Modern LLMs Demonstrate Accurate Metacognitive Knowledge of Decision Weights in Multi-Attribute Choice, Cash and Oppenheimer, PsyArXiv 2025. [Paper]
  38. What Do LLM Agents Do When Left Alone? Evidence of Spontaneous Meta-Cognitive Patterns, Szeider, arXiv 2025. [Paper]
  39. Generative AI as a Metacognitive Agent: A Comparative Mixed-Method Study with Human Participants on ICF-Mimicking Exam Performance, Pavlovic et al., arXiv 2024. [Paper]
  40. Introspective Machines: Are LLMs Better at Self-Reflection Than Humans?, Cappelen and Dever, Philosophical Perspectives 2024. [Paper]
  41. Privileged Self-Access Matters for Introspection in AI, Song et al., arXiv 2025. [Paper]
  42. System Prompt Learning in Large Language Models: Cross-Disciplinary Parallels with Human Cognition, Pajo, ResearchGate 2025. [Paper]
  43. Language Models Fail to Introspect About Their Knowledge of Language, Song et al., arXiv 2025. [Paper]
  44. When Two LLMs Debate, Both Think They'll Win, Prasad and Nguyen, arXiv 2025. [Paper]
  45. Metacognitive Myopia in Large Language Models, Scholten et al., arXiv 2024. [Paper]
  46. Towards Understanding the Cognitive Habits of Large Reasoning Models, Dong et al., arXiv 2025. [Paper]
  47. From Latent Signals to Reflection Behavior: Tracing Meta-Cognitive Activation Trajectory in R1-Style LLMs, Du et al., arXiv 2026. [Paper]
  48. Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs, Eisenstadt et al., arXiv 2025. [Paper]
  49. Cognitive Foundations for Reasoning and Their Manifestation in LLMs, Kargupta et al., arXiv 2025. [Paper]
  50. Reasoning Models Generate Societies of Thought, Kim et al., arXiv 2026. [Paper]
  51. Understanding R1-Zero-Like Training: A Critical Perspective, Liu et al., arXiv 2025. [Paper]
  52. Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models, Lu et al., NeurIPS 2026. [Paper]
  53. DeepSeek-R1 Thoughtology: Let's Think About LLM Reasoning, Marjanovic et al., arXiv 2025. [Paper]
  54. A Trade-Off Between Reasoning Ability and Metacognitive Sensitivity in Large Language Models, Sha et al., PsyArXiv 2026. [Paper]

🛠️ 3. Implementing Metacognition in LLMs

Metacognition for LLMs

  1. Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs, Fan et al., The Web Conference 2026. [Paper]
  2. Fine-Tuning Language Models to Know What They Know, Park et al., arXiv 2026. [Paper]
  3. State Stream Transformer (SST): Emergent Metacognitive Behaviours Through Latent State Persistence, Aviss, arXiv 2025. [Paper]
  4. Pangu Embedded: An Efficient Dual-System LLM Reasoner with Metacognition, Chen et al., arXiv 2025. [Paper]
  5. Toward Autonomy: Metacognitive Learning for Enhanced AI Performance, Conway-Smith and West, AAAI Symposium Series 2024. [Paper]
  6. Meta-R1: Empowering Large Reasoning Models with Metacognition, Dong et al., arXiv 2025. [Paper]
  7. Towards Interpretable and Consistent Multi-Step Mathematical Reasoning in Large Language Models, Huang et al., AIIM 2025. [Paper]
  8. Thinking About Thinking: SAGE-nano's Inverse Reasoning for Self-Aware Language Models, Jha et al., arXiv 2025. [Paper]
  9. Adapting Like Humans: A Metacognitive Agent with Test-Time Reasoning, Li et al., arXiv 2025. [Paper]
  10. Before You <think>, Monitor: Implementing Flavell's Metacognitive Framework in LLMs, Oh, arXiv 2025. [Paper]
  11. Monitor-Generate-Verify (MGV): Formalising Metacognitive Theory for Language Model Reasoning, Oh and Gobet, arXiv 2025. [Paper]
  12. Position: LLMs Need a Bayesian Meta-Reasoning Framework for More Robust and Generalizable Reasoning, Yan et al., ICML 2025. [Paper]
  13. MIRA: An LLM-Driven Dual-Loop Architecture for Metacognitive Reward Design, Zhang et al., Systems 2025. [Paper]
  14. Metacognitive Myopia in Large Language Models, Scholten et al., arXiv 2024. [Paper]

Metacognition for Reasoning Models

  1. PRISM-MCTS: Learning from Reasoning Trajectories with Metacognitive Reflection, Cheng et al., arXiv 2026. [Paper]
  2. Finding RELIEF: Shaping Reasoning Behavior Without Reasoning Supervision via Belief Engineering, Leong et al., arXiv 2026. [Paper]
  3. Enhancing LLM Metacognition via Cognitive Pairwise Training, Li et al., arXiv 2026. [Paper]
  4. Towards Understanding Metacognition in Large Reasoning Models, Li et al., OpenReview 2026. [Paper]
  5. Teaching Large Reasoning Models Effective Reflection, Wang et al., arXiv 2026. [Paper]
  6. When Is Thinking Enough? Early Exit via Sufficiency Assessment for Efficient Reasoning, Xiang et al., arXiv 2026. [Paper]
  7. EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs, Yeom et al., arXiv 2026. [Paper]
  8. ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition, Zhao et al., arXiv 2026. [Paper]
  9. Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors, Didolkar et al., arXiv 2025. [Paper]
  10. From 'Aha Moments' to Controllable Thinking: Toward Meta-Cognitive Reasoning in Large Reasoning Models via Decoupled Reasoning and Control, Ha et al., arXiv 2025. [Paper]
  11. Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement Learning, Kim et al., arXiv 2025. [Paper]
  12. CoRE: Enhancing Metacognition with Label-Free Self-Evaluation in LRMs, Li et al., arXiv 2025. [Paper]
  13. Dynamic Cognitive Orchestration: Eliciting Metacognitive Planning in Large Language Models, Shakoo and Sameti, OpenReview 2025. [Paper]
  14. Cog-Rethinker: Hierarchical Metacognitive Reinforcement Learning for LLM Reasoning, Sun et al., arXiv 2025. [Paper]
  15. Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models, Yang et al., NeurIPS 2024. [Paper]

Metacognition for LLM Agents

  1. To Retrieve or To Think? An Agentic Approach for Context Evolution, Chen et al., arXiv 2026. [Paper]
  2. Learn Like Humans: Use Meta-Cognitive Reflection for Efficient Self-Improvement, Hou et al., arXiv 2026. [Paper]
  3. Learning How to Remember: A Meta-Cognitive Management Method for Structured and Transferable Agent Memory, Liang et al., arXiv 2026. [Paper]
  4. Deep Reasoning in General Purpose Agents via Structured Meta-Cognition, Light et al., arXiv 2026. [Paper]
  5. RefRea: Reference-Guided Reasoning with Meta-Cognition for Accurate Language Model Agents, Mai et al., AAAI 2026. [Paper]
  6. Deep Search with Hierarchical Meta-Cognitive Monitoring Inspired by Cognitive Neuroscience, Sun et al., arXiv 2026. [Paper]
  7. Metacognition for Unknown Situations and Environments (MUSE), Valiente and Pilly, Neural Networks 2026. [Paper]
  8. ReMA: Learning to Meta-Think for LLMs with Multi-Agent Reinforcement Learning, Wan et al., NeurIPS 2026. [Paper]
  9. Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning, Yang et al., arXiv 2026. [Paper]
  10. MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems, Zhang et al., NeurIPS 2026. [Paper]
  11. MAGELLAN: Metacognitive Predictions of Learning Progress Guide Autotelic LLM Agents in Large Goal Spaces, Gaven et al., arXiv 2025. [Paper]
  12. Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction, Shen et al., arXiv 2025. [Paper]
  13. RoboData: Toward Trustable Question Answering over Ontologies Through Metacognitive Agentic Epistemology, Musumeci et al., Wikidata Workshop (ISWC) 2025. [Paper]
  14. Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey, Bilal et al., arXiv 2025. [Paper]
  15. AutoCrit: A Meta-Reasoning Framework for Self-Critique and Iterative Error Correction in LLM Chains-of-Thought, Sang, ICMLCA 2025. [Paper]
  16. Agentic Metacognition: Designing a 'Self-Aware' Low-Code Agent for Failure Prediction and Human Handoff, Xu and Lu, CSSS (Book Chapter) 2025. [Paper]
  17. Agentic Workflows Generation Based on Meta-Cognitive Chain-of-Thought Guided Monte Carlo Tree Search, Zhou et al., ICNLP 2025. [Paper]
  18. Metacognition Is All You Need? Using Introspection in Generative Agents to Improve Goal-Directed Behavior, Toy et al., arXiv 2024. [Paper]
  19. Devil's Advocate: Anticipatory Reflection for LLM Agents, Wang et al., EMNLP Findings 2024. [Paper]
  20. ReAct: Synergizing Reasoning and Acting in Language Models, Yao et al., arXiv 2022. [Paper]

🚀 4. Metacognitive Methods to Improve Capabilities of LLMs

General Capabilities

  1. MP: Endowing Large Language Models with Lateral Thinking, Bai et al., AAAI 2025. [Paper]
  2. Eliciting Metaknowledge in Large Language Models, Longo et al., Cognitive Systems Research 2025. [Paper]
  3. Tuning-Free Accountable Intervention for LLM Deployment – A Metacognitive Approach, Tan et al., AAAI 2025. [Paper]
  4. Self-Reflection in Large Language Model Agents: Effects on Problem-Solving Performance, Renze and Guven, FLLM 2024. [Paper]
  5. Metacognitive Prompting Improves Understanding in Large Language Models, Wang and Zhao, NAACL 2024. [Paper]

Confidence Calibration

  1. Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs, Liu et al., arXiv 2026. [Paper]
  2. MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs, Liu et al., EMNLP 2025. [Paper]
  3. Improving Metacognition and Uncertainty Communication in Language Models, Steyvers et al., arXiv 2025. [Paper]

Hallucination Reduction

  1. Hallucinations Undermine Trust; Metacognition Is a Way Forward, Yona et al., arXiv 2026. [Paper]
  2. Step-By-Step Reasoning with Meta Cognitive Prompts to Reduce Contextual Hallucination, Miki and Vincent, CHI HEAL Workshop 2025. [Paper]
  3. Teach Large Language Models the Concept of Meta-Cognition to Reduce Hallucination Text Generation, Li, OpenReview 2024. [Paper]

Knowledge Boundary Detection

  1. Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models, Chen et al., arXiv 2026. [Paper]
  2. Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions, Huang et al., arXiv 2026. [Paper]
  3. Self-Aware Language Models: A Taxonomy and Evaluation of Epistemic Uncertainty and Hallucination Mitigation, Tiwari and Gupta, Research Square 2026. [Paper]
  4. Whether LLMs Know If They Know: Identifying Knowledge Boundaries via Debiased Historical In-Context Learning, Lv et al., ACL Findings 2025. [Paper]
  5. Physics-Informed Metacognition: Improving LLMs Self-Knowledge via Physical Constraints, Roy and Roy, EurIPS Workshop on Metacognition in Generative AI 2025. [Paper]
  6. Do Large Language Models Know How Much They Know?, Prato et al., EMNLP 2024. [Paper]
  7. Do Large Language Models Know What They Don't Know?, Yin et al., ACL Findings 2023. [Paper]

Resistance to Persuasion

  1. Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions, Huang et al., arXiv 2026. [Paper]

Interpretability

  1. Self-Interpretability: LLMs Can Describe Complex Internal Processes That Drive Their Decisions, Plunkett et al., arXiv 2025. [Paper]
  2. The Metacognitive Demands and Opportunities of Generative AI, Tankelevitch et al., CHI 2024. [Paper]

Reasoning for Non-Reasoning Models

  1. Think²: Grounded Metacognitive Reasoning in Large Language Models, Elenjical et al., arXiv 2026. [Paper]
  2. Probabilistic Chain-of-Evidence: Enhancing Factual Accuracy and Uncertainty Reasoning in Large Language Models via Prompt Engineering, Fang and Chen, Preprints 2026. [Paper]
  3. Flying Pigs, FaR and Beyond: Evaluating LLM Reasoning in Counterfactual Worlds, Balappanawar et al., arXiv 2025. [Paper]
  4. Thinking About Thinking: Metacognitive Influence Tracing for Reliable LLM Reasoning, Cui et al., OpenReview 2025. [Paper]
  5. Meta-of-Thought: Reasoning About Reasoning in Large Language Models, Haque, Authorea Preprints 2025. [Paper]
  6. Towards Interpretable and Consistent Multi-Step Mathematical Reasoning in Large Language Models, Huang et al., AIIM 2025. [Paper]
  7. Language Models Coupled with Metacognition Can Outperform Reasoning Models, Khandelwal et al., arXiv 2025. [Paper]
  8. ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection, Li et al., ACL Findings 2025. [Paper]
  9. Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction, Liu et al., NAACL 2025. [Paper]
  10. Before You <think>, Monitor: Implementing Flavell's Metacognitive Framework in LLMs, Oh, arXiv 2025. [Paper]
  11. Cognitive Reasoing in Translation: Evaluating Chain-of-Thought, Explaining, Metacognition, and Critique in Humans and General-Purpose vs. Advanced-Reasoning Large Language Models, Qian and Yang, HCI International 2025. [Paper]
  12. MDToC: Metacognitive Dynamic Tree of Concepts for Boosting Mathematical Problem-Solving of Large Language Models, Ta et al., arXiv 2025. [Paper]
  13. Meta Reasoning for Large Language Models, Gao et al., arXiv 2024. [Paper]

Retrieval

  1. To Retrieve or To Think? An Agentic Approach for Context Evolution, Chen et al., arXiv 2026. [Paper]
  2. Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger, Li et al., ACL 2025. [Paper]
  3. Whether LLMs Know If They Know: Identifying Knowledge Boundaries via Debiased Historical In-Context Learning, Lv et al., ACL Findings 2025. [Paper]
  4. Metacognitive Retrieval-Augmented Large Language Models, Zhou et al., The Web Conference 2024. [Paper]

Self-Improvement

  1. Learn Like Humans: Use Meta-Cognitive Reflection for Efficient Self-Improvement, Hou et al., arXiv 2026. [Paper]
  2. Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agents, Bao et al., arXiv 2025. [Paper]
  3. Position: Truly Self-Improving Agents Require Intrinsic Metacognitive Learning, Liu and Van Der Schaar, ICML 2025. [Paper]
  4. Recursive Introspection: Teaching LLM Agents How to Self-Improve, Qu et al., ICML Workshop 2024. [Paper]
  5. MoT: Memory-of-Thought Enables ChatGPT to Self-Improve, Li and Qiu, EMNLP 2023. [Paper]

Other Abilities

  1. Could You Be Wrong: Metacognitive Prompts for Improving Human Decision Making Help LLMs Identify Their Own Biases, Hills, AI (MDPI) 2026. [Paper]
  2. Metacognitive Sensitivity for Test-Time Dynamic Model Selection, Trinh et al., CogInterp Workshop 2026. [Paper]
  3. MetaGDPO: Alleviating Catastrophic Forgetting with Metacognitive Knowledge Through Group Direct Preference Optimization, Zhang et al., AAAI 2026. [Paper]
  4. Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection, Lee et al., CHum Workshop (ACL) 2025. [Paper]
  5. Think, Reflect, Create: Metacognitive Learning for Zero-Shot Robotic Planning, Lin et al., arXiv 2025. [Paper]
  6. Meta-Cognitive Prompting: A Comparative Framework for Prompt Engineering in Large Language Models, Maxwell, ResearchGate 2025. [Paper]
  7. R-CHAR: A Metacognition-Driven Framework for Role-Playing in Large Language Models, Qin et al., EMNLP 2025. [Paper]
  8. MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Design, Qiu et al., arXiv 2025. [Paper]
  9. MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Risks in LLMs on Domain Tasks, Shan et al., arXiv 2025. [Paper]
  10. Ontogenia: Ontology Generation with Metacognitive Prompting in Large Language Models, Lippolis et al., ESWC 2024. [Paper]
  11. Does Metacognitive Prompting Improve Causal Inference in Large Language Models?, Ohtani et al., IEEE CAI 2024. [Paper]

🚢 5. Applications of LLM Metacognition

Human-AI Decision-Making

  1. "Can You Tell Me?": Designing Copilots to Support Human Judgement in Online Information Seeking, Bink et al., CHIIR 2026. [Paper]
  2. Skeptical Intelligence: Refining Critical Thinking for AI-Powered Innovation, Ladd and Shrivastava, SSRN 2026. [Paper]
  3. Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction, Yun et al., CHI 2026. [Paper]
  4. Metacognition and Confidence Dynamics in Advice Taking from Generative AI, Colombatto et al., arXiv 2025. [Paper]
  5. Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance, Fan et al., British Journal of Educational Technology 2025. [Paper]
  6. AI Makes You Smarter but None the Wiser: The Disconnect Between Performance and Metacognition, Fernandes et al., Computers in Human Behavior 2025. [Paper]
  7. Metacognitive Sensitivity: The Key to Calibrating Trust and Optimal Decision Making with AI, Lee et al., PNAS Nexus 2025. [Paper]
  8. Beyond Accuracy: How AI Metacognitive Sensitivity Improves AI-Assisted Decision Making, Li and Steyvers, arXiv 2025. [Paper]
  9. The Importance of Metacognitive Sensitivity in Human-AI Decision-Making, Li and Steyvers, CogSci 2025. [Paper]
  10. Meta-Cognitive Competence and AI-Assisted Decision-Making: Revisiting the Role of Explainable AI and Uncertainty Quantification, Sass, Philosophy & Technology 2025. [Paper]
  11. Enhancing Critical Thinking in Generative AI Search with Metacognitive Prompts, Singh et al., ASIS&T 2025. [Paper]
  12. Do You Think GPT Will Be Correct?: Measuring and Improving Generative AI Literacy Through Metacognition Training, Su et al., arXiv 2025. [Paper]
  13. Metacognition and Uncertainty Communication in Humans and Large Language Models, Steyvers and Peters, Current Directions in Psychological Science 2025. [Paper]
  14. What Large Language Models Know and What People Think They Know, Steyvers et al., Nature Machine Intelligence 2025. [Paper]
  15. Impact of Feedback Timing on Metacognition in AI-Mediated Language Learning, Nadila et al., PhD Thesis, HEC Montréal 2024. [Paper]

User Simulation

  1. Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident, Borchers et al., arXiv 2026. [Paper]
  2. Can You Share Your Story? Modeling Clients' Metacognition and Openness for LLM Therapist Evaluation, Kim et al., ACL Findings 2025. [Paper]
  3. Which Type of Students Can LLMs Act? Investigating Authentic Simulation with Graph-Based Human–AI Collaborative System, Li et al., arXiv 2025. [Paper]
  4. Cognitive Echo: Enhancing Think-Aloud Protocols with LLM-Based Simulated Students, Zheng et al., British Journal of Educational Technology 2025. [Paper]

Pedagogy

  1. Autonomous Usage of LLMs in Scenario-Based Laboratory Learning in Engineering Education, Boettcher, 2025 Yearbook Emerging Technologies in Learning 2026. [Paper]
  2. Developing Metacognition Through LLM-Enhanced Writing Assignments: Critical Thinking Exercises for Scientific Writing, Gatzia and Wood, The American Biology Teacher 2026. [Paper]
  3. Reflecting in the Reflection: Integrating a Socratic Questioning Framework into Automated AI-Based Question Generation, Holub et al., arXiv 2026. [Paper]
  4. Rewarding How Models Think Pedagogically: Integrating Pedagogical Reasoning and Thinking Rewards for LLMs in Education, Lee et al., arXiv 2026. [Paper]
  5. LLM Prompt Evaluation for Educational Applications, Holmes et al., arXiv 2026. [Paper]
  6. MetaCLASS: Metacognitive Coaching for Learning with Adaptive Self-Regulation Support, Liu et al., arXiv 2026. [Paper]
  7. Investigating the Role of ChatGPT in Supporting Metacognitive Processes During Problem-Solving Activities, Contel and Cusi, Digital Experiences in Mathematics Education 2025. [Paper]
  8. Learning Behaviors Mediate the Effect of AI-Powered Support for Metacognitive Calibration on Learning Outcomes, Lee et al., CHI 2025. [Paper]
  9. The Cognitive Mirror: A Framework for AI-Powered Metacognition and Self-Regulated Learning, Tomisu et al., Frontiers in Education 2025. [Paper]
  10. Metacognitive Reflection in the Era of Generative AI, Uittenhove et al., Research Square 2025. [Paper]

🌱 6. Broader Directions

AI Metacognition

  1. Toward Artificial Metacognition, Shakarian, AAAI 2026. [Paper]
  2. No Reliable Evidence of Self-Reported Sentience in Small Large Language Models, Kaiser and Enderby, arXiv 2026. [Paper]
  3. Consciousness and AI: A Meta-Reflective Framework, Erbe, SSRN 2025. [Paper]
  4. Knowing (Not) to Know: Explainable Artificial Intelligence and Human Metacognition, Von Zahn et al., Information Systems Research 2025. [Paper]
  5. A Treasure Map to Metacognition, Wright, AGI Conference 2025. [Paper]
  6. Metacognitive AI: Framework and the Case for a Neurosymbolic Approach, Wei et al., NeSy 2024. [Paper]
  7. Design and Validation of a Metamodel for Metacognition Support in Artificial Intelligent Systems, Caro et al., Biologically Inspired Cognitive Architectures 2014. [Paper]
  8. The Role of Metacognition in Robust AI Systems, Schmill et al., AAAI Workshop on Metareasoning 2008. [Paper]

Creativity

  1. Who Owns Creativity and Who Does the Work? Trade-Offs in LLM-Supported Research Ideation, Liu et al., arXiv 2026. [Paper]
  2. Reflexa: Uncovering How LLM-Supported Reflection Scaffolding Reshapes Creativity in Creative Coding, Wang et al., arXiv 2026. [Paper]
  3. Metacognitive Feelings as a Source of Information for the Creative Process: A Conceptual Exploration, Puente-Diaz, Journal of Intelligence 2023. [Paper]

Self-Improvement

  1. Position: Truly Self-Improving Agents Require Intrinsic Metacognitive Learning, Liu and Van Der Schaar, ICML 2025. [Paper]
  2. Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Markets, Zhang et al., arXiv 2025. [Paper]
  3. Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study, Ning et al., NeurIPS 2024. [Paper]

Meta-Metacognition & Theory of Mind

  1. Meta-Metacognition: Processes Underlying Judgments About Metacognition, Schwartz et al., New Ideas in Psychology 2026. [Paper]
  2. Common Computations for Metacognition and Meta-Metacognition, Zheng et al., Neuroscience of Consciousness 2023. [Paper]
  3. Violation of Expectation via Metacognitive Prompting Reduces Theory of Mind Prediction Error in Large Language Models, Leer et al., arXiv 2023. [Paper]

Risks & Ethics

  1. The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance, Maynard, arXiv 2026. [Paper]
  2. Meta-Reflective Capacities, Normative Commitments, and Responsible AI, Fleig-Goldstein, arXiv 2025. [Paper]
  3. Imagining and Building Wise Machines: The Centrality of AI Metacognition, Johnson et al., Trends in Cognitive Sciences 2025. [Paper]
  4. AI Awareness, Li et al., arXiv 2025. [Paper]
  5. Human Versus Artificial Social Cognition and Metacognition: The Normative Difference, Tison and Zawidzki, AI and Ethics 2025. [Paper]
  6. Alignment Faking in Large Language Models, Greenblatt et al., arXiv 2024. [Paper]
  7. Escalation Risks from Language Models in Military and Diplomatic Decision-Making, Rivera et al., FAccT 2024. [Paper]
  8. Metacognition, Accountability and Legal Personhood of AI, Ribeiro et al., Multidisciplinary Perspectives on AI and the Law (Book Chapter) 2023. [Paper]
  9. Truthful AI: Developing and Governing AI That Does Not Lie, Evans et al., arXiv 2021. [Paper]

📚 7. Other Resources

  1. A Survey of Uncertainty Estimation Methods on Large Language Models, Xia et al., ACL Findings 2025. [Paper]
  2. A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice, Huang et al., arXiv 2024. [Paper]

About

Metacognition in LLMs: Foundations, Progress, and Opportunities

Resources

Stars

56 stars

Watchers

2 watching

Forks

Releases

Packages

Contributors