Skip to content

2026 05 08 integrated cascading failure agentic vs generative ai risk

Jump to bottom
github-actions[bot] edited this page May 9, 2026 · 1 revision

How do coupled enterprise risks manifest differently in agentic Artificial Intelligence (AI), meaning autonomous multi-step systems, versus generative AI deployments, and what integrated risk frameworks best predict cascading failures?

Research Question

How do the coupled enterprise risks, capability debt, incentive-driven shadow Artificial Intelligence (AI) adoption, skill decay, and oversight failure, manifest differently in agentic AI, meaning autonomous multi-step systems, versus generative AI deployments? What integrated risk frameworks best predict and prevent cascading failures? What are the long-term organisational impacts of prioritising measurable speed over unmeasured quality in AI tool adoption, and how can enterprises empirically test "AI for risk reduction first" strategies that address debt and incentives before scaling autonomous agents?

Scope

In scope:

  • Comparative analysis of how the four coupled risk modes, capability debt, shadow AI, skill decay, and oversight failure, manifest differently in agentic and generative AI contexts, with emphasis on higher autonomy, longer action chains, tool use, memory, and reduced human friction in agentic systems
  • Integrated risk frameworks and systems models that treat the four risks as a coupled feedback loop rather than as independent variables
  • Long-term organisational impacts of speed-over-quality prioritisation on innovation capacity, incident response competence, and regulatory or governance posture
  • Empirical or theory-backed ways to test "AI for risk reduction first" strategies, including what return on investment or risk-reduction signals can be measured
  • Case material or natural experiments showing what happens when AI adoption outruns capability-building, platform quality, or oversight design

Out of scope:

  • Re-running the full individual literature review for each of the four risk factors already covered in companion items
  • Consumer-tier AI product adoption outside enterprise contexts
  • Macro-level labour-market effects
  • Model-alignment research focused on pre-training or reinforcement learning optimization rather than enterprise deployment

Constraints:

  • Distinguish clearly between agentic AI, meaning systems that can plan, call tools, and act across multiple steps, and generative AI used mainly for single-turn content generation
  • Ground claims in observable enterprise patterns or clearly labeled inference
  • Expand all acronyms on first use
  • Use URL-backed citations for prior completed repository items

Context

Prior completed repository work has already established the four component risk modes and several adjacent governance surfaces, including systems-capability debt, incentive-driven workaround behaviour, review bottlenecks, and alliance-level security guidance. [fact; source: https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://davidamitchell.github.io/Research/research/2026-05-07-five-eyes-ai-risks-and-advice.html]

The remaining gap is a single synthesis that explains how those same weaknesses change once work moves from assistant-style generation to tool-using action, and which framework combination is most useful for predicting that shift before incidents occur. [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai]

Approach

  1. Compare how capability debt, shadow AI, skill decay, and oversight failure behave in generative AI versus agentic AI deployments.
  2. Evaluate which integrated frameworks best explain feedback loops, delayed consequences, and cascading failures across those risk modes.
  3. Assess evidence on the organisational consequences of rewarding speed before platform quality, governance quality, and skill retention.
  4. Derive an evidence-backed testing model for sequencing AI adoption toward risk reduction before broad autonomy.

Sources

Related

Research Skill Output

(Full output from running the research skill, retained verbatim in the completed item. Sections 0 to 5 are the investigation; section 6 seeds the Findings section below.)

§0 Initialise

  • Question: How do capability debt, shadow AI, skill decay, and oversight failure compose into different enterprise failure patterns in generative AI versus agentic AI deployments, and which integrated framework best predicts where those cascades begin?
  • Scope: Enterprise deployment, governance, and operating-model risk, with direct comparison between assistant-style generative use and tool-using agentic use.
  • Constraints: Observable enterprise patterns first, primary or official sources where possible, explicit epistemic labels, URL-backed citations for prior repository items, and acronym expansion on first use.
  • Output: knowledge item with mirrored synthesis and Findings sections, an Evidence Map, assumptions, and open questions.
  • [fact; source: https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://davidamitchell.github.io/Research/research/2026-05-07-five-eyes-ai-risks-and-advice.html] Prior completed items already establish the component failure modes, several control surfaces, and the regulatory-security baseline that this item needs to integrate rather than re-derive.
  • [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] The distinctive work here is to model how the same organisational weaknesses behave differently once AI moves from content generation to delegated action.

§1 Question Decomposition

  • Root question: Which coupled enterprise risks change most when AI moves from generating content to taking multi-step actions, and how should enterprises sequence controls accordingly?
  • A. Risk differentiation
    • A1. How do generative AI and agentic AI differ structurally in what they can do?
    • A2. How does each of the four risk modes change under that structural shift?
    • A3. Which new cascade paths appear only when tool use, memory, or delegated permissions are added?
  • B. Integrated framework choice
    • B1. Which frameworks explain interacting causes rather than isolated controls?
    • B2. Which frameworks help predict delayed or reinforcing failure, not just classify known risks?
    • B3. Which frameworks translate into practical enterprise governance actions?
  • C. Organisational consequence
    • C1. What happens when productivity pressure outruns platform quality and governance quality?
    • C2. What happens to human review capacity and human skill when AI throughput rises?
    • C3. What signals show that a local speed gain is becoming a system-level instability?
  • D. Sequencing and testing
    • D1. Is there evidence for using AI first to reduce risk rather than first to expand autonomy?
    • D2. Which measurable indicators can test whether that sequencing works?
    • D3. Which control design patterns remain meaningful once action volume reaches machine speed?

§2 Investigation

  • Prior completed-item sweep:

    • [fact; source: https://davidamitchell.github.io/Research/research/2026-04-22-enterprise-ai-capability-model.html; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html] The nearest repository items already connect weak platform capability, workaround demand, implicit control removal, review overload, and skill decay, which makes this item a synthesis of interaction effects rather than a first-pass survey of each mechanism.
    • [inference; source: https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html] The unresolved question is not whether the risks exist, but how their composition changes when the deployment mode changes.

  • A. How agentic and generative deployments differ structurally

    • [fact; source: https://www.nist.gov/itl/ai-risk-management-framework; https://doi.org/10.6028/NIST.AI.600-1] National Institute of Standards and Technology (NIST) guidance treats generative AI as a profile with unique or amplified risks such as confabulation, information integrity, harmful bias, intellectual-property exposure, information security, and human over-reliance.
    • [inference; source: https://www.nist.gov/itl/ai-risk-management-framework; https://doi.org/10.6028/NIST.AI.600-1] That profile treatment supports reading generative deployment as a distinct risk class inside the broader AI Risk Management Framework.
    • [fact; source: https://www.anthropic.com/research/trustworthy-agents; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems] Anthropic and Microsoft both define agentic systems by their ability to plan, call tools, access data, observe results, and continue acting with limited human intervention.
    • [inference; source: https://www.anthropic.com/research/trustworthy-agents; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems] Those capabilities make the control problem about action and delegation rather than only output quality.
    • [fact; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems] Cybersecurity and Infrastructure Security Agency (CISA) guidance and Microsoft guidance identify agentic-specific risks that do not appear as strongly in single-turn assistant use, including expanded attack surface, privilege creep, behavioural misalignment, obscure event records, unsafe tool invocation, and the need for deterministic human review for high-risk actions.
    • [fact; source: https://www.ibm.com/think/insights/agentic-ai-security; https://www.anthropic.com/research/trustworthy-agents] International Business Machines (IBM) and Anthropic both describe the key shift as movement from what a model says to what it does, with agents behaving more like digital workers or digital insiders than passive chat interfaces.
    • [fact; source: https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] Prompt injection already matters for generative applications, but once memory, tools, or orchestration are added it can also change future sessions, trigger tool calls, or silently exfiltrate data, which increases both persistence and blast radius.
    • [inference; source: https://doi.org/10.6028/NIST.AI.600-1; https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.ibm.com/think/insights/agentic-ai-security] Generative AI risk is dominated by content, information-quality, and human-over-reliance problems, while agentic AI adds execution, permissions, identity, and runtime-control failure modes on top of those earlier risks rather than replacing them.

  • B. How the four coupled risks change under that shift

    • [fact; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] Shadow AI is already widespread in generative tool use because workers adopt faster or better external tools when enterprise offerings lag, which creates unmanaged data exposure before any agentic capability is even added.
    • [fact; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] International Business Machines (IBM) reports that 80% of surveyed United States office workers use AI at work but only 22% use only employer-provided tools, while IBM and Ponemon Institute report that 97% of organisations with an AI-related security incident lacked proper AI access controls and 63% lacked AI governance policies to manage AI or prevent shadow AI.
    • [fact; source: https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] Cyberhaven reports that corporate data sent to AI tools increased 485% between March 2023 and March 2024, that 73.8% of workplace OpenAI Chat Generative Pre-trained Transformer (ChatGPT) accounts were non-corporate, and that 27.4% of corporate data sent to AI tools in March 2024 was sensitive.
    • [fact; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] Skill decay evidence is not limited to speculation: Macnamara et al. argue that AI assistants can accelerate expert skill decay and hinder skill acquisition without users noticing, while Crowston and Bolici conclude that deskilling is common unless work design preserves prompting, evaluation, and editing skills.
    • [inference; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents] Oversight failure changes qualitatively in agentic settings because the relevant human task is no longer only "check this answer" but also "understand this plan, challenge this action chain, and intervene before an irreversible act happens."
    • [inference; source: https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html; https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://www.anthropic.com/research/trustworthy-agents] Capability debt and shadow AI are upstream causes in both deployment modes, but under agentic deployment they interact with delegated permissions and removed human pacing limits, so a workaround that would have leaked information in a generative workflow can now also take actions, propagate errors, or trigger multi-step failures.
    • [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html] Skill decay and weak oversight are secondary quality risks for many generative deployments, but they become primary containment risks for agentic deployments because human fallback, anomaly recognition, and stop decisions are exactly the capabilities that over-delegation weakens.

  • C. Which integrated frameworks best explain cascading failure

    • [fact; source: https://archive.org/details/mit_press_book_9780262298247] Leveson's Systems-Theoretic Accident Model and Processes (STAMP) treats accidents as failures in the control and enforcement of safety constraints across complex sociotechnical systems rather than as isolated component failures.
    • [inference; source: https://archive.org/details/mit_press_book_9780262298247] That framing makes STAMP directly useful for modelling enterprise AI cascades that involve organisational, human, and technical interaction.
    • [fact; source: https://www.chelseagreen.com/product/thinking-in-systems/] Meadows' systems-thinking primer centers stocks, flows, feedback loops, delays, and leverage points, which are the concepts needed to explain how local productivity gains can reinforce shadow adoption, review overload, and delayed failure recognition.
    • [fact; source: https://www.nist.gov/itl/ai-risk-management-framework; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook] NIST's AI Risk Management Framework includes inventory, roles and responsibilities, training, risk tolerance, ongoing monitoring, decommissioning, and the iterative Govern, Map, Measure, and Manage structure.
    • [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.nist.gov/itl/ai-risk-management-framework; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook] In combination with STAMP, that framework supplies the operational governance layer needed for enterprise intervention design.
    • [fact; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] DORA and Massachusetts Institute of Technology (MIT) Sloan add the organisational-performance layer by showing that AI amplifies existing team weaknesses, that weak platforms and weak feedback loops turn speed into instability, and that human-era management structures strain under agentic speed and scale unless explicit rules, thresholds, tracing, and intervention points are defined.
    • [fact; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development] CISA and the secure AI system development guidelines translate the framework into operational safeguards, including low-risk starting points, least privilege, secure design, secure deployment, logging, monitoring, and explicit alignment with existing security models.
    • [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] The most predictive integrated approach is therefore not one framework but a stack: STAMP for causal structure, systems thinking for reinforcing loops and delays, NIST for lifecycle governance, and DORA plus CISA for enterprise operating signals and sequencing.

  • D. What speed-over-quality prioritisation does organisationally

    • [fact; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] DORA's central conclusion is that AI does not fix a team but amplifies what is already there, and the 2025 report finds that AI adoption now correlates positively with throughput and product performance while still correlating negatively with software-delivery stability.
    • [fact; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] DORA attributes that instability to weak automated testing, weak version-control practice, slow feedback loops, and weak internal platforms, and recommends clarifying AI policies, connecting AI to internal context, prioritising foundational practices, fortifying safety nets, and investing in internal platforms.
    • [fact; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] MIT Sloan and Boston Consulting Group report that 69% of their international expert panel agree that agentic AI requires new management approaches, and the article attributes that need to autonomy, complexity, opacity, and superhuman speed and scale.
    • [fact; source: https://www.ibm.com/reports/data-breach] IBM and Ponemon Institute report an average global data-breach cost of 4.4 million United States dollars, identify missing AI access controls and missing AI governance as common features in AI-related incidents, and report 1.9 million United States dollars in average savings for organisations that make extensive use of AI in security.
    • [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] When organisations reward visible speed before platform quality, policy clarity, and review capacity, the likely system outcome is a reinforcing loop of more shadow use, more unmanaged data flow, more change volume, weaker verification, and greater compliance or security exposure.

  • E. What the evidence says about "AI for risk reduction first"

    • [fact; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] CISA explicitly recommends beginning with low-risk, non-sensitive agentic use cases, avoiding broad or unrestricted access, and aligning agentic AI with the organisation's existing security model and risk posture.
    • [fact; source: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents] Microsoft and Anthropic both recommend bounded permissions, explicit action schemas or tool permissions, plan visibility, runtime logging, anomaly detection, and deterministic human review for high-risk actions rather than unconstrained autonomy.
    • [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] The clearest measurable evidence for risk-reduction-first value is indirect but real: DORA finds that internal platforms and safety nets are the foundation for AI value, while IBM and Ponemon Institute report material breach-cost savings from using AI extensively in security.
    • [inference; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] No reviewed source states a named "AI for risk reduction first" doctrine, but multiple sources converge on the same sequencing rule: strengthen inventory, policy, platform context, least privilege, monitoring, and safety nets before scaling sensitive or high-autonomy use cases.
    • [inference; source: https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] The strongest empirical test is a sequencing experiment rather than a simple tool comparison: compare autonomy-first rollouts against risk-reduction-first rollouts on shadow-tool usage, privileged-action volume, instability or incident rates, human override rates, review queue depth, and skill-maintenance signals such as independent problem solving and challenge behaviour.

§3 Reasoning

  • [inference; source: https://doi.org/10.6028/NIST.AI.600-1; https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems] The decisive difference between generative and agentic deployment is not model family by itself but whether the system crosses from advice generation into delegated action, because that is the point where permissions, orchestration, and runtime observability become first-order controls.
  • [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/] A cascade model needs both causal structure and operational governance, so STAMP or general systems thinking alone is insufficient, while checklist governance alone is weak on reinforcing loops and delayed side effects.
  • [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://www.ibm.com/reports/data-breach] The best-supported organisational story is a pressure-amplification loop: speed pressure creates workaround demand, workaround demand erodes governance visibility, and eroded visibility increases the chance that AI failures are discovered only after they have already spread.
  • [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] Skill decay matters here because the humans most needed to supervise exceptions, challenge plans, and recover from anomalies are the same humans whose judgment degrades when AI handles too much of the ordinary work.
  • [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] The evidence for "AI for risk reduction first" is strong enough for a practical recommendation but not strong enough to call it an established named framework, so the conclusion should remain explicitly synthetic rather than presented as settled doctrine.

§4 Consistency Check

  • [fact; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] MIT Sloan's panel is not unanimous, because 25% argue that existing management approaches can be adapted rather than replaced.
  • [inference; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] That disagreement does not overturn the main conclusion, because both sides still converge on explicit rules, traceability, intervention paths, and bounded permissions as necessary controls.
  • [fact; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] Skill outcomes are mixed in the literature, because some work can upskill prompting, evaluation, and editing even while other work deskills core domain performance.
  • [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] This tension is best resolved as a work-design question rather than a contradiction, which means the item should treat skill decay as contingent but operationally important.
  • [fact; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] The shadow AI prevalence evidence is partly vendor-produced, so it should be used for magnitude and directional signal, not as the sole basis for causal claims.

§5 Depth and Breadth Expansion

  • [inference; source: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Technical lens: the move from generative to agentic systems is a move from model-output risk to model-plus-runtime risk, because the control surface now includes memory, tools, identities, and orchestration state.
  • [inference; source: https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development; https://www.nist.gov/itl/ai-risk-management-framework; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] Governance lens: mature agentic deployment depends less on a new regulation than on making existing security and risk-governance disciplines explicit at AI-specific control points.
  • [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach] Economic lens: the strongest business case for sequencing is that foundational controls convert AI from a cost amplifier into a force multiplier, while uncontrolled rollout externalises cost into instability, breaches, and recovery work.
  • [inference; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] Behavioural lens: user demand for speed and convenience is strong enough that enterprises should assume workaround pressure and over-delegation will happen unless the sanctioned path is both safer and easier.

§6 Synthesis

Executive summary:

  • [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1] Agentic deployments fail differently from generative deployments because they convert the same upstream weaknesses, capability debt, shadow use, skill decay, and weak oversight, into delegated action risk rather than mostly content and information-quality risk.
  • [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] The most useful predictive model is a layered one that combines control-structure analysis, feedback-loop analysis, lifecycle governance, and operational platform signals rather than relying on a single checklist or maturity score.
  • [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] Organisations that optimise for visible speed before platform quality, governance quality, and skill retention create reinforcing loops that increase shadow AI, weaken review, and raise the chance that small local shortcuts become enterprise-wide incidents.
  • [inference; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach] A cautious sequencing rule supported by this evidence base is "AI for risk reduction first": use AI to strengthen inventory, monitoring, security, platform context, and low-risk workflows before granting broad autonomy or sensitive access.

Key findings:

  1. [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1] Once planning, tool use, memory, and delegated action are added, the upstream weaknesses already familiar from generative deployments become execution and permissions failures with a materially wider blast radius.
  2. [inference; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html] In practice, capability debt worsens shadow AI under agentic deployment because weak sanctioned rails push workers toward unmanaged tools just as those unmanaged tools gain access, state, and action authority.
  3. [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/] Human skill decay and weak oversight move from quality concerns to containment concerns in agentic settings, since the people asked to challenge plans and stop unsafe actions are the same people whose judgment erodes under repeated over-delegation.
  4. [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] A layered STAMP-plus-systems-thinking-plus-NIST-plus-DORA-CISA stack is the most predictive option because it joins causal structure, feedback dynamics, lifecycle governance, and operational sequencing in one frame.
  5. [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] Evidence from DORA, IBM, and Cyberhaven points to a reinforcing loop in which local speed gains raise shadow demand and change volume faster than platforms, policies, and review systems can absorb them.
  6. [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] Measurable support for "AI for risk reduction first" is strongest where AI is used to improve security, monitoring, and platform context before broad autonomy, since those investments are the ones most consistently associated with lower incident cost and more stable delivery.
  7. [inference; source: https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] A credible enterprise experiment should compare rollout sequence, not only tool choice, by tracking shadow use, privileged-action volume, stability, incident rate, override behavior, queue depth, and skill-maintenance measures across autonomy-first and risk-reduction-first cohorts.
  8. [inference; source: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://www.anthropic.com/research/trustworthy-agents] For scaled agentic deployment, the best-supported control pattern is bounded autonomy with least privilege, explicit action schemas, deterministic human review for irreversible actions, and strong logging or observability instead of universal per-step approval.

Evidence map:

Claim Source Confidence Notes
[inference] Agentic systems convert shared upstream weaknesses into execution and permissions failures, not only content failures. https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1 high Deployment-mode shift
[inference] Capability debt and shadow AI reinforce each other more severely once unmanaged systems can act with tools and state. https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html medium External prevalence plus prior corpus mechanism
[inference] Skill decay and weak oversight become containment risks in agentic settings. https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/ medium Design-contingent but consistent
[inference] A layered STAMP plus systems-thinking plus NIST plus DORA-CISA stack is the most predictive integrated framework. https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services medium Synthesis rather than named standard
[inference] Speed-over-quality creates a reinforcing loop of shadow demand, rising change volume, weak review, and higher incident exposure. https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk high Cross-source convergence
[inference] Risk-reduction-first sequencing has the clearest support when AI is used first for security, monitoring, and platform context. https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services medium Best evidence is indirect
[inference] The correct enterprise experiment compares rollout sequence on shadow use, privileged actions, stability, incidents, and skill retention. https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143 medium Proposed measurement design
[inference] Bounded autonomy with least privilege, explicit actions, and deterministic review for irreversible acts is the strongest practical control pattern. https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://www.anthropic.com/research/trustworthy-agents high Strong operational convergence

Assumptions:

  • [assumption; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services] Enterprise management and security guidance drawn from critical infrastructure, software, and major-platform environments generalises to broader enterprise deployments because the relevant control surfaces, permissions, logging, review rights, and intervention paths, are shared.
  • [assumption; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] The skill-decay evidence, which includes medicine and more general information-systems work, is directionally applicable to enterprise AI operations because the shared mechanism is reduced human practice in judgment, verification, and recovery tasks.

Analysis:

  • [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/] STAMP and systems thinking carry most of the causal weight here, since the four target risks reinforce one another over time and would be flattened by a single-cause framework.
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook] By contrast, NIST matters because it turns a cascade story into an intervention map through inventory, role clarity, monitoring, risk tolerance, and decommissioning guidance.
  • [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] DORA, IBM, and Cyberhaven were weighted for operational signal because they quantify what happens when user demand outruns sanctioned platforms, even though some of that evidence is vendor-produced rather than fully independent.
  • [inference; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents] CISA, Microsoft, and Anthropic are the strongest differentiators between agentic and generative deployment because they describe the action layer, not only the harm categories.
  • [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] The remaining uncertainty sits around the sequencing claim itself: current evidence strongly favors safety nets, internal platforms, and AI-enabled security, but it still falls short of a standardised longitudinal benchmark for rollout order.

Risks, gaps, uncertainties:

  • [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents] Public evidence on agentic AI is still weighted toward official guidance and provider experiments, so long-horizon enterprise incident datasets remain thin.
  • [inference; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk] Shadow AI prevalence evidence is directionally consistent across sources, but precise magnitudes should be treated cautiously because two of the strongest public sources are vendor-affiliated.
  • [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143] Skill outcomes remain design-contingent, so enterprises should avoid treating deskilling as inevitable and instead measure whether work design is producing upskilling or deskilling.
  • [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Return-on-investment evidence for risk-reduction-first sequencing is strongest in adjacent signals, security savings and delivery stability, not yet in direct head-to-head rollout trials.

Open questions:

  • Which enterprise sectors will publish the first credible longitudinal comparisons of autonomy-first and risk-reduction-first deployment sequences?
  • Which skill-maintenance measures are the best leading indicators that human exception-handling capability is degrading before incidents reveal it?
  • What is the minimum viable observability package for agentic systems that preserves auditability without recreating the same review bottlenecks it is meant to reduce?

§7 Recursive Review

  • Status: pass
  • Prior completed-item sweep: completed
  • Claim-label and source audit: completed
  • Confidence outcome: medium

Findings

Executive Summary

Agentic deployments fail differently from generative deployments because they convert the same upstream weaknesses, capability debt, shadow use, skill decay, and weak oversight, into delegated action risk rather than mostly content and information-quality risk. [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1]

The strongest predictive model is not a single checklist but a layered combination of Systems-Theoretic Accident Model and Processes (STAMP) control analysis, systems-feedback reasoning, National Institute of Standards and Technology (NIST) lifecycle governance, and enterprise operating signals from DevOps Research and Assessment (DORA) and Cybersecurity and Infrastructure Security Agency (CISA) guidance. [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services]

Enterprises that optimise for visible speed before platform quality, policy clarity, and skill retention create reinforcing loops that increase shadow AI, weaken review, and raise the chance that small local shortcuts become enterprise-wide incidents. [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk]

A cautious sequencing rule supported by this evidence base is "AI for risk reduction first": use AI to strengthen inventory, monitoring, security, platform context, and low-risk workflows before granting broad autonomy or sensitive access. [inference; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach]

Key Findings

  1. Once planning, tool use, memory, and delegated action are added, the upstream weaknesses already familiar from generative deployments become execution and permissions failures with a materially wider blast radius. ([inference]; high confidence; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1)
  2. In practice, capability debt worsens shadow AI under agentic deployment because weak sanctioned rails push workers toward unmanaged tools just as those unmanaged tools gain access, state, and action authority. ([inference]; medium confidence; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html)
  3. Human skill decay and weak oversight move from quality concerns to containment concerns in agentic settings, since the people asked to challenge plans and stop unsafe actions are the same people whose judgment erodes under repeated over-delegation. ([inference]; medium confidence; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/)
  4. A layered STAMP-plus-systems-thinking-plus-NIST-plus-DORA-CISA stack is the most predictive option because it joins causal structure, feedback dynamics, lifecycle governance, and operational sequencing in one frame. ([inference]; medium confidence; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services)
  5. Evidence from DORA, IBM, and Cyberhaven points to a reinforcing loop in which local speed gains raise shadow demand and change volume faster than platforms, policies, and review systems can absorb them. ([inference]; high confidence; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk)
  6. Measurable support for "AI for risk reduction first" is strongest where AI is used to improve security, monitoring, and platform context before broad autonomy, since those investments are the ones most consistently associated with lower incident cost and more stable delivery. ([inference]; medium confidence; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services)
  7. A credible enterprise experiment should compare rollout sequence, not only tool choice, by tracking shadow use, privileged-action volume, stability, incident rate, override behavior, queue depth, and skill-maintenance measures across autonomy-first and risk-reduction-first cohorts. ([inference]; medium confidence; source: https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143)
  8. For scaled agentic deployment, the best-supported control pattern is bounded autonomy with least privilege, explicit action schemas, deterministic human review for irreversible actions, and strong logging or observability instead of universal per-step approval. ([inference]; high confidence; source: https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://www.anthropic.com/research/trustworthy-agents)

Evidence Map

Claim Source Confidence Notes
[inference] Agentic systems convert shared upstream weaknesses into execution and permissions failures, not only content failures. https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents; https://doi.org/10.6028/NIST.AI.600-1 high Deployment-mode shift
[inference] Capability debt and shadow AI reinforce each other more severely once unmanaged systems can act with tools and state. https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-agentic-ai-risk-synthesis.html; https://davidamitchell.github.io/Research/research/2026-05-02-incentive-misalignment-shadow-ai-skill-decay-controls.html medium External prevalence plus prior corpus mechanism
[inference] Skill decay and weak oversight become containment risks in agentic settings. https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143; https://davidamitchell.github.io/Research/research/2026-05-02-hitl-review-volume-bottleneck-rubber-stamp.html; https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/ medium Design-contingent but consistent
[inference] A layered STAMP plus systems-thinking plus NIST plus DORA-CISA stack is the most predictive integrated framework. https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/; https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services medium Synthesis rather than named standard
[inference] Speed-over-quality creates a reinforcing loop of shadow demand, rising change volume, weak review, and higher incident exposure. https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk high Cross-source convergence
[inference] Risk-reduction-first sequencing has the clearest support when AI is used first for security, monitoring, and platform context. https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services medium Best evidence is indirect
[inference] The correct enterprise experiment compares rollout sequence on shadow use, privileged actions, stability, incidents, and skill retention. https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/reports/data-breach; https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143 medium Proposed measurement design
[inference] Bounded autonomy with least privilege, explicit actions, and deterministic review for irreversible acts is the strongest practical control pattern. https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://www.anthropic.com/research/trustworthy-agents high Strong operational convergence

Assumptions

  • The management and security guidance drawn from critical infrastructure, software, and major-platform environments generalises to broader enterprise deployments because the relevant control surfaces, permissions, logging, review rights, and intervention paths, are shared. [assumption; source: https://sloanreview.mit.edu/article/agentic-ai-at-scale-redefining-management-for-a-superhuman-workforce/; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services]
  • The skill-decay evidence, which includes medicine and more general information-systems work, is directionally applicable to enterprise AI operations because the shared mechanism is reduced human practice in judgment, verification, and recovery tasks. [assumption; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143]

Analysis

STAMP and systems thinking carry most of the causal weight here, since the four target risks reinforce one another over time and would be flattened by a single-cause framework. [inference; source: https://archive.org/details/mit_press_book_9780262298247; https://www.chelseagreen.com/product/thinking-in-systems/]

By contrast, NIST matters because it turns a cascade story into an intervention map through inventory, role clarity, monitoring, risk tolerance, and decommissioning guidance. [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook]

DORA, IBM, and Cyberhaven were weighted for operational signal because they quantify what happens when user demand outruns sanctioned platforms, even though some of that evidence is vendor-produced rather than fully independent. [inference; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk]

CISA, Microsoft, and Anthropic are the strongest differentiators between agentic and generative deployment because they describe the action layer, not only the harm categories. [inference; source: https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents]

The remaining uncertainty sits around the sequencing claim itself: current evidence strongly favors safety nets, internal platforms, and AI-enabled security, but it still falls short of a standardised longitudinal benchmark for rollout order. [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services]

Risks, Gaps, and Uncertainties

  • Public evidence on agentic AI is still weighted toward official guidance and provider experiments, so long-horizon enterprise incident datasets remain thin. [inference; source: https://www.cisa.gov/news-events/news/cisa-us-and-international-partners-release-guide-secure-adoption-agentic-ai; https://learn.microsoft.com/en-us/security/zero-trust/sfi/secure-agentic-systems; https://www.anthropic.com/research/trustworthy-agents]
  • Shadow AI prevalence evidence is directionally consistent across sources, but precise magnitudes should be treated cautiously because two of the strongest public sources are vendor-affiliated. [inference; source: https://www.ibm.com/think/insights/rising-ai-adoption-creating-shadow-risks; https://www.ibm.com/reports/data-breach; https://www.cyberhaven.com/blog/shadow-ai-how-employees-are-leading-the-charge-in-ai-adoption-and-putting-company-data-at-risk]
  • Skill outcomes remain design-contingent, so enterprises should avoid treating deskilling as inevitable and instead measure whether work design is producing upskilling or deskilling. [inference; source: https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00572-8; https://publicera.kb.se/ir/article/view/47143]
  • Return-on-investment evidence for risk-reduction-first sequencing is strongest in adjacent signals, security savings and delivery stability, not yet in direct head-to-head rollout trials. [inference; source: https://www.ibm.com/reports/data-breach; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report]

Open Questions

  • Which enterprise sectors will publish the first credible longitudinal comparisons of autonomy-first and risk-reduction-first deployment sequences?
  • Which skill-maintenance measures are the best leading indicators that human exception-handling capability is degrading before incidents reveal it?
  • What is the minimum viable observability package for agentic systems that preserves auditability without recreating the same review bottlenecks it is meant to reduce?

Output

Navigation

Home

By Tag

bureaucracy

change-management

coase

constraint-analysis

control-model

decision-rights

delegation

delivery-risk

demand-segmentation

enterprise

exception-handling

execution

flow

flow-design

flow-metrics

governance

governance-patterns

incentives

instability

institutional-economics

leading-indicators

operating-model

organisation

organisational-design

queue-design

queueing

regulated-enterprise

routing

throughput

throughput-risk

transaction-costs

triage

williamson

Clone this wiki locally