Skip to content

2026 04 26 ai lowcode failure modes governance mitigation

github-actions[bot] edited this page Apr 30, 2026 · 2 revisions

What are the primary failure modes in enterprise Artificial Intelligence (AI) and low-code deployments, and how can governance systems be designed to mitigate them?

Research Question

What are the primary failure modes in enterprise Artificial Intelligence (AI) and low-code deployments, including data leakage, conflicting automations, unintended actions by AI agents, and loss of auditability, and how can governance systems be designed with preventative and corrective controls that address each identified failure scenario?

Scope

In scope:

  • Adversarial and failure-focused analysis of AI and low-code deployments in enterprise environments
  • Concrete failure scenarios: data leakage (intentional and inadvertent), conflicting or cascading automations, unintended or unsafe agent actions, loss of auditability, unauthorized privilege escalation, prompt injection in AI agents, silent failures (incorrect output without error signal), and feedback loop failures
  • Preventative controls: design-time controls that reduce the probability of failure (permission scoping, output constraints, sandboxing, review gates)
  • Corrective controls: runtime and post-incident controls that reduce the impact of failure (circuit breakers, rollback mechanisms, incident response playbooks, audit reconstruction)
  • Whether failure modes are common across AI and low-code systems or specific to each
  • Evidence from the literature on AI system failure in production (published incident analyses, red team findings, failure taxonomies)

Out of scope:

  • Model-level failure (hallucination, bias, distributional shift) unless it produces enterprise governance failures
  • Security vulnerabilities in AI model training or supply chain (focus is on deployment and governance failures)
  • Failure modes specific to AI research or experimental settings (focus is production enterprise deployment)

Constraints:

  • Distinguish between failure modes that are preventable by governance design and failure modes that are inherent to AI systems regardless of governance
  • Sources must include empirical evidence (incident reports, red team studies, production failure analyses) not just theoretical taxonomies

Context

  • [inference; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-lowcode-decision-rights-accountability-liability.html] Prior completed repository work already shows that bounded maker enablement, explicit accountability, and a central control plane are prerequisites for safe enterprise scale, so this item focuses on the failure taxonomy those governance structures must actually handle.
  • [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] The practical design problem is to connect adversarial AI failure modes and low-code change failure modes to concrete preventative and corrective controls, because modern enterprise platforms already expose policy, environment, and monitoring surfaces that can be used if the taxonomy is specific enough.

Cross-references:

  • Q3: 2026-04-26-ai-lowcode-governance-enforcement-architecture
  • Q4: 2026-04-26-ai-lowcode-observability-telemetry-governance
  • Q9: 2026-04-26-human-in-the-loop-ai-automated-workflows
  • Q16: 2026-04-26-ai-agent-control-plane-architecture-enterprise

Approach

  1. Failure taxonomy construction: Review the literature on AI system failures in production and construct a failure taxonomy covering data exposure failures, action failures, attribution failures, coordination failures, and systemic failures.
  2. Low-code-specific failure analysis: Identify failure modes specific to low-code deployments, including citizen-developer-created automations without adequate testing, fragile integrations, and ungoverned access to enterprise Application Programming Interfaces (APIs), and assess the overlap with AI agent failures.
  3. Adversarial failure analysis: Review prompt injection research, adversarial input studies, and social engineering scenarios specific to AI agent deployments; assess what governance controls can and cannot prevent adversarially induced failures.
  4. Control mapping: For each identified failure mode, map preventative controls (design-time, deployment-time) and corrective controls (runtime, post-incident).
  5. Detection gap analysis: Identify failure modes where current observability and monitoring approaches are insufficient to detect the failure before significant harm occurs.
  6. Synthesis: Produce a failure mode analysis for enterprise AI and low-code governance, with explicit preventative and corrective control recommendations for each failure mode.

Sources

Related


Research Skill Output

(Full output from running the research skill - retained verbatim in the completed item. Sections 0-5 are the investigation; section 6 seeds the Findings section below.)

§0 Initialise

  • [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Research question restated: Which enterprise Artificial Intelligence (AI) and low-code failure modes create the most material governance risk in production, and which preventative and corrective controls directly reduce the likelihood or blast radius of each one?
  • [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8] Scope confirmed: the investigation covers prompt injection, data leakage, unintended actions, silent failure, fragile low-code change, audit loss, conflicting automation, and feedback-loop failure across enterprise deployments rather than laboratory model behavior.
  • [fact; source: https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] Constraints confirmed: empirical or production-oriented evidence was prioritized, inaccessible analyst material was checked but excluded from downstream claims, and the output must distinguish governance-preventable failure from residual model uncertainty.
  • [fact; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-lowcode-decision-rights-accountability-liability.html] Prior work cross-reference: prior completed items already established bounded maker enablement, activity-based accountability, and a central control plane as architectural prerequisites, so this item extends that work by specifying the failure classes those controls must govern.
  • [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/] Output format confirmed: knowledge, specifically a failure taxonomy with preventative controls, corrective controls, and detection-gap analysis.

§1 Question Decomposition

  • Root question: Which failures matter most when enterprise AI and low-code systems can read, decide, and act across business systems?
  • A. Data exposure
    • A1. How do prompt injection and external-content ingestion create sensitive-information disclosure or data exfiltration?
    • A2. Which controls reduce disclosure risk before and after compromise?
  • B. Unintended action
    • B1. When do model outputs turn into unauthorized or unsafe actions?
    • B2. Which control patterns limit action scope and require approval?
  • C. Silent failure
    • C1. How do plausible but incorrect outputs become enterprise governance failures?
    • C2. Which controls create verification, containment, and accountability?
  • D. Low-code coordination failure
    • D1. Which low-code failure modes stem from citizen-development sprawl, weak promotion discipline, or fragile integrations?
    • D2. Which platform controls reduce conflicting automation and release fragility?
  • E. Auditability and attribution
    • E1. What causes loss of auditability or poor reconstruction after incidents?
    • E2. Which logging and lifecycle controls restore traceability?
  • F. Detection gap
    • F1. Which failures remain hard to detect early even with current monitoring?
    • F2. Which corrective controls are required because prevention is incomplete?
  • G. Common versus specific
    • G1. Which failures are shared across AI and low-code systems?
    • G2. Which failures are AI-specific or low-code-specific?

§2 Investigation

Source access and replacement notes

  • [fact; source: https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook] Replacement source used: the current NIST playbook landing page rather than older seed links.

A. Data exposure through prompt injection, retrieval, and memory

  • [fact; source: https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://owasp.org/www-community/attacks/PromptInjection] Prompt injection is a first-order enterprise failure mode because user or external content can alter a Large Language Model (LLM) application's behavior in unintended ways, including disclosing sensitive information, enabling unauthorized access, and manipulating critical decisions.
  • [fact; source: https://arxiv.org/abs/2302.12173] The indirect prompt injection paper demonstrates that LLM-integrated applications blur the line between data and instructions, and that remotely injected prompts can cause data theft, arbitrary code-like behavior, and manipulated Application Programming Interface (API) calls in real-world systems.
  • [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Microsoft states that indirect prompt injection is one of the most widely used techniques reported to it against AI systems and that security impacts range from exfiltration of user data to unintended actions performed with the user's credentials.
  • [fact; source: https://arxiv.org/abs/2307.15043] The adversarial suffix paper shows that jailbreak-style attacks can be generated automatically and transfer across multiple aligned and black-box models, so provider choice alone is not a reliable mitigation.
  • [fact; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] The Unit 42 proof of concept shows that indirect prompt injection can poison long-term agent memory, persist across sessions, and later drive silent exfiltration of conversation history through an agent's own tools.
  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Data exposure failures in enterprise AI are therefore boundary-failure problems: once untrusted content is allowed into the prompt, tool output, or memory path, confidentiality depends less on model alignment and more on privilege scoping, content segregation, and outbound-channel control.

B. Unintended actions, excessive agency, and privilege amplification

  • [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Microsoft documents that indirect prompt injection can trigger unintended actions such as phishing emails sent from trusted accounts or remote command execution through tools available to the agent.
  • [fact; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance] Copilot Studio administrators can centrally block unauthenticated chat, public-website knowledge sources, document knowledge sources, Hypertext Transfer Protocol (HTTP) requests, skills, publishing channels, and event triggers, which are exactly the action surfaces through which excessive agency becomes consequential.
  • [fact; source: https://genai.owasp.org/llmrisk/llm01-prompt-injection/] The Open Worldwide Application Security Project (OWASP) recommends least privilege, segregated external content, validated output formats, and human approval for high-risk actions because prompt injection severity depends on the degree of agency the system has been granted.
  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Unintended action is rarely a model-only failure; it is usually a composition failure in which an over-privileged toolset, over-broad publishing surface, or missing approval gate converts model misbehavior into business-side effects.

C. Silent failure, overreliance, and no-error wrong answers

  • [fact; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] The Air Canada chatbot case is a production example of silent failure: the chatbot gave plausible but incorrect bereavement-fare advice, the customer acted on it, and the tribunal held Air Canada responsible for inaccurate information on its website regardless of whether the source was a chatbot or a static page.
  • [fact; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] DevOps Research and Assessment (DORA) reports that Artificial Intelligence (AI) amplifies existing weaknesses and that instability remains negatively related to AI adoption unless organizations already have strong automated testing, mature version control, and fast feedback loops.
  • [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Silent failure is a governance failure when output is trusted enough to change customer-facing or operational behavior without an independent verification step, because the absence of an explicit error signal does not reduce legal or operational liability.

D. Low-code sprawl, fragile automations, and conflicting change

  • [fact; source: https://aisel.aisnet.org/misqe/vol23/iss3/6] The 2024 MIS Quarterly Executive article identifies substandard software quality, shadow Information Technology (IT), and technical debt as core low-code citizen-development risks and argues that low-code governance differs from traditional software governance because it must explicitly involve technical experts and platform-specific guardrails.
  • [fact; source: https://link.springer.com/article/10.1007/s10257-022-00553-8] The Springer study states that reliable and scalable robotic process automation still requires expert knowledge and cites estimates that up to 50% of initial robotic process automation implementations fail, which the paper uses to argue for more systematic socio-technical implementation support.
  • [fact; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] Digital.gov recommends centralized repositories, code review, role-based access control, and separate development, test, and production environments for citizen development because governance and automation security are major concerns.
  • [fact; source: https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/admin/managed-governance] Microsoft's managed-governance features route makers into managed personal developer environments, apply environment-group rules, preconfigure sharing limits and solution checks, and expose inventory, actions, and recommendations for reactive governance.
  • [fact; source: https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Power Platform pipelines enforce sequential stage promotion, prevalidate dependencies and environment variables, support approval-based delegated deployments, store deployment artifacts, and enable rollback to previous versions.
  • [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing] Conflicting automations and fragile low-code change emerge when local builders can author, test, and publish in shared or weakly governed spaces without enforced promotion discipline, because the platform otherwise accumulates overlapping logic, brittle connectors, and unclear ownership.

E. Auditability loss and post-incident reconstruction failure

  • [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/] The National Institute of Standards and Technology (NIST) Govern 1.5 outcome requires ongoing monitoring and periodic review with clearly defined roles and responsibilities, Govern 4.3 calls for practices that enable testing and incident identification, and Govern 6.2 requires contingency processes for failures in third-party data or AI systems deemed high risk.
  • [fact; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Microsoft documents maker audit logs in Microsoft Purview, audit logs and alerts in Microsoft Sentinel, and pipeline customization and audit logs saved automatically, which are the vendor-native traces needed for attribution and rollback.
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Loss of auditability is not only missing logs; it is the inability to reconstruct which prompt, knowledge source, connector, identity, approval, and deployment artifact produced a consequential output or action, so lifecycle and runtime evidence have to be designed as one chain.

F. Detection gaps and why corrective controls remain necessary

  • [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Microsoft says indirect prompt injection is an inherent risk of modern LLM-based systems, that deterministic detection remains an open research challenge, and that defense therefore has to span prevention, detection, and impact mitigation.
  • [fact; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Power Platform states that data-policy changes propagate across environments and resources asynchronously, that violating apps, flows, and chatbots may later be suspended or quarantined, and that full enforcement can take up to 24 hours in extreme cases.
  • [fact; source: https://www.anthropic.com/responsible-scaling-policy] Anthropic's Responsible Scaling Policy describes a layered safeguard stack with access controls, real-time prompt and completion classifiers, asynchronous monitoring classifiers, post-hoc jailbreak detection, and rapid response procedures, explicitly assuming that no single layer is sufficient.
  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy] The control implication is that preventative governance alone is insufficient: some failure modes will only be caught after execution has begun, so rate limits, reversible workflows, containment, monitoring, and practiced incident response are not secondary controls but core design requirements.

G. Common versus system-specific failure classes

  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Shared failure classes across AI and low-code systems are boundary failure, privilege failure, promotion failure, and audit failure, because both types of systems break when action paths cross systems without scoped authority, staged release, or traceable evidence.
  • [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] AI-specific failure classes are prompt injection, retrieval contamination, memory poisoning, and stochastic silent error, because they depend on the model's inability to reliably separate data from instructions or certainty from correctness.
  • [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing] Low-code-specific failure classes are citizen-development sprawl, shared-environment collision, connector sprawl, and promotion bypass, because they arise from democratized build surfaces and not from probabilistic model behavior itself.

§3 Reasoning

  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://arxiv.org/abs/2302.12173; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] The strongest AI-specific evidence is consistent on one point: the main enterprise failure is not that a model "hallucinates" in the abstract, but that untrusted input can hijack tool use, memory, or retrieval context and turn the system into an unauthorized actor.
  • [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code literature points to the same organizational mechanism from a different angle: decentralized builders can create value only when central engineering and governance functions constrain the build, test, release, and ownership surfaces.
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls reduce the chance of bad states by narrowing what can be built, connected, or published, while corrective controls reduce loss after inevitable misses by making the estate observable, reversible, and attributable.
  • [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Silent failure matters because enterprise harm often arrives before anyone recognizes a technical incident: customers act on wrong advice, staff act on bad summaries, and throughput gains amplify the reach of each unnoticed error.

§4 Consistency Check

  • [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://www.anthropic.com/responsible-scaling-policy] No contradiction remains between prevention-heavy and monitoring-heavy sources, because both explicitly describe layered defenses and treat prevention alone as insufficient.
  • [fact; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] No contradiction remains between low-code democratization and central governance, because the low-code evidence supports bounded self-service inside centrally administered lifecycle controls rather than uncontrolled maker autonomy.
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] The evidence supports treating silent misinformation as a governance failure even when no malicious actor is present, because the accountability and monitoring obligations remain regardless of attack intent.

§5 Depth and Breadth Expansion

  • [inference; source: https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Manage] Regulatory and control lens: the National Institute of Standards and Technology (NIST) playbook makes incident response, change management, monitoring cadence, and third-party coverage explicit, which means enterprise governance cannot stop at model safety policy and must cover operational failure management.
  • [inference; source: https://cloud.google.com/resources/content/2025-dora-ai-capabilities-model-report; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Economic and operating-model lens: DORA implies that AI and low-code failures become economically material when they are multiplied by poor workflow systems, so investment in platform teams and internal platforms is itself a risk-reduction strategy.
  • [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/managed-governance; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Behavioral lens: many low-code failures are desire-path failures, where makers choose the shortest path available; managed environments, routing, and preconfigured pipelines matter because they make the safer path the easier path.
  • [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Temporal lens: AI failures can persist or compound over time through memory, retrieval, and asynchronous execution, so post-incident cleanup must include state reset, not only prompt or policy edits.

§6 Synthesis

(This section seeds the Findings below.)

Executive summary:

  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] The primary enterprise failure modes in Artificial Intelligence (AI) and low-code deployments are boundary failures, specifically untrusted content crossing into prompts or memory, over-broad action authority, silent wrong outputs, uncontrolled maker publication, and missing end-to-end audit trails, but their frequency and severity are amplified by broader delivery immaturity and systems-capability debt.
  • [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Artificial Intelligence (AI)-specific failures are concentrated in prompt injection, retrieval contamination, and persistent memory poisoning, while low-code-specific failures are concentrated in citizen-development sprawl, connector sprawl, fragile promotion, and conflicting automations.
  • [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls should therefore narrow who can build, what can connect, where work can run, and how changes can promote, using managed environments, scoped connectors, authenticated channels, least privilege, and sequential release gates.
  • [inference; source: https://www.anthropic.com/responsible-scaling-policy; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern] Corrective controls must assume prevention will miss material cases, so runtime monitoring, immutable logs, quarantine, rollback, rate limits, memory reset, and practiced incident response are core governance components rather than optional add-ons.

Key findings:

  1. [inference; source: https://arxiv.org/abs/2302.12173; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html] High confidence: Prompt injection is a primary enterprise failure mode because untrusted external content can be reinterpreted as instructions, causing data exfiltration, unauthorized tool use, and manipulated downstream decisions in real-world Large Language Model (LLM) applications.
  2. [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html] High confidence: Unintended actions become materially dangerous only when agents are granted excessive authority, so the decisive governance control is not better prompting alone but least privilege, narrowed tool surfaces, authenticated identities, and human approval for high-impact actions.
  3. [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Persistent memory and asynchronous agent workflows create a distinctive Artificial Intelligence (AI) failure class in which poisoned state can survive beyond the triggering session, so governance must include memory scoping, memory reset, and post-incident state cleanup.
  4. [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] High confidence: Silent but plausible wrong answers are governance failures rather than mere quality issues when organizations let operational or customer decisions depend on model output without verification, because liability and instability arrive before explicit technical alarms do.
  5. [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] High confidence: Low-code estates fail through shadow Information Technology (IT), technical debt, and conflicting automation when citizen developers can build and publish without centralized repositories, expert review, role clarity, and separate development, test, and production environments, and that pattern is reinforced when systems-capability debt keeps generating workaround demand.
  6. [inference; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html] High confidence: A consistently supported preventative control pattern is managed isolation plus staged promotion, specifically controlled maker environments, scoped connectors and channels, prevalidated deployments, and approval-based release gates, because those controls jointly remove the easiest unmanaged path from authoring to production.
  7. [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] High confidence: Loss of auditability is a primary failure mode because enterprises cannot reconstruct accountability without one evidence chain spanning prompts, knowledge sources, identities, approvals, runtime activity, and deployment artifacts.
  8. [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Detection gaps remain material because prompt injection is not deterministically solvable today and low-code policy enforcement can lag across a large tenant, so circuit breakers, quarantine, rollback, rate limiting, and rehearsed incident response are indispensable corrective controls.
  9. [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] High confidence: The shared governance pattern across Artificial Intelligence (AI) and low-code systems is that failures concentrate at control-boundary crossings, while recurring severity is amplified by delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence; the main Artificial Intelligence (AI)-specific additions are instruction ambiguity and persistent state and the main low-code-specific additions are maker sprawl and release fragility.

Evidence map:

Claim Source Confidence Notes
[inference] Prompt injection is a primary enterprise failure mode because external content can become instructions that drive disclosure or unauthorized actions. https://arxiv.org/abs/2302.12173 ; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://genai.owasp.org/llmrisk/llm01-prompt-injection/ ; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html high Cross-verified across academic, vendor, standards-style, and prior architecture evidence.
[inference] Excessive authority is what turns model failure into business-side effects, so least privilege and approval gates are decisive controls. https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention ; https://genai.owasp.org/llmrisk/llm01-prompt-injection/ ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html high External mitigation guidance aligns with prior completed work on identity and access amplification.
[inference] Persistent memory creates a distinct failure class because poisoned instructions can survive sessions and later drive silent exfiltration. https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/ ; https://www.anthropic.com/responsible-scaling-policy medium Direct proof of concept plus supporting layered-safeguard logic, but public production incident evidence is thinner.
[inference] Silent plausible wrong answers become governance failures when they drive customer or operational decisions without verification. https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416 ; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report high Production liability case plus platform-level evidence that AI amplifies weak controls.
[inference] Low-code estates fail through shadow IT, debt, and conflicting automation when maker publication is not centrally governed, and systems-capability debt keeps generating workaround demand. https://aisel.aisnet.org/misqe/vol23/iss3/6 ; https://link.springer.com/article/10.1007/s10257-022-00553-8 ; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/ ; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html high Academic, public-sector, and prior completed empirical synthesis all point to the same workaround mechanism.
[inference] Managed isolation and staged promotion are a consistently supported preventative platform control pattern because they remove the easiest unmanaged path from authoring to production. https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/ ; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing ; https://learn.microsoft.com/en-us/power-platform/alm/pipelines ; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html high Uses both Microsoft platform evidence and independent governance evidence without making a platform-wide superiority claim.
[inference] Auditability fails when enterprises cannot connect runtime behavior to identities, approvals, and deployment artifacts. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ ; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance ; https://learn.microsoft.com/en-us/power-platform/alm/pipelines ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html high NIST obligation plus concrete vendor evidence and prior identity-governance synthesis.
[inference] Corrective controls are mandatory because prompt injection is not fully preventable, access failures can compound downstream effects, and low-code policy enforcement can lag. https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention ; https://www.anthropic.com/responsible-scaling-policy ; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html medium Strong design logic plus prior completed work on rate-limiting and access-control amplification.
[inference] Failures cluster at control-boundary crossings across both AI and low-code systems, while delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence explain why those crossings proliferate and recur. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ ; https://aisel.aisnet.org/misqe/vol23/iss3/6 ; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report ; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html ; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html ; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/ high Common mechanism and adjacent alternative explanations are both represented.

Assumptions:

  • [assumption; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://link.springer.com/article/10.1007/s10257-022-00553-8] Assumption: Robotic process automation governance lessons transfer materially to low-code application and agent programs. Justification: both shift automation authoring toward non-specialists while relying on central platform controls for safe release and support.
  • [assumption; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Assumption: Publicly documented prompt injection patterns on general LLM applications are representative of the same risk class in enterprise internal assistants. Justification: the cited sources describe the vulnerability as architectural and content-path dependent, not consumer-product specific.

Analysis:

  • [inference; source: https://arxiv.org/abs/2302.12173; https://arxiv.org/abs/2307.15043; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] The most convincing evidence for Artificial Intelligence (AI)-specific failure is the combination of academic attack papers and Microsoft's production guidance, because together they show both feasibility and practical impact. The attack papers establish that adversarial prompt control is real and transferable, while Microsoft establishes that the same failure class appears in enterprise deployments and requires layered mitigations.
  • [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code evidence is less about spectacular exploits and more about repeated organizational failure, which is exactly what governance needs. Across academic, public-sector, and enterprise platform guidance, the same preventive pattern repeats: isolate build spaces, centralize repositories, enforce review, and separate promotion from authoring.
  • [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] That consistency matters because it means governance is not merely aspirational. The platform surfaces already exist to narrow knowledge sources, channels, triggers, maker environments, and release paths, so the remaining problem is disciplined design and operating-model enforcement.
  • [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.anthropic.com/responsible-scaling-policy] The evidence also weighs against a purely preventative mindset. Silent failure, incomplete prompt-injection prevention, and post-hoc safeguard layers all point to the same conclusion: enterprises need reversible workflows, emergency stop paths, incident playbooks, and state cleanup because some failures will only be visible after an output or action has already occurred.

Risks, gaps, uncertainties:

  • [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] Public evidence for memory-poisoning incidents is still thinner than public evidence for prompt injection generally, so the memory-governance recommendations are based on a high-quality proof of concept plus architectural reasoning rather than on a large public incident corpus.
  • [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Vendor documentation confirms enforcement latency and runtime quarantine behavior, but it does not quantify how often material harm occurs before enforcement completes, so the size of this exposure window is still uncertain.

Open questions:

  • [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Which public platform patterns are most effective for memory sanitization and trusted-state reconstruction after agent compromise?
  • [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] How should enterprises quantify acceptable enforcement lag and rollback time across large low-code estates with mixed maker maturity?
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] What minimum monitoring and approval set is sufficient for medium-risk write-capable agents before the control burden outweighs the productivity gain?

§7 Recursive Review

  • [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Each substantive section now resolves to sourced facts, explicit inferences, or explicit assumptions, and the taxonomy remains inside the original scope of enterprise deployment and governance failure.
  • [fact; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html] Prior completed repository work was used only as context and synthesis support, not as a substitute for external evidence.
  • [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] The final synthesis is internally consistent: failure modes are grouped by mechanism, controls are mapped by preventative versus corrective purpose, and the remaining uncertainties are explicitly carried into Risks, Gaps, and Uncertainties.

Findings

Executive Summary

[inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] The primary enterprise failure modes in Artificial Intelligence (AI) and low-code deployments are boundary failures, specifically untrusted content crossing into prompts or memory, over-broad action authority, silent wrong outputs, uncontrolled maker publication, and missing end-to-end audit trails, but their frequency and severity are amplified by broader delivery immaturity and systems-capability debt. [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Artificial Intelligence (AI)-specific failures are concentrated in prompt injection, retrieval contamination, and persistent memory poisoning, while low-code-specific failures are concentrated in citizen-development sprawl, connector sprawl, fragile promotion, and conflicting automations. [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls should therefore narrow who can build, what can connect, where work can run, and how changes can promote, using managed environments, scoped connectors, authenticated channels, least privilege, and sequential release gates. [inference; source: https://www.anthropic.com/responsible-scaling-policy; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern] Corrective controls must assume prevention will miss material cases, so runtime monitoring, immutable logs, quarantine, rollback, rate limits, memory reset, and practiced incident response are core governance components rather than optional add-ons.

Key Findings

  1. [inference; source: https://arxiv.org/abs/2302.12173; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] High confidence: Prompt injection is a primary enterprise failure mode because untrusted external content can be reinterpreted as instructions, causing data exfiltration, unauthorized tool use, and manipulated downstream decisions in real-world Large Language Model (LLM) applications.
  2. [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] High confidence: Unintended actions become materially dangerous only when agents are granted excessive authority, so the decisive governance control is not better prompting alone but least privilege, narrowed tool surfaces, authenticated identities, and human approval for high-impact actions.
  3. [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Persistent memory and asynchronous agent workflows create a distinctive Artificial Intelligence (AI) failure class in which poisoned state can survive beyond the triggering session, so governance must include memory scoping, memory reset, and post-incident state cleanup.
  4. [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] High confidence: Silent but plausible wrong answers are governance failures rather than mere quality issues when organizations let operational or customer decisions depend on model output without verification, because liability and instability arrive before explicit technical alarms do.
  5. [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] High confidence: Low-code estates fail through shadow Information Technology (IT), technical debt, and conflicting automation when citizen developers can build and publish without centralized repositories, expert review, role clarity, and separate development, test, and production environments, and that pattern is reinforced when systems-capability debt keeps generating workaround demand.
  6. [inference; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html] High confidence: A consistently supported preventative control pattern is managed isolation plus staged promotion, specifically controlled maker environments, scoped connectors and channels, prevalidated deployments, and approval-based release gates, because those controls jointly remove the easiest unmanaged path from authoring to production.
  7. [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html] High confidence: Loss of auditability is a primary failure mode because enterprises cannot reconstruct accountability without one evidence chain spanning prompts, knowledge sources, identities, approvals, runtime activity, and deployment artifacts.
  8. [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html] Medium confidence: Detection gaps remain material because prompt injection is not deterministically solvable today, access failures can compound downstream effects, and low-code policy enforcement can lag across a large tenant, so circuit breakers, quarantine, rollback, rate limiting, and rehearsed incident response are indispensable corrective controls.
  9. [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] High confidence: The shared governance pattern across Artificial Intelligence (AI) and low-code systems is that failures concentrate at control-boundary crossings, while recurring severity is amplified by delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence; the main Artificial Intelligence (AI)-specific additions are instruction ambiguity and persistent state and the main low-code-specific additions are maker sprawl and release fragility.

Evidence Map

Claim Source Confidence Notes
[inference] Prompt injection is a primary enterprise failure mode because external content can become instructions that drive disclosure or unauthorized actions. https://arxiv.org/abs/2302.12173 ; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://genai.owasp.org/llmrisk/llm01-prompt-injection/ ; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html high Cross-verified across academic, vendor, standards-style, and prior architecture evidence.
[inference] Excessive authority is what turns model failure into business-side effects, so least privilege and approval gates are decisive controls. https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention ; https://genai.owasp.org/llmrisk/llm01-prompt-injection/ ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html high External mitigation guidance aligns with prior completed work on identity and access amplification.
[inference] Persistent memory creates a distinct failure class because poisoned instructions can survive sessions and later drive silent exfiltration. https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/ ; https://www.anthropic.com/responsible-scaling-policy medium Strong proof of concept, thinner public incident corpus.
[inference] Silent plausible wrong answers become governance failures when they drive customer or operational decisions without verification. https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416 ; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report high Production liability case plus control-amplification evidence.
[inference] Low-code estates fail through shadow IT, debt, and conflicting automation when maker publication is not centrally governed, and systems-capability debt keeps generating workaround demand. https://aisel.aisnet.org/misqe/vol23/iss3/6 ; https://link.springer.com/article/10.1007/s10257-022-00553-8 ; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/ ; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html high Academic, public-sector, and prior completed empirical synthesis all point to the same workaround mechanism.
[inference] Managed isolation and staged promotion are a consistently supported preventative platform control pattern because they remove the easiest unmanaged path from authoring to production. https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/ ; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing ; https://learn.microsoft.com/en-us/power-platform/alm/pipelines ; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html high Uses both Microsoft platform evidence and independent governance evidence without making a platform-wide superiority claim.
[inference] Auditability fails when enterprises cannot connect runtime behavior to identities, approvals, and deployment artifacts. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ ; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance ; https://learn.microsoft.com/en-us/power-platform/alm/pipelines ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html high NIST obligation plus concrete vendor evidence and prior identity-governance synthesis.
[inference] Corrective controls are mandatory because prompt injection is not fully preventable, access failures can compound downstream effects, and low-code policy enforcement can lag. https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks ; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention ; https://www.anthropic.com/responsible-scaling-policy ; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html medium Strong design logic plus prior completed work on rate-limiting and access-control amplification.
[inference] Failures cluster at control-boundary crossings across both AI and low-code systems, while delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence explain why those crossings proliferate and recur. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ ; https://aisel.aisnet.org/misqe/vol23/iss3/6 ; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report ; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html ; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html ; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html ; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html ; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/ high Common mechanism and adjacent alternative explanations are both represented.

Assumptions

  • [assumption; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://link.springer.com/article/10.1007/s10257-022-00553-8] Assumption: Robotic process automation governance lessons transfer materially to low-code application and agent programs. Justification: both shift automation authoring toward non-specialists while relying on central platform controls for safe release and support.
  • [assumption; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Assumption: Publicly documented prompt injection patterns on general LLM applications are representative of the same risk class in enterprise internal assistants. Justification: the cited sources describe the vulnerability as architectural and content-path dependent, not consumer-product specific.

Analysis

[inference; source: https://arxiv.org/abs/2302.12173; https://arxiv.org/abs/2307.15043; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] The most convincing evidence for Artificial Intelligence (AI)-specific failure is the combination of academic attack papers and Microsoft's production guidance, because together they show both feasibility and practical impact. [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code evidence is less about spectacular exploits and more about repeated organizational failure, which is exactly what governance needs; across academic, public-sector, and enterprise platform guidance, the same preventive pattern repeats: isolate build spaces, centralize repositories, enforce review, and separate promotion from authoring. [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] That consistency matters because it means governance is not merely aspirational: the platform surfaces already exist to narrow knowledge sources, channels, triggers, maker environments, and release paths, so the remaining problem is disciplined design and operating-model enforcement. [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.anthropic.com/responsible-scaling-policy] The evidence also weighs against a purely preventative mindset, because silent failure, incomplete prompt-injection prevention, and post-hoc safeguard layers all point to the same conclusion: enterprises need reversible workflows, emergency stop paths, incident playbooks, and state cleanup because some failures will only be visible after an output or action has already occurred.

Risks, Gaps, and Uncertainties

  • [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] Public evidence for memory-poisoning incidents is still thinner than public evidence for prompt injection generally, so the memory-governance recommendations are based on a high-quality proof of concept plus architectural reasoning rather than on a large public incident corpus.
  • [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Vendor documentation confirms enforcement latency and runtime quarantine behavior, but it does not quantify how often material harm occurs before enforcement completes, so the size of this exposure window is still uncertain.

Open Questions

  • [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Which public platform patterns are most effective for memory sanitization and trusted-state reconstruction after agent compromise?
  • [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] How should enterprises quantify acceptable enforcement lag and rollback time across large low-code estates with mixed maker maturity?
  • [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] What minimum monitoring and approval set is sufficient for medium-risk write-capable agents before the control burden outweighs the productivity gain?

Output

Navigation

Home

By Tag

bureaucracy

change-management

coase

constraint-analysis

control-model

decision-rights

delegation

delivery-risk

demand-segmentation

enterprise

exception-handling

execution

flow

flow-design

flow-metrics

governance

governance-patterns

incentives

instability

institutional-economics

leading-indicators

operating-model

organisation

organisational-design

queue-design

queueing

regulated-enterprise

routing

throughput

throughput-risk

transaction-costs

triage

williamson

Clone this wiki locally