-
Notifications
You must be signed in to change notification settings - Fork 0
2026 04 26 ai lowcode failure modes governance mitigation
What are the primary failure modes in enterprise Artificial Intelligence (AI) and low-code deployments, and how can governance systems be designed to mitigate them?
What are the primary failure modes in enterprise Artificial Intelligence (AI) and low-code deployments, including data leakage, conflicting automations, unintended actions by AI agents, and loss of auditability, and how can governance systems be designed with preventative and corrective controls that address each identified failure scenario?
In scope:
- Adversarial and failure-focused analysis of AI and low-code deployments in enterprise environments
- Concrete failure scenarios: data leakage (intentional and inadvertent), conflicting or cascading automations, unintended or unsafe agent actions, loss of auditability, unauthorized privilege escalation, prompt injection in AI agents, silent failures (incorrect output without error signal), and feedback loop failures
- Preventative controls: design-time controls that reduce the probability of failure (permission scoping, output constraints, sandboxing, review gates)
- Corrective controls: runtime and post-incident controls that reduce the impact of failure (circuit breakers, rollback mechanisms, incident response playbooks, audit reconstruction)
- Whether failure modes are common across AI and low-code systems or specific to each
- Evidence from the literature on AI system failure in production (published incident analyses, red team findings, failure taxonomies)
Out of scope:
- Model-level failure (hallucination, bias, distributional shift) unless it produces enterprise governance failures
- Security vulnerabilities in AI model training or supply chain (focus is on deployment and governance failures)
- Failure modes specific to AI research or experimental settings (focus is production enterprise deployment)
Constraints:
- Distinguish between failure modes that are preventable by governance design and failure modes that are inherent to AI systems regardless of governance
- Sources must include empirical evidence (incident reports, red team studies, production failure analyses) not just theoretical taxonomies
- [inference; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-lowcode-decision-rights-accountability-liability.html] Prior completed repository work already shows that bounded maker enablement, explicit accountability, and a central control plane are prerequisites for safe enterprise scale, so this item focuses on the failure taxonomy those governance structures must actually handle.
- [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] The practical design problem is to connect adversarial AI failure modes and low-code change failure modes to concrete preventative and corrective controls, because modern enterprise platforms already expose policy, environment, and monitoring surfaces that can be used if the taxonomy is specific enough.
Cross-references:
- Q3:
2026-04-26-ai-lowcode-governance-enforcement-architecture - Q4:
2026-04-26-ai-lowcode-observability-telemetry-governance - Q9:
2026-04-26-human-in-the-loop-ai-automated-workflows - Q16:
2026-04-26-ai-agent-control-plane-architecture-enterprise
- Failure taxonomy construction: Review the literature on AI system failures in production and construct a failure taxonomy covering data exposure failures, action failures, attribution failures, coordination failures, and systemic failures.
- Low-code-specific failure analysis: Identify failure modes specific to low-code deployments, including citizen-developer-created automations without adequate testing, fragile integrations, and ungoverned access to enterprise Application Programming Interfaces (APIs), and assess the overlap with AI agent failures.
- Adversarial failure analysis: Review prompt injection research, adversarial input studies, and social engineering scenarios specific to AI agent deployments; assess what governance controls can and cannot prevent adversarially induced failures.
- Control mapping: For each identified failure mode, map preventative controls (design-time, deployment-time) and corrective controls (runtime, post-incident).
- Detection gap analysis: Identify failure modes where current observability and monitoring approaches are insufficient to detect the failure before significant harm occurs.
- Synthesis: Produce a failure mode analysis for enterprise AI and low-code governance, with explicit preventative and corrective control recommendations for each failure mode.
- AI, Algorithmic, and Automation Incidents and Controversies (AIAAIC) Repository — - accessible landing page for a public incident repository; used only as evidence that public incident collections exist because granular entries were not required for downstream claims.
- AIAAIC harms taxonomy project — - public description of AIAAIC's evidence-based harms taxonomy work.
- National Institute of Standards and Technology (NIST) Artificial Intelligence Risk Management Framework (AI RMF) 1.0 publication page — - canonical framework publication page.
- NIST AI RMF Core — - authoritative govern and map outcomes used for control mapping.
- NIST AI RMF Playbook landing page — - current official landing page for the playbook.
- NIST AI RMF Playbook - Govern — - suggested actions for policies, monitoring, incident response, and change management.
- NIST AI RMF Playbook - Manage — - public references for incident databases, resilience resources, and contingency thinking.
- MITRE Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) home page — - checked, but the site rendered only a JavaScript shell in this runtime and was not used for downstream factual claims.
- Gartner enterprise AI risk document — - checked, but remained analyst-access gated in this runtime and was not used for downstream factual claims.
- Anthropic Responsible Scaling Policy — - frontier developer view of layered safeguards, asynchronous monitoring, and rapid jailbreak response.
- Open Worldwide Application Security Project (OWASP) Top 10 for Large Language Model (LLM) Applications project — - checked as the project landing page.
- Open Worldwide Application Security Project (OWASP) LLM01 Prompt Injection — - current detailed prompt injection taxonomy and mitigation guidance.
- Open Worldwide Application Security Project (OWASP) Prompt Injection overview — - prompt injection overview with incident examples and mitigations.
- Universal and Transferable Adversarial Attacks on Aligned Language Models — - evidence that jailbreak-style attacks can transfer across models.
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — - core indirect prompt injection paper demonstrating real-world system exploitation.
- How Microsoft defends against indirect prompt injection attacks — - vendor documentation of observed impacts and mitigations.
- When AI Remembers Too Much: Persistent Behaviors in Agents' Memory — - proof of concept showing prompt injection poisoning of long-term agent memory.
- Air Canada ordered to compensate customer misled by chatbot — - authoritative coverage of a production silent-failure liability case.
- Governing Citizen Development to Address Low-Code Platform Challenges — - interview-based academic evidence on low-code governance debt, shadow Information Technology (IT), and technical debt.
- A framework for implementing robotic process automation projects — - peer-reviewed evidence on robotic process automation implementation failure and socio-technical governance needs.
- Digital.gov - 5 tips for implementing citizen development in your robotic process automation program — - public-sector guidance on governance, role-based access control, and environment separation for citizen development.
- Copilot Studio security and governance — - Microsoft control surface overview for agent governance.
- Copilot Studio data loss prevention — - granular low-code agent restrictions for authentication, knowledge, tools, channels, triggers, and Hypertext Transfer Protocol (HTTP) requests.
- Power Platform data loss prevention overview — - design-time and runtime enforcement behavior, including suspension, quarantine, disabled connections, and enforcement latency.
- Managed Environments overview — - baseline managed-environment control capability.
- Managed governance in Power Platform — - environment groups, delegated admin, inventory, actions, and recommendations.
- Environment routing — - maker isolation into managed developer environments with preconfigured governance settings.
- Pipelines in Power Platform — - sequential promotion, approval, prevalidation, audit, and rollback controls.
- 2025 DevOps Research and Assessment (DORA) report overview — - evidence that AI amplifies existing workflow and control weaknesses.
- DevOps Research and Assessment (DORA) AI Capabilities Model report landing page — - evidence that platforms and platform teams are core scaling mechanisms.
- Business-led low-code agent governance
- AI agent control-plane architecture for enterprise environments
- Decision rights, accountability, and liability for AI systems and low-code applications
- Systems capability debt as the root cause of citizen development
- Deployment pipeline as the enforceable control gate for citizen-developed agents
(Full output from running the research skill - retained verbatim in the completed item. Sections 0-5 are the investigation; section 6 seeds the Findings section below.)
- [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Research question restated: Which enterprise Artificial Intelligence (AI) and low-code failure modes create the most material governance risk in production, and which preventative and corrective controls directly reduce the likelihood or blast radius of each one?
- [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8] Scope confirmed: the investigation covers prompt injection, data leakage, unintended actions, silent failure, fragile low-code change, audit loss, conflicting automation, and feedback-loop failure across enterprise deployments rather than laboratory model behavior.
- [fact; source: https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook; https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] Constraints confirmed: empirical or production-oriented evidence was prioritized, inaccessible analyst material was checked but excluded from downstream claims, and the output must distinguish governance-preventable failure from residual model uncertainty.
- [fact; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-lowcode-decision-rights-accountability-liability.html] Prior work cross-reference: prior completed items already established bounded maker enablement, activity-based accountability, and a central control plane as architectural prerequisites, so this item extends that work by specifying the failure classes those controls must govern.
- [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/] Output format confirmed: knowledge, specifically a failure taxonomy with preventative controls, corrective controls, and detection-gap analysis.
- Root question: Which failures matter most when enterprise AI and low-code systems can read, decide, and act across business systems?
-
A. Data exposure
- A1. How do prompt injection and external-content ingestion create sensitive-information disclosure or data exfiltration?
- A2. Which controls reduce disclosure risk before and after compromise?
-
B. Unintended action
- B1. When do model outputs turn into unauthorized or unsafe actions?
- B2. Which control patterns limit action scope and require approval?
-
C. Silent failure
- C1. How do plausible but incorrect outputs become enterprise governance failures?
- C2. Which controls create verification, containment, and accountability?
-
D. Low-code coordination failure
- D1. Which low-code failure modes stem from citizen-development sprawl, weak promotion discipline, or fragile integrations?
- D2. Which platform controls reduce conflicting automation and release fragility?
-
E. Auditability and attribution
- E1. What causes loss of auditability or poor reconstruction after incidents?
- E2. Which logging and lifecycle controls restore traceability?
-
F. Detection gap
- F1. Which failures remain hard to detect early even with current monitoring?
- F2. Which corrective controls are required because prevention is incomplete?
-
G. Common versus specific
- G1. Which failures are shared across AI and low-code systems?
- G2. Which failures are AI-specific or low-code-specific?
- [fact; source: https://www.nist.gov/itl/ai-risk-management-framework/nist-ai-rmf-playbook] Replacement source used: the current NIST playbook landing page rather than older seed links.
- [fact; source: https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://owasp.org/www-community/attacks/PromptInjection] Prompt injection is a first-order enterprise failure mode because user or external content can alter a Large Language Model (LLM) application's behavior in unintended ways, including disclosing sensitive information, enabling unauthorized access, and manipulating critical decisions.
- [fact; source: https://arxiv.org/abs/2302.12173] The indirect prompt injection paper demonstrates that LLM-integrated applications blur the line between data and instructions, and that remotely injected prompts can cause data theft, arbitrary code-like behavior, and manipulated Application Programming Interface (API) calls in real-world systems.
- [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Microsoft states that indirect prompt injection is one of the most widely used techniques reported to it against AI systems and that security impacts range from exfiltration of user data to unintended actions performed with the user's credentials.
- [fact; source: https://arxiv.org/abs/2307.15043] The adversarial suffix paper shows that jailbreak-style attacks can be generated automatically and transfer across multiple aligned and black-box models, so provider choice alone is not a reliable mitigation.
- [fact; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] The Unit 42 proof of concept shows that indirect prompt injection can poison long-term agent memory, persist across sessions, and later drive silent exfiltration of conversation history through an agent's own tools.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Data exposure failures in enterprise AI are therefore boundary-failure problems: once untrusted content is allowed into the prompt, tool output, or memory path, confidentiality depends less on model alignment and more on privilege scoping, content segregation, and outbound-channel control.
- [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Microsoft documents that indirect prompt injection can trigger unintended actions such as phishing emails sent from trusted accounts or remote command execution through tools available to the agent.
- [fact; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance] Copilot Studio administrators can centrally block unauthenticated chat, public-website knowledge sources, document knowledge sources, Hypertext Transfer Protocol (HTTP) requests, skills, publishing channels, and event triggers, which are exactly the action surfaces through which excessive agency becomes consequential.
- [fact; source: https://genai.owasp.org/llmrisk/llm01-prompt-injection/] The Open Worldwide Application Security Project (OWASP) recommends least privilege, segregated external content, validated output formats, and human approval for high-risk actions because prompt injection severity depends on the degree of agency the system has been granted.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Unintended action is rarely a model-only failure; it is usually a composition failure in which an over-privileged toolset, over-broad publishing surface, or missing approval gate converts model misbehavior into business-side effects.
- [fact; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] The Air Canada chatbot case is a production example of silent failure: the chatbot gave plausible but incorrect bereavement-fare advice, the customer acted on it, and the tribunal held Air Canada responsible for inaccurate information on its website regardless of whether the source was a chatbot or a static page.
- [fact; source: https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] DevOps Research and Assessment (DORA) reports that Artificial Intelligence (AI) amplifies existing weaknesses and that instability remains negatively related to AI adoption unless organizations already have strong automated testing, mature version control, and fast feedback loops.
- [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Silent failure is a governance failure when output is trusted enough to change customer-facing or operational behavior without an independent verification step, because the absence of an explicit error signal does not reduce legal or operational liability.
- [fact; source: https://aisel.aisnet.org/misqe/vol23/iss3/6] The 2024 MIS Quarterly Executive article identifies substandard software quality, shadow Information Technology (IT), and technical debt as core low-code citizen-development risks and argues that low-code governance differs from traditional software governance because it must explicitly involve technical experts and platform-specific guardrails.
- [fact; source: https://link.springer.com/article/10.1007/s10257-022-00553-8] The Springer study states that reliable and scalable robotic process automation still requires expert knowledge and cites estimates that up to 50% of initial robotic process automation implementations fail, which the paper uses to argue for more systematic socio-technical implementation support.
- [fact; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] Digital.gov recommends centralized repositories, code review, role-based access control, and separate development, test, and production environments for citizen development because governance and automation security are major concerns.
- [fact; source: https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/admin/managed-governance] Microsoft's managed-governance features route makers into managed personal developer environments, apply environment-group rules, preconfigure sharing limits and solution checks, and expose inventory, actions, and recommendations for reactive governance.
- [fact; source: https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Power Platform pipelines enforce sequential stage promotion, prevalidate dependencies and environment variables, support approval-based delegated deployments, store deployment artifacts, and enable rollback to previous versions.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing] Conflicting automations and fragile low-code change emerge when local builders can author, test, and publish in shared or weakly governed spaces without enforced promotion discipline, because the platform otherwise accumulates overlapping logic, brittle connectors, and unclear ownership.
- [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/] The National Institute of Standards and Technology (NIST) Govern 1.5 outcome requires ongoing monitoring and periodic review with clearly defined roles and responsibilities, Govern 4.3 calls for practices that enable testing and incident identification, and Govern 6.2 requires contingency processes for failures in third-party data or AI systems deemed high risk.
- [fact; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Microsoft documents maker audit logs in Microsoft Purview, audit logs and alerts in Microsoft Sentinel, and pipeline customization and audit logs saved automatically, which are the vendor-native traces needed for attribution and rollback.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Loss of auditability is not only missing logs; it is the inability to reconstruct which prompt, knowledge source, connector, identity, approval, and deployment artifact produced a consequential output or action, so lifecycle and runtime evidence have to be designed as one chain.
- [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Microsoft says indirect prompt injection is an inherent risk of modern LLM-based systems, that deterministic detection remains an open research challenge, and that defense therefore has to span prevention, detection, and impact mitigation.
- [fact; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Power Platform states that data-policy changes propagate across environments and resources asynchronously, that violating apps, flows, and chatbots may later be suspended or quarantined, and that full enforcement can take up to 24 hours in extreme cases.
- [fact; source: https://www.anthropic.com/responsible-scaling-policy] Anthropic's Responsible Scaling Policy describes a layered safeguard stack with access controls, real-time prompt and completion classifiers, asynchronous monitoring classifiers, post-hoc jailbreak detection, and rapid response procedures, explicitly assuming that no single layer is sufficient.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy] The control implication is that preventative governance alone is insufficient: some failure modes will only be caught after execution has begun, so rate limits, reversible workflows, containment, monitoring, and practiced incident response are not secondary controls but core design requirements.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Shared failure classes across AI and low-code systems are boundary failure, privilege failure, promotion failure, and audit failure, because both types of systems break when action paths cross systems without scoped authority, staged release, or traceable evidence.
- [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] AI-specific failure classes are prompt injection, retrieval contamination, memory poisoning, and stochastic silent error, because they depend on the model's inability to reliably separate data from instructions or certainty from correctness.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing] Low-code-specific failure classes are citizen-development sprawl, shared-environment collision, connector sprawl, and promotion bypass, because they arise from democratized build surfaces and not from probabilistic model behavior itself.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://arxiv.org/abs/2302.12173; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] The strongest AI-specific evidence is consistent on one point: the main enterprise failure is not that a model "hallucinates" in the abstract, but that untrusted input can hijack tool use, memory, or retrieval context and turn the system into an unauthorized actor.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code literature points to the same organizational mechanism from a different angle: decentralized builders can create value only when central engineering and governance functions constrain the build, test, release, and ownership surfaces.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls reduce the chance of bad states by narrowing what can be built, connected, or published, while corrective controls reduce loss after inevitable misses by making the estate observable, reversible, and attributable.
- [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Silent failure matters because enterprise harm often arrives before anyone recognizes a technical incident: customers act on wrong advice, staff act on bad summaries, and throughput gains amplify the reach of each unnoticed error.
- [fact; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://www.anthropic.com/responsible-scaling-policy] No contradiction remains between prevention-heavy and monitoring-heavy sources, because both explicitly describe layered defenses and treat prevention alone as insufficient.
- [fact; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] No contradiction remains between low-code democratization and central governance, because the low-code evidence supports bounded self-service inside centrally administered lifecycle controls rather than uncontrolled maker autonomy.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416] The evidence supports treating silent misinformation as a governance failure even when no malicious actor is present, because the accountability and monitoring obligations remain regardless of attack intent.
- [inference; source: https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Manage] Regulatory and control lens: the National Institute of Standards and Technology (NIST) playbook makes incident response, change management, monitoring cadence, and third-party coverage explicit, which means enterprise governance cannot stop at model safety policy and must cover operational failure management.
- [inference; source: https://cloud.google.com/resources/content/2025-dora-ai-capabilities-model-report; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] Economic and operating-model lens: DORA implies that AI and low-code failures become economically material when they are multiplied by poor workflow systems, so investment in platform teams and internal platforms is itself a risk-reduction strategy.
- [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/managed-governance; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Behavioral lens: many low-code failures are desire-path failures, where makers choose the shortest path available; managed environments, routing, and preconfigured pipelines matter because they make the safer path the easier path.
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] Temporal lens: AI failures can persist or compound over time through memory, retrieval, and asynchronous execution, so post-incident cleanup must include state reset, not only prompt or policy edits.
(This section seeds the Findings below.)
Executive summary:
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] The primary enterprise failure modes in Artificial Intelligence (AI) and low-code deployments are boundary failures, specifically untrusted content crossing into prompts or memory, over-broad action authority, silent wrong outputs, uncontrolled maker publication, and missing end-to-end audit trails, but their frequency and severity are amplified by broader delivery immaturity and systems-capability debt.
- [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Artificial Intelligence (AI)-specific failures are concentrated in prompt injection, retrieval contamination, and persistent memory poisoning, while low-code-specific failures are concentrated in citizen-development sprawl, connector sprawl, fragile promotion, and conflicting automations.
- [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls should therefore narrow who can build, what can connect, where work can run, and how changes can promote, using managed environments, scoped connectors, authenticated channels, least privilege, and sequential release gates.
- [inference; source: https://www.anthropic.com/responsible-scaling-policy; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern] Corrective controls must assume prevention will miss material cases, so runtime monitoring, immutable logs, quarantine, rollback, rate limits, memory reset, and practiced incident response are core governance components rather than optional add-ons.
Key findings:
- [inference; source: https://arxiv.org/abs/2302.12173; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html] High confidence: Prompt injection is a primary enterprise failure mode because untrusted external content can be reinterpreted as instructions, causing data exfiltration, unauthorized tool use, and manipulated downstream decisions in real-world Large Language Model (LLM) applications.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html] High confidence: Unintended actions become materially dangerous only when agents are granted excessive authority, so the decisive governance control is not better prompting alone but least privilege, narrowed tool surfaces, authenticated identities, and human approval for high-impact actions.
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Persistent memory and asynchronous agent workflows create a distinctive Artificial Intelligence (AI) failure class in which poisoned state can survive beyond the triggering session, so governance must include memory scoping, memory reset, and post-incident state cleanup.
- [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] High confidence: Silent but plausible wrong answers are governance failures rather than mere quality issues when organizations let operational or customer decisions depend on model output without verification, because liability and instability arrive before explicit technical alarms do.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] High confidence: Low-code estates fail through shadow Information Technology (IT), technical debt, and conflicting automation when citizen developers can build and publish without centralized repositories, expert review, role clarity, and separate development, test, and production environments, and that pattern is reinforced when systems-capability debt keeps generating workaround demand.
- [inference; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html] High confidence: A consistently supported preventative control pattern is managed isolation plus staged promotion, specifically controlled maker environments, scoped connectors and channels, prevalidated deployments, and approval-based release gates, because those controls jointly remove the easiest unmanaged path from authoring to production.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] High confidence: Loss of auditability is a primary failure mode because enterprises cannot reconstruct accountability without one evidence chain spanning prompts, knowledge sources, identities, approvals, runtime activity, and deployment artifacts.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Detection gaps remain material because prompt injection is not deterministically solvable today and low-code policy enforcement can lag across a large tenant, so circuit breakers, quarantine, rollback, rate limiting, and rehearsed incident response are indispensable corrective controls.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] High confidence: The shared governance pattern across Artificial Intelligence (AI) and low-code systems is that failures concentrate at control-boundary crossings, while recurring severity is amplified by delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence; the main Artificial Intelligence (AI)-specific additions are instruction ambiguity and persistent state and the main low-code-specific additions are maker sprawl and release fragility.
Evidence map:
Assumptions:
- [assumption; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://link.springer.com/article/10.1007/s10257-022-00553-8] Assumption: Robotic process automation governance lessons transfer materially to low-code application and agent programs. Justification: both shift automation authoring toward non-specialists while relying on central platform controls for safe release and support.
- [assumption; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Assumption: Publicly documented prompt injection patterns on general LLM applications are representative of the same risk class in enterprise internal assistants. Justification: the cited sources describe the vulnerability as architectural and content-path dependent, not consumer-product specific.
Analysis:
- [inference; source: https://arxiv.org/abs/2302.12173; https://arxiv.org/abs/2307.15043; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] The most convincing evidence for Artificial Intelligence (AI)-specific failure is the combination of academic attack papers and Microsoft's production guidance, because together they show both feasibility and practical impact. The attack papers establish that adversarial prompt control is real and transferable, while Microsoft establishes that the same failure class appears in enterprise deployments and requires layered mitigations.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code evidence is less about spectacular exploits and more about repeated organizational failure, which is exactly what governance needs. Across academic, public-sector, and enterprise platform guidance, the same preventive pattern repeats: isolate build spaces, centralize repositories, enforce review, and separate promotion from authoring.
- [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] That consistency matters because it means governance is not merely aspirational. The platform surfaces already exist to narrow knowledge sources, channels, triggers, maker environments, and release paths, so the remaining problem is disciplined design and operating-model enforcement.
- [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.anthropic.com/responsible-scaling-policy] The evidence also weighs against a purely preventative mindset. Silent failure, incomplete prompt-injection prevention, and post-hoc safeguard layers all point to the same conclusion: enterprises need reversible workflows, emergency stop paths, incident playbooks, and state cleanup because some failures will only be visible after an output or action has already occurred.
Risks, gaps, uncertainties:
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] Public evidence for memory-poisoning incidents is still thinner than public evidence for prompt injection generally, so the memory-governance recommendations are based on a high-quality proof of concept plus architectural reasoning rather than on a large public incident corpus.
- [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Vendor documentation confirms enforcement latency and runtime quarantine behavior, but it does not quantify how often material harm occurs before enforcement completes, so the size of this exposure window is still uncertain.
Open questions:
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Which public platform patterns are most effective for memory sanitization and trusted-state reconstruction after agent compromise?
- [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] How should enterprises quantify acceptable enforcement lag and rollback time across large low-code estates with mixed maker maturity?
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] What minimum monitoring and approval set is sufficient for medium-risk write-capable agents before the control burden outweighs the productivity gain?
- [fact; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Each substantive section now resolves to sourced facts, explicit inferences, or explicit assumptions, and the taxonomy remains inside the original scope of enterprise deployment and governance failure.
- [fact; source: https://davidamitchell.github.io/Research/research/2026-04-24-business-led-low-code-agent-governance.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-control-plane-architecture-enterprise.html] Prior completed repository work was used only as context and synthesis support, not as a substitute for external evidence.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] The final synthesis is internally consistent: failure modes are grouped by mechanism, controls are mapped by preventative versus corrective purpose, and the remaining uncertainties are explicitly carried into Risks, Gaps, and Uncertainties.
[inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] The primary enterprise failure modes in Artificial Intelligence (AI) and low-code deployments are boundary failures, specifically untrusted content crossing into prompts or memory, over-broad action authority, silent wrong outputs, uncontrolled maker publication, and missing end-to-end audit trails, but their frequency and severity are amplified by broader delivery immaturity and systems-capability debt. [inference; source: https://arxiv.org/abs/2302.12173; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Artificial Intelligence (AI)-specific failures are concentrated in prompt injection, retrieval contamination, and persistent memory poisoning, while low-code-specific failures are concentrated in citizen-development sprawl, connector sprawl, fragile promotion, and conflicting automations. [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] Preventative controls should therefore narrow who can build, what can connect, where work can run, and how changes can promote, using managed environments, scoped connectors, authenticated channels, least privilege, and sequential release gates. [inference; source: https://www.anthropic.com/responsible-scaling-policy; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern] Corrective controls must assume prevention will miss material cases, so runtime monitoring, immutable logs, quarantine, rollback, rate limits, memory reset, and practiced incident response are core governance components rather than optional add-ons.
- [inference; source: https://arxiv.org/abs/2302.12173; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] High confidence: Prompt injection is a primary enterprise failure mode because untrusted external content can be reinterpreted as instructions, causing data exfiltration, unauthorized tool use, and manipulated downstream decisions in real-world Large Language Model (LLM) applications.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] High confidence: Unintended actions become materially dangerous only when agents are granted excessive authority, so the decisive governance control is not better prompting alone but least privilege, narrowed tool surfaces, authenticated identities, and human approval for high-impact actions.
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Medium confidence: Persistent memory and asynchronous agent workflows create a distinctive Artificial Intelligence (AI) failure class in which poisoned state can survive beyond the triggering session, so governance must include memory scoping, memory reset, and post-incident state cleanup.
- [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report] High confidence: Silent but plausible wrong answers are governance failures rather than mere quality issues when organizations let operational or customer decisions depend on model output without verification, because liability and instability arrive before explicit technical alarms do.
- [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html] High confidence: Low-code estates fail through shadow Information Technology (IT), technical debt, and conflicting automation when citizen developers can build and publish without centralized repositories, expert review, role clarity, and separate development, test, and production environments, and that pattern is reinforced when systems-capability debt keeps generating workaround demand.
- [inference; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-deployment-pipeline-citizen-development-governed-gate.html] High confidence: A consistently supported preventative control pattern is managed isolation plus staged promotion, specifically controlled maker environments, scoped connectors and channels, prevalidated deployments, and approval-based release gates, because those controls jointly remove the easiest unmanaged path from authoring to production.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://learn.microsoft.com/en-us/microsoft-copilot-studio/security-and-governance; https://learn.microsoft.com/en-us/power-platform/alm/pipelines; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html] High confidence: Loss of auditability is a primary failure mode because enterprises cannot reconstruct accountability without one evidence chain spanning prompts, knowledge sources, identities, approvals, runtime activity, and deployment artifacts.
- [inference; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://www.anthropic.com/responsible-scaling-policy; https://davidamitchell.github.io/Research/research/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html] Medium confidence: Detection gaps remain material because prompt injection is not deterministically solvable today, access failures can compound downstream effects, and low-code policy enforcement can lag across a large tenant, so circuit breakers, quarantine, rollback, rate limiting, and rehearsed incident response are indispensable corrective controls.
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://aisel.aisnet.org/misqe/vol23/iss3/6; https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report; https://davidamitchell.github.io/Research/research/2026-04-26-systems-capability-debt-citizen-development-empirical-evidence.html; https://davidamitchell.github.io/Research/research/2026-04-26-access-control-amplification-agentic-operations.html; https://davidamitchell.github.io/Research/research/2026-04-26-ai-agent-identity-access-management-enterprise.html; https://davidamitchell.github.io/Research/research/2026-04-26-permission-safe-rag-enterprise-information-architecture.html; https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] High confidence: The shared governance pattern across Artificial Intelligence (AI) and low-code systems is that failures concentrate at control-boundary crossings, while recurring severity is amplified by delivery immaturity, systems-capability debt, identity-model failure, access-control amplification, and information-architecture incoherence; the main Artificial Intelligence (AI)-specific additions are instruction ambiguity and persistent state and the main low-code-specific additions are maker sprawl and release fragility.
- [assumption; source: https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/; https://link.springer.com/article/10.1007/s10257-022-00553-8] Assumption: Robotic process automation governance lessons transfer materially to low-code application and agent programs. Justification: both shift automation authoring toward non-specialists while relying on central platform controls for safe release and support.
- [assumption; source: https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks; https://genai.owasp.org/llmrisk/llm01-prompt-injection/] Assumption: Publicly documented prompt injection patterns on general LLM applications are representative of the same risk class in enterprise internal assistants. Justification: the cited sources describe the vulnerability as architectural and content-path dependent, not consumer-product specific.
[inference; source: https://arxiv.org/abs/2302.12173; https://arxiv.org/abs/2307.15043; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] The most convincing evidence for Artificial Intelligence (AI)-specific failure is the combination of academic attack papers and Microsoft's production guidance, because together they show both feasibility and practical impact. [inference; source: https://aisel.aisnet.org/misqe/vol23/iss3/6; https://link.springer.com/article/10.1007/s10257-022-00553-8; https://digital.gov/2021/08/16/5-tips-for-implementing-citizen-development-in-your-rpa-program/] The low-code evidence is less about spectacular exploits and more about repeated organizational failure, which is exactly what governance needs; across academic, public-sector, and enterprise platform guidance, the same preventive pattern repeats: isolate build spaces, centralize repositories, enforce review, and separate promotion from authoring. [inference; source: https://learn.microsoft.com/en-us/microsoft-copilot-studio/admin-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/admin/default-environment-routing; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] That consistency matters because it means governance is not merely aspirational: the platform surfaces already exist to narrow knowledge sources, channels, triggers, maker environments, and release paths, so the remaining problem is disciplined design and operating-model enforcement. [inference; source: https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416; https://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook/Govern; https://www.anthropic.com/responsible-scaling-policy] The evidence also weighs against a purely preventative mindset, because silent failure, incomplete prompt-injection prevention, and post-hoc safeguard layers all point to the same conclusion: enterprises need reversible workflows, emergency stop paths, incident playbooks, and state cleanup because some failures will only be visible after an output or action has already occurred.
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/] Public evidence for memory-poisoning incidents is still thinner than public evidence for prompt injection generally, so the memory-governance recommendations are based on a high-quality proof of concept plus architectural reasoning rather than on a large public incident corpus.
- [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention] Vendor documentation confirms enforcement latency and runtime quarantine behavior, but it does not quantify how often material harm occurs before enforcement completes, so the size of this exposure window is still uncertain.
- [inference; source: https://unit42.paloaltonetworks.com/indirect-prompt-injection-poisons-ai-longterm-memory/; https://www.anthropic.com/responsible-scaling-policy] Which public platform patterns are most effective for memory sanitization and trusted-state reconstruction after agent compromise?
- [inference; source: https://learn.microsoft.com/en-us/power-platform/admin/wp-data-loss-prevention; https://learn.microsoft.com/en-us/power-platform/alm/pipelines] How should enterprises quantify acceptable enforcement lag and rollback time across large low-code estates with mixed maker maturity?
- [inference; source: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/; https://www.microsoft.com/en-us/msrc/blog/2025/07/how-microsoft-defends-against-indirect-prompt-injection-attacks] What minimum monitoring and approval set is sufficient for medium-risk write-capable agents before the control burden outweighs the productivity gain?
- Type: knowledge
- Description: Failure taxonomy and control mapping for enterprise AI and low-code deployments, covering prompt injection, excessive agency, silent failure, low-code sprawl, auditability loss, and required preventative and corrective controls.
- Links:
Navigation
By Tag
bureaucracy
change-management
coase
constraint-analysis
control-model
decision-rights
delegation
- Q4: Decision rights that should move closer to execution
- Q5: Control model for the best throughput-risk trade-off
delivery-risk
- Operating model synthesis for split-authority delivery systems
- Q6: Leading indicators of instability in split-authority flow systems
demand-segmentation
enterprise
exception-handling
execution
flow
flow-design
flow-metrics
governance
- Operating model synthesis for split-authority delivery systems
- Q1: Dominant flow constraint in split-authority delivery systems
- Q2: Demand segmentation for fast-path vs controlled-path flow
- Q4: Decision rights that should move closer to execution
- Conditions under which internal governance controls minimise coordination costs in regulated enterprises
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
- Barriers to governance reform, leadership failure modes, and reform mechanisms in regulated enterprises
governance-patterns
incentives
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
- Barriers to governance reform, leadership failure modes, and reform mechanisms in regulated enterprises
instability
institutional-economics
- Conditions under which internal governance controls minimise coordination costs in regulated enterprises
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
- Barriers to governance reform, leadership failure modes, and reform mechanisms in regulated enterprises
leading-indicators
operating-model
organisation
- Conditions under which internal governance controls minimise coordination costs in regulated enterprises
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
- Barriers to governance reform, leadership failure modes, and reform mechanisms in regulated enterprises
organisational-design
queue-design
queueing
regulated-enterprise
- Conditions under which internal governance controls minimise coordination costs in regulated enterprises
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
- Barriers to governance reform, leadership failure modes, and reform mechanisms in regulated enterprises
routing
throughput
throughput-risk
transaction-costs
- Conditions under which internal governance controls minimise coordination costs in regulated enterprises
- Failure mechanisms of internal governance controls: bureaucratic inefficiency and informal circumvention in regulated enterprises
triage
- Q2: Demand segmentation for fast-path vs controlled-path flow
- Q3: Routing design that isolates exceptions from routine flow
williamson