Skip to content

2026 05 01 sustainable ai software development synthesis

Jump to bottom
github-actions[bot] edited this page May 2, 2026 · 1 revision

What principles and governance practices enable sustainable, high-quality software development with Artificial Intelligence (AI) coding agents?

Research Question

What principles and governance practices, spanning harness design, task selection, human oversight, and open-source software (OSS) ecosystem health, enable sustainable, high-quality software development with Artificial Intelligence (AI) coding agents, and what does the current evidence say about where the field sits in its maturation arc?

Scope

In scope:

  • Synthesis of the eight primary items from the Pi talk research cluster:
    1. TerminalBench and minimal toolset benchmarks
    2. Context management transparency
    3. Self-modifying agent architectures
    4. Extension systems
    5. OSS sustainability under AI-generated contributions
    6. Compound error accumulation
    7. Appropriate task selection
    8. Human oversight as a quality gate
  • A unified framework for where these findings converge and diverge
  • The current maturation arc of AI coding agents in 2025 to 2026
  • Actionable principles for development teams, harness designers, and OSS maintainers

Out of scope:

  • Primary research on any individual topic
  • Model training, Artificial General Intelligence (AGI) safety, and alignment outside their direct impact on software-engineering reliability
  • Legal and intellectual-property analysis of AI-generated code

Constraints:

  • Every synthesis claim must stay grounded in the completed primary items, adjacent completed items on the same governance surfaces, or the external sources those items already used
  • Confidence levels must remain explicit and conservative
  • The two planned primary items on self-modification and extension systems remain evidence gaps until they are completed

Context

This item synthesizes a Pi-inspired research cluster that examined benchmark design, context control, task selection, human review, compound error, and OSS intake pressure around coding agents. [fact; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

Across those items, the recurring failure mode is not code generation in isolation but socio-technical system design that hides state, widens task scope, or scales change volume faster than verification and governance can keep up. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

Two planned primary inputs, one on self-modifying architectures and one on extension systems, are still backlog items, so this synthesis can judge current governance and quality controls more confidently than it can judge whether a running harness can safely alter its own tools, prompts, or extensions as a mature performance advantage. [fact; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]

Approach

This is a synthesis item. It does not conduct primary research. It:

  1. Summarises the completed primary findings from the Pi-cluster items that already exist
  2. Checks adjacent completed governance items when the synthesis touches the same control surface
  3. Identifies cross-cutting themes across harness design, scoping, review, throughput, and ecosystem governance
  4. Constructs a unified framework for sustainable use of coding agents
  5. Assesses the maturation arc by asking where the evidence is already strong and where it remains thin
  6. Produces actionable outputs for development teams, harness designers, and OSS maintainers

Sources

Related

Research Skill Output

(Full output from running the research skill - retained verbatim in the completed item. Sections 0-5 are the investigation, and section 6 seeds the Findings section below.)

§0 Initialise

§1 Question Decomposition

  • A. What does the strongest completed evidence say about harness design?
    • A1. What do benchmark and harness-quality items show about minimal versus richer harnesses?
    • A2. What does the context-transparency item show about hidden state, compaction, and explicit control?
  • B. What does the strongest completed evidence say about safe delegation?
    • B1. Which task shapes reliably benefit from coding agents?
    • B2. What role do verifiers and change isolation play?
  • C. What does the strongest completed evidence say about quality gates?
    • C1. What remains the strongest contextual release control?
    • C2. What happens when output volume outruns verification capacity?
  • D. What does the strongest completed evidence say about ecosystem sustainability?
    • D1. How are OSS maintainers responding to AI-generated contribution pressure?
    • D2. Which internal-team analogues follow from those OSS governance patterns?
  • E. What wider governance surfaces qualify the same conclusions?
    • E1. What lower-layer prerequisites must exist before deployment gates can govern write-capable agents safely?
    • E2. How does machine-speed automation change access-control and rate-limiting assumptions?
  • F. What remains materially uncertain?
    • F1. How much evidence exists for self-modifying architectures as a quality lever?
    • F2. How much evidence exists for extension systems as a sustainable governance primitive?
  • G. What overall maturation-arc claim is justified?
    • G1. Which capabilities are already evidence-backed?
    • G2. Which capabilities remain promising but under-evidenced?

§2 Investigation

A. Harness design and benchmark evidence

  • [fact] TerminalBench and the harness-quality benchmark synthesis both show that harness architecture materially changes observed coding-agent performance, while public leaderboards do not support the stronger claim that minimal terminal-native harnesses always beat richer native harnesses. [source: https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-ai-coding-harness-quality-benchmarks.md]
  • [inference] The completed benchmark evidence supports "minimal by default, explicit by exception" more strongly than either "minimal always wins" or "more built-in tooling is always better." [source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]

B. Context legibility and explicit automation

  • [fact] The completed context-management item converges with Anthropic and open-harness documentation on one core point: dynamic context curation is necessary, and Anthropic defines that work as curating and maintaining the tokens available during inference, but hidden prompt changes, silent pruning, and unsignaled tool or provider shifts are meaningful reliability risks. [source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://research.trychroma.com/context-rot; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]
  • [inference] Sustainable harnesses therefore need visible state transitions, inspectable context members, and explicit mutation surfaces, because debugging and trust calibration both depend on seeing what changed and when. [source: https://docs.continue.dev/reference; https://aider.chat/docs/usage/commands.html; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]

C. Task selection and verifier strength

  • [fact] The task-selection evidence says current coding agents add value most reliably on locally bounded, objectively verifiable tasks with low downstream consequence and low reversibility cost, while long-context, multi-file, judgment-heavy work remains materially weaker. [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://arxiv.org/abs/2310.06770; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]
  • [inference] Verifier availability is a more decision-useful gating question than abstract model capability, because the strongest completed positive cases all depend on tests, repro cases, review rubrics, or similarly external success functions. [source: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]

D. Human oversight and release quality

  • [fact] The human-oversight item finds that review expertise, ownership, and maintenance exposure remain the strongest directly evidenced contextual quality controls for high-consequence software changes. [source: https://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf; https://link.springer.com/article/10.1007/s10664-015-9381-9; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md]
  • [inference] The human bottleneck is therefore a feature when it concentrates scarce judgment on scoping, acceptance, architecture, and ambiguous cross-cutting work rather than acting as a uniform line-by-line gate for every low-risk change. [source: https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]

E. Throughput, compound error, and sustainability

  • [fact] The compound-error item links AI-heavy throughput to warning load, duplication, complexity growth, and review burden when change volume scales faster than independent verification. [source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md]
  • [inference] Sustainable velocity is therefore bounded by verification capacity rather than by generation capacity, because faster local patch production becomes a quality loss when review and oracle strength stay flat. [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://arxiv.org/html/2511.04427v2; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md]

F. OSS sustainability and accountability-first intake

  • [fact] The OSS sustainability item shows broad convergence on accountability-first governance, including disclosure, human-understanding requirements, focused changes, trust gates, and selective throttles when reviewer overload becomes acute. [source: https://raw.githubusercontent.com/ghostty-org/ghostty/main/AI_POLICY.md; https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]
  • [inference] Internal engineering teams face the same asymmetry as OSS maintainers, because cheap generation and expensive review make reviewer attention the scarce resource that governance must defend. [source: https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md]

G. Governance preconditions on adjacent control surfaces

  • [fact] Adjacent completed governance items converge on a dependency chain in which coherent policy, representable access boundaries, scoped machine identity, and bounded retrieval or tool access must exist before a deployment gate can validate anything meaningful. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-permission-safe-rag-enterprise-information-architecture.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-access-control-amplification-agentic-operations.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-deployment-pipeline-citizen-development-governed-gate.md]
  • [inference] For write-capable coding agents, governance is not just a final approval step, because deployment into incoherent policy, permission, or information estates amplifies risk at machine speed before a pipeline gate can compensate. [source: https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://csrc.nist.gov/pubs/sp/800/207/final; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-access-control-amplification-agentic-operations.md]

H. Missing primary surfaces and explicit evidence gaps

  • [inference] The planned primary items on self-modifying architectures and extension systems remain backlog-only, so the synthesis has direct completed evidence for transparency, scoping, review, throughput, and OSS intake, but not yet for whether running-harness self-modification itself measurably improves sustainable quality. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]
  • [assumption] Until those two primary items are completed, claims that hot-reload extensibility or self-modification are core enablers of sustainable quality should stay tentative. Justification: the available direct material is still mostly practitioner philosophy and design description, not a completed comparative synthesis with the same evidential weight as the other six items.

§3 Reasoning

  • [inference] The strongest completed evidence points to the same control pattern from multiple directions: bounded task scope, explicit state, strong verifiers, selective human review, and protected reviewer capacity. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]
  • [inference] The apparent tension between minimal harnesses and some higher-performing richer harnesses resolves once the operative principle is reframed from austerity to explicit justified scaffolding. [source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]
  • [inference] The apparent tension between positive local AI studies and negative repository-scale evidence also resolves once task shape, verifier strength, and timescale are separated. [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://arxiv.org/html/2511.04427v2; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md]
  • [inference] The right unit of analysis is therefore the socio-technical delivery system, not the model alone, because sustainability depends on how harness, workflow, review, permissions, and contribution intake interact. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-ai-coding-harness-quality-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

§4 Consistency Check

  • [fact] No completed primary item contradicts the claim that current reliability is strongest under bounded, verifier-backed conditions and weakest where hidden state, long context, or overloaded review surfaces dominate. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]
  • [fact] The main unresolved evidence gap is not contradiction but missing coverage, because the two primaries most directly about self-modifying harness behavior and extension-system design are still incomplete. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]
  • [inference] Overall confidence should therefore remain medium rather than high, because the completed evidence is coherent but not yet fully complete across all planned control surfaces. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]

§5 Depth and Breadth Expansion

  • [inference] Technical lens: sustainable coding-agent systems depend on legible boundaries, because explicit context state, explicit tool surfaces, and explicit verifiers are what make failures debuggable and recoverable. [source: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]
  • [inference] Economic lens: generation cost has fallen faster than review cost, so governance now has to protect scarce verification capacity rather than only maximize raw output. [source: https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]
  • [inference] Behavioral lens: trust should be calibrated around visible state changes and explicit acceptance gates, not around the presence of an explanation layer after the fact. [source: https://doi.org/10.1371/journal.pone.0229132; https://arxiv.org/abs/2006.14779; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]
  • [inference] Governance lens: deployment gates and least-privilege controls matter earlier in the lifecycle than many agent narratives imply, because machine-speed execution can amplify a weak lower layer before a human notices. [source: https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-access-control-amplification-agentic-operations.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-implicit-rate-limiting-controls-agentic-ai-removal.md]
  • [inference] Ecosystem lens: OSS maintainers are a leading indicator for enterprise governance, because they encounter the review-capacity asymmetry and contributor-accountability problem earlier and more visibly than many internal teams. [source: https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

§6 Synthesis

Executive summary

  • [inference] Today, sustainable high-quality software development with AI coding agents depends on bounded automation under explicit human and governance gates, not on broad end-to-end agent autonomy. [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]
  • [inference] Rather than maximizing built-in tooling, the strongest evidence-backed harness pattern keeps the core action surface and context state explicit, then adds helpers only when they justify themselves on real tasks. [source: https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]
  • [inference] Verification capacity, not generation capacity, is the main sustainability bottleneck, since review, acceptance, and maintainer follow-through remain scarce while AI raises output volume. [source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en]
  • [inference] Taken together, the evidence points to an intermediate maturity stage: scoping, transparency, review, and intake governance already show recurring patterns, while self-modifying behavior and extension-first architectures still lack equally strong coverage. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]

Key findings

  1. [inference] The strongest positive evidence for coding-agent use comes from bounded automation, not from end-to-end autonomy, and the winning task profile is locally scoped, objectively verifiable work with low downstream consequence and low reversibility cost. Confidence: medium. Source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md
  2. [inference] TerminalBench leaderboards and context-curation evidence both favor harnesses whose core action and context surfaces stay small, explicit, and inspectable, while opaque tool abundance and hidden context mutation add reliability risk without guaranteed payoff. Confidence: medium. Source: https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md
  3. [inference] More than abstract model capability, verifier strength and change isolation determine whether delegation is reliable, with bounded tasks that have tests or rubrics outperforming long-context, multi-file, and high-coupling work. Confidence: medium. Source: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/; https://arxiv.org/abs/2310.06770; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md
  4. [inference] For high-consequence work, human oversight still acts as the decisive quality gate, since review expertise, ownership, and maintenance responsibility outperform purely throughput-maximizing agent loops on cross-cutting or ambiguous changes. Confidence: medium. Source: https://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf; https://link.springer.com/article/10.1007/s10664-015-9381-9; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md
  5. [inference] Repository-scale studies show what happens when AI-assisted throughput outruns independent verification: warning load, duplication, complexity, and review burden rise, and maintainability debt grows instead of compounding durable productivity. Confidence: medium. Source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md
  6. [inference] Across the retrieved OSS policy set, maintainers are moving toward accountability-first selective openness, using disclosure, small-change expectations, trust gates, and selective throttles to ration scarce review time instead of treating all AI-assisted submissions as equally reviewable. Confidence: medium. Source: https://raw.githubusercontent.com/ghostty-org/ghostty/main/AI_POLICY.md; https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md
  7. [inference] Deployment gates help only after policy, access, identity, and information-architecture prerequisites are machine-checkable and able to block promotion when the required evidence is missing. Confidence: medium. Source: https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-deployment-pipeline-citizen-development-governed-gate.md
  8. [inference] Current evidence describes an intermediate maturity stage: harness and workflow design already show material outcome effects, yet extension-first and self-modifying architectures remain under-evidenced because the two dedicated primary items are still backlog-only. Confidence: medium. Source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md

Evidence map

Claim Source Confidence Notes
[inference] Sustainable use currently means bounded automation rather than end-to-end autonomy. https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md medium Converges across field data, guidance, and prior synthesis
[inference] Small explicit harness surfaces are more sustainable than opaque tool abundance. https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/ medium Competitive minimal baselines, not universal dominance
[inference] Verifier strength and change isolation dominate reliable delegation. https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/; https://arxiv.org/abs/2310.06770; https://www.anthropic.com/engineering/claude-code-best-practices medium Strong agreement on bounded, checked work
[inference] Human oversight remains the decisive gate on high-consequence work. https://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf; https://link.springer.com/article/10.1007/s10664-015-9381-9; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md medium Review evidence stronger than any AI-only replacement evidence
[inference] Verification-capacity limits drive compound quality decay under high throughput. https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md medium Repository-scale evidence, not one definitive randomized trial
[inference] The retrieved OSS evidence converges on accountability-first selective openness. https://raw.githubusercontent.com/ghostty-org/ghostty/main/AI_POLICY.md; https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en medium Retrieved-policy convergence, not a field census
[inference] Governance prerequisites determine whether deployment gates can block promotion effectively. https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-deployment-pipeline-citizen-development-governed-gate.md medium Shared governance-surface synthesis
[inference] The current maturation arc is intermediate because design effects are evident while self-modifying behavior evidence is still missing. https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md medium Evidence-coverage claim, not a universal maturity taxonomy

Assumptions

  • [assumption] Six completed primary items are enough to answer the main governance question even though two planned primary items remain unfinished. Justification: the completed items already cover benchmark design, context control, task shape, review, throughput, and ecosystem governance, which are the dominant control surfaces in the retrieved evidence.
  • [assumption] Adjacent April 2026 governance items are used as qualification on shared control surfaces, not as substitutes for the missing primary items on self-modification and extension systems. Justification: they sharpen permission, pipeline, and systems-capability claims that the Pi-cluster items touch but do not explore in full.
  • [assumption] Pi-related practitioner material is sufficient to mark malleability and extensibility as plausible future levers without treating them as proven maturity markers yet. Justification: the evidence available in this session is descriptive and design-philosophy heavy rather than comparative.

Analysis

  • [inference] The completed evidence repeatedly separates bounded execution from open-ended judgment, which is why the most stable synthesis is about governance of task shape and verification rather than about which frontier model is "best." [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]
  • [inference] Minimal harness evidence should not be read as anti-extensibility doctrine, because richer systems can outperform benchmark-native minimal agents when their extra structure is well engineered and justified by the workload. [source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]
  • [inference] Positive local productivity studies and negative repository-scale quality studies are compatible once timescale changes, because the same acceleration that helps a bounded task can overwhelm review and maintenance capacity at the portfolio level. [source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research]
  • [inference] The main reason the maturation-arc claim stays moderate rather than strong is evidence distribution, not source contradiction, because the pending self-modification and extension-system items leave the part of the thesis focused on running-harness self-modification underdeveloped. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]

Risks, gaps, uncertainties

  • [fact] Two planned primary inputs remain backlog items, so the synthesis has weaker direct evidence on whether self-modifying or extension-first harnesses improve software quality rather than merely customization speed. [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]
  • [fact] Benchmark and field evidence remains much stronger for bounded coding tasks than for multi-session engineering work that spans external services, prolonged review loops, and deployment boundaries. [source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.swebench.com/verified.html; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-ai-coding-harness-quality-benchmarks.md]
  • [inference] Several governance conclusions rely on combining adjacent repository syntheses with official framework or platform material, so they are decision-useful but not equivalent to a single definitive longitudinal study of write-capable coding-agent deployment. [source: https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md]
  • [inference] The current evidence base is strongest on what to bound and gate, and weaker on which positive architecture patterns most reliably unlock safe autonomy beyond today's bounded-envelope use cases. [source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md]

Open questions

  • [inference] Does hot-reload extensibility improve software quality outcomes, or does it mainly improve customization speed and local developer experience? [source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/]
  • [inference] Which measurable verifier-capacity metric best predicts when AI-assisted throughput becomes unsustainable for a team or repository? [source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research]
  • [inference] What benchmark best captures long-horizon engineering work that crosses review, integration, deployment, and rollback boundaries rather than only task completion inside a development environment? [source: https://www.swebench.com/verified.html; https://www.tbench.ai/leaderboard/terminal-bench/2.0]
  • [inference] Which internal-team trust-gate patterns are the best analogue of OSS disclosure, vouch, and selective-throttle policies for high-volume AI-assisted change intake? [source: https://github.com/mitchellh/vouch; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

§7 Recursive Review

  • Acronym audit: complete; first-use expansions checked for Artificial Intelligence (AI), open-source software (OSS), Command Line Interface (CLI), Application Programming Interface (API), and Retrieval-Augmented Generation (RAG)
  • Claim-label audit: complete
  • Findings and §6 Synthesis: aligned
  • Evidence Map source cells: URL-backed
  • Overall confidence: medium; six planned primaries complete, two backlog-only

Findings

Executive Summary

Today, sustainable high-quality software development with Artificial Intelligence (AI) coding agents depends on bounded automation under explicit human and governance gates, not on broad end-to-end agent autonomy. [inference; source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]

Rather than maximizing built-in tooling, the strongest evidence-backed harness pattern keeps the core action surface and context state explicit, then adds helpers only when they justify themselves on real tasks. [inference; source: https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md]

Verification capacity, not generation capacity, is the main sustainability bottleneck, since review, acceptance, and maintainer follow-through remain scarce while AI raises output volume. [inference; source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en]

Taken together, the evidence points to an intermediate maturity stage: scoping, transparency, review, and intake governance already show recurring patterns, while self-modifying behavior and extension-first architectures still lack equally strong coverage. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]

Key Findings

  1. The strongest positive evidence for coding-agent use comes from bounded automation, not from end-to-end autonomy, and the winning task profile is locally scoped, objectively verifiable work with low downstream consequence and low reversibility cost. ([inference]; medium confidence; source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md)
  2. TerminalBench leaderboards and context-curation evidence both favor harnesses whose core action and context surfaces stay small, explicit, and inspectable, while opaque tool abundance and hidden context mutation add reliability risk without guaranteed payoff. ([inference]; medium confidence; source: https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md)
  3. More than abstract model capability, verifier strength and change isolation determine whether delegation is reliable, with bounded tasks that have tests or rubrics outperforming long-context, multi-file, and high-coupling work. ([inference]; medium confidence; source: https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/; https://arxiv.org/abs/2310.06770; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md)
  4. For high-consequence work, human oversight still acts as the decisive quality gate, since review expertise, ownership, and maintenance responsibility outperform purely throughput-maximizing agent loops on cross-cutting or ambiguous changes. ([inference]; medium confidence; source: https://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf; https://link.springer.com/article/10.1007/s10664-015-9381-9; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md)
  5. Repository-scale studies show what happens when AI-assisted throughput outruns independent verification: warning load, duplication, complexity, and review burden rise, and maintainability debt grows instead of compounding durable productivity. ([inference]; medium confidence; source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md)
  6. Across the retrieved OSS policy set, maintainers are moving toward accountability-first selective openness, using disclosure, small-change expectations, trust gates, and selective throttles to ration scarce review time instead of treating all AI-assisted submissions as equally reviewable. ([inference]; medium confidence; source: https://raw.githubusercontent.com/ghostty-org/ghostty/main/AI_POLICY.md; https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md)
  7. Deployment gates help only after policy, access, identity, and information-architecture prerequisites are machine-checkable and able to block promotion when the required evidence is missing. ([inference]; medium confidence; source: https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-deployment-pipeline-citizen-development-governed-gate.md)
  8. Current evidence describes an intermediate maturity stage: harness and workflow design already show material outcome effects, yet extension-first and self-modifying architectures remain under-evidenced because the two dedicated primary items are still backlog-only. ([inference]; medium confidence; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-coding-agent-context-management-transparency.md)

Evidence Map

Claim Source Confidence Notes
[inference] Sustainable use currently means bounded automation rather than end-to-end autonomy. https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md medium Converges across field data, guidance, and prior synthesis
[inference] Small explicit harness surfaces are more sustainable than opaque tool abundance. https://www.tbench.ai/leaderboard/terminal-bench/1.0; https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/ medium Competitive minimal baselines, not universal dominance
[inference] Verifier strength and change isolation dominate reliable delegation. https://github.blog/news-insights/research/research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/; https://arxiv.org/abs/2310.06770; https://www.anthropic.com/engineering/claude-code-best-practices medium Strong agreement on bounded, checked work
[inference] Human oversight remains the decisive gate on high-consequence work. https://www.research.ibm.com/journal/sj/153/ibmsj1503C.pdf; https://link.springer.com/article/10.1007/s10664-015-9381-9; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-human-oversight-ai-software-development.md medium Review evidence stronger than AI-only replacement evidence
[inference] Verification-capacity limits drive compound quality decay under high throughput. https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-compound-error-accumulation-ai-codebases.md medium Repository-scale evidence, not one definitive randomized trial
[inference] The retrieved OSS evidence converges on accountability-first selective openness. https://raw.githubusercontent.com/ghostty-org/ghostty/main/AI_POLICY.md; https://www.eff.org/deeplinks/2026/02/effs-policy-llm-assisted-contributions-our-open-source-projects; https://project.linuxfoundation.org/hubfs/LF%20Research/Open%20Source%20Maintainers%202023%20-%20Report.pdf?hsLang=en medium Retrieved-policy convergence, not a field census
[inference] Governance prerequisites determine whether deployment gates can block promotion effectively. https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-deployment-pipeline-citizen-development-governed-gate.md medium Shared governance-surface synthesis
[inference] The current maturation arc is intermediate because design effects are evident while self-modifying behavior evidence is still missing. https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md medium Evidence-coverage claim, not a universal maturity taxonomy

Assumptions

  • [assumption] Six completed primary items are enough to answer the main governance question even though two planned primary items remain unfinished. Justification: the completed items already cover benchmark design, context control, task shape, review, throughput, and ecosystem governance, which are the dominant control surfaces in the retrieved evidence.
  • [assumption] Adjacent April 2026 governance items are used as qualification on shared control surfaces, not as substitutes for the missing primary items on self-modification and extension systems. Justification: they sharpen permission, pipeline, and systems-capability claims that the Pi-cluster items touch but do not explore in full.
  • [assumption] Pi-related practitioner material is sufficient to mark malleability and extensibility as plausible future levers without treating them as proven maturity markers yet. Justification: the evidence available in this session is descriptive and design-philosophy heavy rather than comparative.

Analysis

The completed evidence repeatedly separates bounded execution from open-ended judgment, which is why the most stable synthesis is about governance of task shape and verification rather than about which frontier model is "best." [inference; source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://www.anthropic.com/engineering/claude-code-best-practices; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-appropriate-task-selection-coding-agents.md]

Minimal harness evidence should not be read as anti-extensibility doctrine, because richer systems can outperform benchmark-native minimal agents when their extra structure is well engineered and justified by the workload. [inference; source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md]

Positive local productivity studies and negative repository-scale quality studies are compatible once timescale changes, because the same acceleration that helps a bounded task can overwhelm review and maintenance capacity at the portfolio level. [inference; source: https://github.blog/news-insights/research/does-github-copilot-improve-code-quality-heres-what-the-data-says/; https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research]

The main reason the maturation-arc claim stays moderate rather than strong is evidence distribution, not source contradiction, because the pending self-modification and extension-system items leave the part of the thesis focused on running-harness self-modification underdeveloped. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]

Risks, Gaps, and Uncertainties

  • Two planned primary inputs remain backlog items, so the synthesis has weaker direct evidence on whether self-modifying or extension-first harnesses improve software quality rather than merely customization speed. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md]
  • Benchmark and field evidence remains much stronger for bounded coding tasks than for multi-session engineering work that spans external services, prolonged review loops, and deployment boundaries. [inference; source: https://www.tbench.ai/leaderboard/terminal-bench/2.0; https://www.swebench.com/verified.html; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-ai-coding-harness-quality-benchmarks.md]
  • Several governance conclusions rely on combining adjacent repository syntheses with official framework or platform material, so they are decision-useful but not equivalent to a single definitive longitudinal study of write-capable coding-agent deployment. [inference; source: https://csrc.nist.gov/pubs/sp/800/207/final; https://aws.amazon.com/blogs/security/four-security-principles-for-agentic-ai-systems/; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-04-26-agentic-ai-foundational-conditions-dependency-ordering.md]
  • The current evidence base is strongest on what to bound and gate, and weaker on which positive architecture patterns most reliably unlock safe autonomy beyond today's bounded-envelope use cases. [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-terminal-bench-minimal-coding-agent-benchmarks.md; https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-self-modifying-agent-architectures.md]

Open Questions

  • Does hot-reload extensibility improve software quality outcomes, or does it mainly improve customization speed and local developer experience? [inference; source: https://github.com/davidamitchell/Research/blob/main/Research/backlog/2026-05-01-extension-systems-ai-coding-agents.md; https://mariozechner.at/posts/2025-11-30-pi-coding-agent/]
  • Which measurable verifier-capacity metric best predicts when AI-assisted throughput becomes unsustainable for a team or repository? [inference; source: https://arxiv.org/html/2511.04427v2; https://www.gitclear.com/ai_assistant_code_quality_2025_research]
  • What benchmark best captures long-horizon engineering work that crosses review, integration, deployment, and rollback boundaries rather than only task completion inside a development environment? [inference; source: https://www.swebench.com/verified.html; https://www.tbench.ai/leaderboard/terminal-bench/2.0]
  • Which internal-team trust-gate patterns are the best analogue of OSS disclosure, vouch, and selective-throttle policies for high-volume AI-assisted change intake? [inference; source: https://github.com/mitchellh/vouch; https://github.com/davidamitchell/Research/blob/main/Research/completed/2026-05-01-oss-sustainability-ai-generated-contributions.md]

Output

Navigation

Home

By Tag

bureaucracy

change-management

coase

constraint-analysis

control-model

decision-rights

delegation

delivery-risk

demand-segmentation

enterprise

exception-handling

execution

flow

flow-design

flow-metrics

governance

governance-patterns

incentives

instability

institutional-economics

leading-indicators

operating-model

organisation

organisational-design

queue-design

queueing

regulated-enterprise

routing

throughput

throughput-risk

transaction-costs

triage

williamson

Clone this wiki locally