Skip to content

Add ADR for allowed and disallowed tools for agents#293

Merged
maruiz93 merged 4 commits into
fullsend-ai:mainfrom
maruiz93:allowed-disallowed-tools
May 5, 2026
Merged

Add ADR for allowed and disallowed tools for agents#293
maruiz93 merged 4 commits into
fullsend-ai:mainfrom
maruiz93:allowed-disallowed-tools

Conversation

@maruiz93

@maruiz93 maruiz93 commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds an ADR deciding how to restrict what tools agents can execute
  • Empirical findings: tools and disallowedTools frontmatter have no effect in --agent sessions (only work for subagents); permissions.deny is enforced even with --dangerously-skip-permissions; bypassPermissions is a partial bypass
  • Evaluates 5 options across a spectrum from least to most restrictive; decides on --dangerously-skip-permissions + permissions.deny for steering, with sandbox as sole security layer
  • Annotates agent-infrastructure open question with link to this ADR
  • Supporting experiment: Add tool scoping experiment for agent tool restrictions experiments#3

Test plan

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Apr 21, 2026

Copy link
Copy Markdown

Site preview

Preview: https://1dc74fd6-site.fullsend-ai.workers.dev

Commit: 90de6d19f5ca04a904f5dd59939be81de3ec9e08

@ben-alkov

Copy link
Copy Markdown
Member

@maruiz93; Do you know about claude --permission-mode auto?

- Parameterized restrictions (e.g. `Bash(git push *)`) become enforced by
Claude Code's permission system instead of prompt-only, closing the
prompt-injection bypass path.
- The `permissions.allow` list must be maintained — if a legitimate tool pattern

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not actionable; merely a comment.

For this to work seriously as a security mechanism, the allowlist must be quite fine-grained, on the order of, in the case of my personal use, over 100 distinct rules...

It's not a small job to maintain this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I was thinking that we might want for each agent a small amount of tools to be used

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's ideal, but in practice it's difficult.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this is based on my understanding (which could be incorrect), and is based on my personal interactive use, but I think the reasoning still applies...

On the deny side, the following are necessary to prevent Claude Code from doing bulk adds and not thinking about how to split things up into atomic commits (among other things):

"Bash(git add --all:*)",
"Bash(git add --force:*)",
"Bash(git add -A:*)",
"Bash(git add -f:*)"

tl;dr - blocking "Bash(git add :*)" is insufficient (again, IIUC)

On the allow side, it's arguably more difficult, because you don't want to "open the door" too far, so you end up with even more granularity, e.g.

"Bash(gh api graphql --raw-field:*)",
"Bash(gh issue view:*)",
"Bash(gh pr checkout:*)",
"Bash(gh pr checks:*)",
"Bash(gh pr diff:*)",
"Bash(gh pr view:*)",
"Bash(gh repo view:*)",
"Bash(git apply:*)",
"Bash(git checkout:*)",
"Bash(git cherry-pick:*)",
"Bash(git commit:*)",
"Bash(git fetch:*)",
"Bash(git log:*)",
"Bash(git rebase:*)",
"Bash(git rev-parse:*)",
"Bash(git show:*)",
"Bash(git stash:*)",
"Bash(git status:*)"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are right, I did more experimentation and came to the conclussion that the best thing would be relying on the sandbox for security, and using the permissions.deny eventually on the agents for steering, as a way to redirect the agent when doing something we don't want it to do as soon as possible, saving context and tokens. PR is ready for review again. This is the related experiment I did: fullsend-ai/experiments#3

@maruiz93

Copy link
Copy Markdown
Contributor Author

@ben-alkov I think I've made some assumptions that I'm not sure about and I don't have enough data in the docs, I'll run some experiments along this PR. Moving to draft

@maruiz93 maruiz93 marked this pull request as draft April 21, 2026 14:46
@maruiz93 maruiz93 force-pushed the allowed-disallowed-tools branch from b320c1e to 0b3612f Compare April 23, 2026 02:19
@maruiz93 maruiz93 changed the title ADR 0021: Allowed and disallowed tools for agents Add ADR for allowed and disallowed tools for agents Apr 23, 2026
@maruiz93 maruiz93 marked this pull request as ready for review April 23, 2026 09:50
@maruiz93 maruiz93 force-pushed the allowed-disallowed-tools branch from 0b3612f to 03a3798 Compare April 23, 2026 10:03
@maruiz93 maruiz93 force-pushed the allowed-disallowed-tools branch from 03a3798 to 01aab48 Compare April 23, 2026 10:14
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@maruiz93

Copy link
Copy Markdown
Contributor Author

Can I get some reviews @ben-alkov ?

@rh-hemartin rh-hemartin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so sandbox as security and the other stuff as steering, that sounds good. In practice we may never use the steering, as steering also comes from skills and that may be enough.

@maruiz93

Copy link
Copy Markdown
Contributor Author

as steering also comes from skills and that may be enough.

hmm, right the skills frontmatter include allowed-tools but they don't allow disallowed-tools or similar.

I think everything should work together and eliminating from the agent the posibility to run a specific thing, even if this doesn't forbid to write its own tool, it's a greate way of steering.

Also, note that the fullsend agents that are provided by the fullsend agent are compatible with skills in the target repo, but the agent permissions.allow/permissions.deny has more priority than the skills allowed-tools

To sum up, I believe that fullsend agents should include permissions.deny to properly guide the agent of how we want them to work

@ralphbean ralphbean left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@rh-hemartin rh-hemartin added this pull request to the merge queue May 4, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 4, 2026
@maruiz93 maruiz93 added this pull request to the merge queue May 4, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 4, 2026
0024 was already taken on main by the harness-definitions ADR.
Renumber to 0026 (next available) and update all cross-references.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@rh-hemartin rh-hemartin added this pull request to the merge queue May 5, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 5, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented May 5, 2026

Copy link
Copy Markdown

Review: #293

Head SHA: f85900a
Timestamp: 2026-05-05T00:00:00Z
Outcome: approve

Summary

Clean documentation PR that adds ADR 0027 deciding on --dangerously-skip-permissions + permissions.deny as the tool restriction strategy for agents, with sandbox as the sole security enforcement layer. The ADR is well-structured with five clearly differentiated options, empirical evidence from linked experiments, and a decision rationale consistent with the repo's existing security model (ADR 0017, ADR 0020). Cross-reference annotations in architecture.md, agent-infrastructure.md, and security-threat-model.md correctly point to the new ADR and preserve the "remains open" framing for broader questions. No findings across any review dimension.

Findings

No findings.

Footer

Outcome: approve
This review applies to SHA f85900af471f6df7dc74e6780b9b1084b8e1311d. Any push to the PR head clears this review and requires a new evaluation.

Previous run

Review: #293

Head SHA: f08648d
Timestamp: 2026-05-05T00:00:00Z
Outcome: request-changes

Summary

The ADR is well-reasoned and the experimental evidence is valuable, but the PR has two issues that need resolution before merging. First, there is an ADR number collision: ADR 0026 already exists on main (0026-stage-based-dispatch-for-agent-workflow-decoupling.md), so this ADR and all cross-references must be renumbered to 0027. Second, the cross-reference added to security-threat-model.md frames permissions.deny as a blast-radius mitigation ("Tool access limited by permissions.deny hard-blocks"), but the ADR itself explicitly states that permissions.deny is a steering tool, not a security control — placing it in the security threat model's blast-radius discussion contradicts the ADR's own framing and risks misleading readers into treating it as a security boundary.

Findings

Critical

  • [Correctness] docs/ADRs/0026-allowed-and-disallowed-tools-for-agents.md — ADR number 0026 already exists on main as 0026-stage-based-dispatch-for-agent-workflow-decoupling.md. This PR introduces a second ADR 0026, creating a numbering collision. The new ADR must be renumbered to 0027, and all cross-references in architecture.md, agent-infrastructure.md, and security-threat-model.md must be updated accordingly.
    Remediation: Rename the file to 0027-allowed-and-disallowed-tools-for-agents.md, update the title and frontmatter to use 0027, and update all three cross-reference sites.

High

  • [Intent alignment] docs/problems/security-threat-model.md:98 — The added text "Tool access limited by permissions.deny hard-blocks" frames permissions.deny as a security mitigation for blast radius. However, the ADR's Decision section explicitly states: "Security: the sandbox is the sole enforcement layer. Tool-level restrictions are always bypassable" and "permissions.deny serves as a steering tool... Not a security control." Listing it in the threat model's blast-radius answer implies it provides security value, contradicting the ADR.
    Remediation: Either remove the permissions.deny reference from the security threat model's blast-radius bullet, or reframe it to match the ADR's characterization — e.g., "Tool access is steered by permissions.deny for focus, but security enforcement relies solely on the sandbox."

Info

  • [Style/conventions] docs/ADRs/0026-allowed-and-disallowed-tools-for-agents.md — The ADR is clearly written with a good options analysis. The table-based comparison of mechanisms is effective and easy to scan. The distinction between steering and security is well-articulated.

Footer

Outcome: request-changes
This review applies to SHA f08648d2d16d38285d49bfe13daac51579dff401. Any push to the PR head clears this review and requires a new evaluation.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

maruiz93 and others added 2 commits May 5, 2026 15:20
0026 was merged to upstream main (stage-based-dispatch ADR) since the
last renumber. Bump to 0027 and update all cross-references.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

@maruiz93 maruiz93 added this pull request to the merge queue May 5, 2026
Merged via the queue into fullsend-ai:main with commit 15871cd May 5, 2026
15 checks passed
@maruiz93 maruiz93 deleted the allowed-disallowed-tools branch May 5, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants