From d422f00f39c35f9a23bd6008a5b77067947869a3 Mon Sep 17 00:00:00 2001 From: jariy17 Date: Mon, 3 Aug 2026 15:10:58 +0000 Subject: [PATCH] fix(docs): document lambda:InvokeFunction for code-based evaluators `run eval` against a code-based evaluator fails with AccessDeniedException for callers using the documented user policy: Access denied when invoking Lambda function: ... User: ... is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:...:function:-eval- `bedrock-agentcore:Evaluate` invokes the evaluator's Lambda under the *caller's* identity, so the caller needs `lambda:InvokeFunction`. A resource-based policy on the function alone does not cover it. Neither iam-policy-user.json nor the PERMISSIONS.md Evaluation table granted or mentioned it, because until the DeepEval/Autoevals tests added in #1828 every eval path exercised was either `Builtin.*` or `llm-as-a-judge` -- neither of which touches Lambda. Scoped to `arn:*:lambda:*:*:function:*-eval-*` (partition wildcard per the multi-partition rules in AGENTS.md) rather than `*`. The same statement was added to the e2e-github-actions CI role, which had no lambda:InvokeFunction on any of its 9 policies -- that is what broke e2e shard 5 on main: https://github.com/aws/agentcore-cli/actions/runs/30663047263/job/91263525976 --- docs/PERMISSIONS.md | 7 +++++++ docs/policies/iam-policy-user.json | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/docs/PERMISSIONS.md b/docs/PERMISSIONS.md index 2d22ed453..231fd13d8 100644 --- a/docs/PERMISSIONS.md +++ b/docs/PERMISSIONS.md @@ -381,8 +381,15 @@ These EC2 and EFS `Describe*` actions do not support resource-level scoping, so | Action | CLI Commands | Purpose | | ------------------------------------------------ | ----------------------------------------- | --------------------------------------------- | | `bedrock-agentcore:Evaluate` | `run evals` | Run on-demand evaluation against agent traces | +| `lambda:InvokeFunction` | `run evals` | Invoke code-based evaluator Lambda functions | | `bedrock-agentcore:UpdateOnlineEvaluationConfig` | `pause online-eval`, `resume online-eval` | Pause or resume online evaluation | +`bedrock-agentcore:Evaluate` invokes a code-based evaluator's Lambda function using the **caller's** identity, so +`lambda:InvokeFunction` is required on the caller when running `run eval` against a `code-based` evaluator (including +DeepEval and Autoevals third-party evaluators). Evaluator functions are named `-eval-`, so the +permission can be scoped to `arn:*:lambda:*:*:function:*-eval-*`. Builtin and `llm-as-a-judge` evaluators do not need +this permission. + ### Batch evaluation and recommendations | Action | CLI Commands | Purpose | diff --git a/docs/policies/iam-policy-user.json b/docs/policies/iam-policy-user.json index f3378bb83..b9100d39e 100644 --- a/docs/policies/iam-policy-user.json +++ b/docs/policies/iam-policy-user.json @@ -56,6 +56,12 @@ ], "Resource": "*" }, + { + "Sid": "CodeBasedEvaluatorInvocation", + "Effect": "Allow", + "Action": "lambda:InvokeFunction", + "Resource": "arn:*:lambda:*:*:function:*-eval-*" + }, { "Sid": "AgentCoreResourceStatus", "Effect": "Allow",