Skip to content

fix: guard create_discussion against PLACEHOLDER-only bodies#33595

Merged
pelikhan merged 4 commits into
mainfrom
copilot/fix-safe-output-health-monitor
May 20, 2026
Merged

fix: guard create_discussion against PLACEHOLDER-only bodies#33595
pelikhan merged 4 commits into
mainfrom
copilot/fix-safe-output-health-monitor

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 20, 2026

The safe-output-health workflow submitted a discussion with a body of literally "PLACEHOLDER" (11 chars), consuming 30.4M tokens and producing a useless report. Two gaps enabled this: no minimum body length in the safe-output validation layer, and no explicit prompt-level guard against stub content.

Validation layer: minLength field

Added MinLength int to FieldValidation (Go struct + JS typedef), wired through the GH_AW_VALIDATION_CONFIG JSON pipeline. The validateField string path now checks finalValue.trim().length after sanitization:

// Check minimum length after any sanitization
if (validation.minLength && finalValue.trim().length < validation.minLength) {
  return {
    isValid: false,
    error: `Line ${lineNum}: ${itemType} '${fieldName}' is too short (minimum ${validation.minLength} characters)`,
  };
}

create_discussion.body is the first field to use it, with MinDiscussionBodyLength = 64. This rejects "PLACEHOLDER" (11 chars) and any whitespace-padded equivalent at processing time, before the GitHub API is called.

Prompt layer: safe-output-health.md

Added an Analysis Integrity section to the workflow's Important Guidelines that explicitly:

  • Forbids submitting "PLACEHOLDER" or any unfilled template token verbatim
  • Requires substantive content even when no errors are found (explain the absence, don't stub it)
  • Notes that the 64-char minimum is enforced by the validation system

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cafe.github.com
    • Triggering command: /usr/bin/gh /usr/bin/gh send-telemetry GOMODCACHE go env -json GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link (dns block)
    • Triggering command: /usr/bin/gh /usr/bin/gh send-telemetry 64/bin/go git init�� --bare --initial-branch=master (dns block)
    • Triggering command: /usr/bin/gh /usr/bin/gh send-telemetry 64/bin/go 6022914/b001/importcfg unpa�� TZ6jdpoJ_ -q o.git url GO111MODULE 64/bin/go /opt/hostedtoolc--jq ortc�� GOMODCACHE stmain.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -json GO111MODULE 64/bin/go ache/go/1.25.8/x--jq (dns block)
  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login &#43; &#34;/&#34; &#43; .name 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login &#43; &#34;/&#34; &#43; .name ortcfg.link GOINSECURE GOMOD GOMODCACHE snuMJCPlM-tBNP9L4_/-fJLWRjMGKE55lyzYe-t/2f9A2AZ_cPw5asrtA6O4 env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh pr list --repo github/gh-aw --state all --author app/github-actions --search created:2026-05-11T23:58:00Z..2026-05-12T00:05:00Z --limit 1 --json number --jq .[0].number GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/orgs/owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/owner/actions/secrets --jq .secrets[].name _handler=\(.*\)$GOINSECURE (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name sions/ race 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyBlockedUsersApprovalLabelsCompiledOutput3550229270/001 config /usr/bin/git remote.upstream.gh GO111MODULE 64/bin/go git -C waysRecompiles3639984666/001 config /usr/bin/git remote.upstream./usr/bin/git GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv vaScript461988220/001/test-inlined-imports-enabl-test.timeout=10m0s /tmp/go-build2142323320/b403/vet.cfg (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.github/workflows/approach-validator.md -trimpath ache/node/24.15.0/x64/bin/node -p main -lang=go1.25 ache/node/24.15.--package-lock-only 2831�� ensions/gh-aw -dwarf=false /usr/bin/git go1.25.8 -c=4 -nolocalimports git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv cal/share/gh/extensions/gh-aw config (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv 3936-32221/test-687749285/.github/workflows GO111MODULE ts.test GOINSECURE GOMOD GOMODCACHE ts.test 1423�� -bool 2323320/b495/_testmain.go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/link -errorsas -ifaceassert -nilfunc /opt/hostedtoolcconfig (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv g_.a tPath,Error,Dir,--repo 64/pkg/tool/linuowner/repo GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -uns�� g_.a /tmp/go-build2142323320/b113/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/xconfig (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv ser.test GO111MODULE ortcfg.link GOINSECURE GOMOD GOMODCACHE tMwfXWWKdsaWiu4LGg/KzOcgzpND8C2vCPKB0Ju/7QqomWN27wQ09GEIgmnN -uns�� architecture-guardian.md k/gh-aw/gh-aw/pkg/gitutil/gitutil_test.go g_.a GOSUMDB LeHU_UtIqXruo/47-C 64/bin/go 528390/b001/_pkgconfig (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv /var/run/docker.sock&#34; GH_AW_DOCKER_HOST_PATH_PREFIX_ARGS=&#34;&#34; if [[ &#34;${DOCKER_HOST:-}&#34; =~ ^tcp:// gh x_amd64/vet e/git-remote-https -json GO111MODULE x_amd64/vet e/git-remote-htt--auto om/g�� om/github/gh-aw.--detach upstream /opt/hostedtoolcache/node/24.15.0/x64/bin/node lic_871990930/00/usr/bin/git GO111MODULE 64/bin/go node (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --get remote.origin.url /usr/bin/git -json GO111MODULE 64/pkg/tool/linu/home/REDACTED/.local/share/gh/extensions/gh-aw git remo�� /ref/tags/v9 origin sv -json GO111MODULE 64/pkg/tool/linuxterm-color /usr/bin/git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv /tmp/go-build351GOINSECURE -trimpath 64/bin/go -p github.com/githuenv -lang=go1.25 go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -c=4 -nolocalimports -importcfg /tmp/go-build2142323320/b473/importcfg -embedcfg /tmp/go-build2142323320/b473/embedcfg fhRyG14Fp1pI env axlifetime=\(.*\GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9.0.0
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/go-build351GOINSECURE -trimpath 64/bin/go -p main -lang=go1.25 go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sed -ne s/^session\.save_handler=\(.*\)$GOINSECURE GOPROXY x_amd64/vet GOSUMDB GOWORK .cfg x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv GOMODCACHE (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ai-moderator.md tname) ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolc--jq -ato�� -bool }} {{context.Compiler}} /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -errorsas s,MFiles,HFiles,-C -nilfunc /opt/hostedtoolcconfig (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv sistency_GoAndJavaScript461988220/001/test-inlined-imports-enabl-p /home/REDACTED/work/gh-aw/gh-aw/pkg/jsonutil/json_test.go /usr/bin/git -json GO111MODULE 64/bin/go git -C ons-test1663713490 config ow-without-reaction.lock.yml remote.origin.ur/usr/bin/git GO111MODULE 64/bin/go /usr/bin/git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 go sv -json GO111MODULE x_amd64/vet git conf�� --get resolved$ /usr/bin/git -json GO111MODULE x_amd64/compile git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv me.go o sv s GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-trimpath -ato�� -bool -buildtags /usr/bin/git -errorsas /&#34; &#43; .name -nilfunc git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link -uns�� les.test /tmp/go-build2142323320/b112/vet.cfg 0/x64/bin/node GOSUMDB GOWORK 64/bin/go P2ARhfQO4T-rPmiEconfig (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv ensions/gh-aw GO111MODULE ache/go/1.25.8/x64/bin/go =receive GOMOD GOMODCACHE (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ensions/gh-aw GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuconfig 2323�� -unreachable=false 2323320/b503/_testmain.go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-nilfunc GOSUMDB GOWORK 64/bin/go /opt/hostedtoolcconfig (http block)
  • https://api.github.com/repos/aws-actions/configure-aws-credentials/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 origin sv -json GO111MODULE x_amd64/compile git add test.txt x_amd64/compile /usr/bin/git b/workflows GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 remote.origin.url sv -json GO111MODULE x_amd64/vet git add test.txt x_amd64/vet /usr/bin/git &#34;${DOCKER_HOST:infocmp GO111MODULE x_amd64/compile /usr/bin/git (http block)
  • https://api.github.com/repos/azure/login/git/ref/tags/v2
    • Triggering command: /usr/bin/gh gh api /repos/azure/login/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 l sv -json GO111MODULE 64/bin/go git conf�� cal/share/gh/extensions/gh-aw Test User l -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/docker/login-action/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 x_amd64/vet sv -json GO111MODULE x_amd64/compile node /tmp�� cal/share/gh/extensions/gh-aw x_amd64/compile l ub/workflows GO111MODULE nch,headSha,displayTitle git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv ithub/workflows/archie.md 2323320/b555/_testmain.go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/link h156019910/001&#39; h156019910/001&#39; 64/bin/go /opt/hostedtoolc--jq -o OST_PATH_PREFIX_ARGS=&#34;&#34; if [[ &#34;${DOCKER_HOST:-}&#34; =~ ^tcp:// ]]; then GH_AW_DOCKER_HOST_PATH_PRgh -importcfg 2323320/b563/vet.cfg -s -w -buildmode=exe git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 3936-32221/test-3189431733/.github/workflows faultBranchFromLsRemoteWithRealGitcustom_branch927929689/002/work /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolc-f -ato�� aw/issues/17 -buildtags trepo.git -errorsas -ifaceassert -nilfunc /tmp/go-build214config (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv w.md tname) 0/x64/bin/node GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolc-f -has�� vaScript46198822-f }} {{context.Comowner=github /usr/bin/git -errorsas -ifaceassert -nilfunc git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-05-13 GOMOD GOMODCACHE x_amd64/compile env e-frontmatter.md l_test.go 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linu--auto (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-04-20 GOMOD GOMODCACHE x_amd64/vet env matter-with-env-template-expressions.md GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created &gt;=2026-02-19 vS/xNiwe4cDJCbe3pull GOMODCACHE x_amd64/compile env -json GO111MODULE util.test GOINSECURE GOMOD GOMODCACHE util.test (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env b/workflows GO111MODULE sv GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE ortcfg env 1132252498 GO111MODULE .yml GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env mpiledOutput3739341843/001 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE ache/go/1.25.8/x-test.short=true GOINSECURE GOMOD GOMODCACHE go epOn�� rity4013964041/001 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1234567890
    • Triggering command: /usr/bin/gh gh api repos/{owner}/{repo}/actions/runs/1234567890 --jq {databaseId: .id, number: .run_number, url: .html_url, status: .status, conclusion: .conclusion, workflowName: .name, workflowPath: .path, createdAt: .created_at, startedAt: .run_started_at, updatedAt: .updated_at, event: .event, headBranch: .head_branch, -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env mpiledOutput3739341843/001 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuremote.origin.url env rity4013964041/001 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE ters.test GOINSECURE GOMOD GOMODCACHE ters.test 1423�� -json GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1132252498 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linu-buildmode=exe GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuorigin env (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE ortcfg.link GOINSECURE GOMOD GOMODCACHE fGxALuajTMJLeV66im/u-z1GioK92kjxremote.upstream.url env t2888753155/.github/workflows GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linu-test.short=true GOINSECURE j7/8R3yW1lqQg_24remote GOMODCACHE 64/pkg/tool/linuorigin env g_.a GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1132252498 GO111MODULE sv GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg ache/go/1.25.8/x-nilfunc GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,Swconfig essivefuncparams.test GOINSECURE GOMOD GOMODCACHE essivefuncparamsconfig (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path 300741141/001&#39; 300741141/001&#39; 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOWORK .cfg x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE go env 2096453597/.gith@{u} GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build2142323320/b479/cli.test /tmp/go-build2142323320/b479/cli.test -test.testlogfile=/tmp/go-build2142323320/b479/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true -p github.com/githuenv -lang=go1.25 go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/dev
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/dev --jq [.object.sha, .object.type] | @tsv ensions/gh-aw Test User (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/dev --jq [.object.sha, .object.type] | @tsv ensions/gh-aw go /usr/bin/gh b/workflows GO111MODULE x_amd64/link gh run cal/share/gh/extensions/gh-aw --json l --limit 100 --created /opt/hostedtoolcconfig (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv ensions/gh-aw epo}/actions/rund /usr/bin/git e -test.run=^Test -test.short=true git remo�� ithub-script/git/ref/tags/v9 origin 2323320/b596/scripts.test -json GO111MODULE 64/bin/go th-prefix /tmp/gconfig (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 9883673/001 GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv 56019910/001 56019910/002/wormain x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json herFiles,CFiles,-ifaceassert 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE iAIRm4X/auac8X6jconfig (http block)
  • https://api.github.com/repos/github/gh-aw/issues/17
    • Triggering command: /usr/bin/gh gh api repos/github/gh-aw/issues/17 go env -json GO111MODULE 64/bin/go GOINSECURE estGoFiles,XTestGoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap,TestImports�� GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/google-github-actions/auth/git/ref/tags/v2
    • Triggering command: /usr/bin/gh gh api /repos/google-github-actions/auth/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 Test User sv -json GO111MODULE x_amd64/vet git conf�� user.name Test User /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv cxLVluo7C GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile ortc�� 2323320/b465/_pkg_.a stmain.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/xconfig (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE hqTyUHF/pN7cv-DOtest@example.com env -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/org/repo/pulls/1
    • Triggering command: /usr/bin/gh gh api repos/org/repo/pulls/1 /home/REDACTED/work/gh-aw/gh-aw/pkg/jsonutil/json_test.go 89/001 -json GO111MODULE 64/bin/go GOINSECURE estGoFiles,XTestGoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap,TestImports�� GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/owner/repo/actions/secrets --jq .secrets[].name axlifetime=\(.*\GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-remote-workflow --limit 30 --repo owner/repo go env 3936-32221/test-source-field-varremote.upstream.url GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state 64/pkg/tool/linu--created GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuremote.origin.url env 0/001/test-simple-frontmatter.md g/envutil/envutil_test.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu--auto (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name 00741141/001&#39; 00741141/001&#39; -importcfg /tmp/go-build1519217812/b001/importcfg -pack /tmp/testenv-2332510548/main.go go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch TLToKHbFM GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuconfig ortc�� -json stmain.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI and others added 3 commits May 20, 2026 17:41
…-output-health prompt

- Add MinLength field to FieldValidation struct (Go + JS)
- Set MinLength: 64 on create_discussion.body to reject placeholder-only submissions
- Add minLength check in safe_output_type_validator.cjs validateField
- Add explicit anti-placeholder instructions to safe-output-health.md prompt
- Add tests for minLength validation in Go and JavaScript

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f2fd932b-7874-44f6-aec2-a4af80e7ef21

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…ngth for minLength check

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f2fd932b-7874-44f6-aec2-a4af80e7ef21

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…d placeholders

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f2fd932b-7874-44f6-aec2-a4af80e7ef21

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Safe Output Health Monitor literal placeholder body fix: guard create_discussion against PLACEHOLDER-only bodies May 20, 2026
Copilot AI requested a review from gh-aw-bot May 20, 2026 17:46
@pelikhan pelikhan marked this pull request as ready for review May 20, 2026 19:43
Copilot AI review requested due to automatic review settings May 20, 2026 19:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a minimum-length safeguard to safe-output validation to prevent create_discussion submissions with placeholder-only bodies (e.g., literal "PLACEHOLDER"), and reinforces the workflow prompt to require substantive analysis content.

Changes:

  • Added minLength / MinLength support to the safe-output validation schema and JSON pipeline.
  • Enforced create_discussion.body minimum length (64 chars) and added JS validator logic to check trimmed length after sanitization.
  • Updated the safe-output-health prompt guidelines to explicitly forbid placeholder/template-token submissions and mention the 64-char enforcement.
Show a summary per file
File Description
pkg/workflow/safe_outputs_validation_config.go Adds MinLength to FieldValidation and enforces a 64-char minimum for create_discussion.body.
pkg/workflow/safe_output_validation_config_test.go Adds Go tests to assert the new MinLength config and JSON marshal/unmarshal behavior.
actions/setup/js/safe_output_type_validator.cjs Implements minLength validation for string fields (post-sanitization, trimmed).
actions/setup/js/safe_output_type_validator.test.cjs Adds create_discussion sample config and unit tests covering minLength acceptance/rejection cases.
.github/workflows/safe-output-health.md Updates prompt instructions to prevent placeholder-only discussion bodies and require substantive reporting.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 0

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧪 Test Quality Sentinel completed test quality analysis.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #33595 does not have the 'implementation' label and has only 44 new lines in business logic directories (threshold: 100).

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

PR Code Quality Reviewer completed the code quality review.

@github-actions github-actions Bot mentioned this pull request May 20, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Review ✅

Overall Assessment: The implementation is solid and well-designed. The PR correctly addresses the placeholder submission issue with proper validation logic and comprehensive tests.

What's Done Well:

  • ✅ Length check happens after sanitization (prevents circumventing the check)
  • ✅ Uses .trim() before length check (prevents whitespace padding)
  • ✅ Excellent test coverage including edge cases (whitespace, exact boundary, placeholders)
  • ✅ Clear workflow documentation with explicit examples
  • ✅ Consistent field naming between Go and JavaScript

One Non-Blocking Consideration:

  • The minLength validation is only applied to create_discussion body, while other safe output types (create_issue, add_comment, create_pull_request) have required body fields without minimum length checks. This creates a validation inconsistency across safe output types. See inline comment for details.

This is a minor consistency observation—not a blocker for merging.

Note

🔒 Integrity filter blocked 1 item

The following item was blocked because it doesn't meet the GitHub integrity level.

  • #33595 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

🔎 Code quality review by PR Code Quality Reviewer · ● 2.5M

Fields: map[string]FieldValidation{
"title": {Required: true, Type: "string", Sanitize: true, MaxLength: 128},
"body": {Required: true, Type: "string", Sanitize: true, MaxLength: MaxBodyLength},
"body": {Required: true, Type: "string", Sanitize: true, MaxLength: MaxBodyLength, MinLength: MinDiscussionBodyLength},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider extending minLength to other body fields

The PR adds MinLength: MinDiscussionBodyLength only to create_discussion body (line 248), but many other safe output types have required body fields that could suffer from the same placeholder issue:

  • create_issue (line 53)
  • add_comment (line 71)
  • create_pull_request (line 90)
  • create_agent_session (line 64)
  • comment_memory (line 80)

While the immediate issue is in the discussion health workflow, adding minLength validation only to create_discussion creates an inconsistency. An agent could still submit "PLACEHOLDER" as an issue body or PR body and waste tokens.

Recommendation: Consider either:

  1. Applying the same minLength validation to all required body fields for consistency
  2. Or document why create_discussion needs stricter validation than other types

This is not blocking—the PR solves the stated problem correctly. Just flagging for future consideration.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #33595 does not have the implementation label and has only 44 additions in business logic directories (≤100 threshold).

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnose and /tdd based on the bug fix nature of this PR (preventing PLACEHOLDER-only discussion submissions).

Key Themes

From /diagnose perspective:

  • ✅ Root cause properly identified: dual gaps in validation + prompt guidance
  • ✅ Both layers addressed comprehensively
  • ⚠️ Trim inconsistency: minLength uses .trim().length but maxLength doesn't — could confuse users
  • ⚠️ Error message clarity: Doesn't explain why minimum exists (not actionable for agents)

From /tdd perspective:

  • ✅ Good test coverage with 4 new test cases
  • ✅ Descriptive test names that read as specifications
  • ⚠️ Missing boundary test: Exactly 64 chars not tested (common off-by-one risk)
  • ⚠️ Missing Unicode test: .length behavior with multi-byte chars unclear

Positive Highlights

  • Excellent defense-in-depth: Validation layer + prompt-level guidance is the right approach
  • Go/JS consistency: Both implementations kept perfectly in sync with matching constants
  • No magic numbers in code: Used named constants (MinDiscussionBodyLength) throughout
  • Clean test structure: Good use of async/await and clear assertions

Verdict

Requesting changes on the test coverage gaps (boundary + Unicode cases) and the trim/error message inconsistencies. The core fix is solid but these issues should be addressed before merge to prevent future edge case bugs.

Recommended Follow-ups

  1. Add boundary test for exactly 64 characters
  2. Add Unicode/emoji test to clarify length calculation intent
  3. Document why 64 chars was chosen (inline comment)
  4. Make trim behavior consistent or document the difference
  5. Enhance error message to explain why minimum exists

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 5.1M

it("should reject body shorter than minLength", async () => {
const { validateItem } = await import("./safe_output_type_validator.cjs");

const result = validateItem({ type: "create_discussion", title: "Test Discussion", body: "PLACEHOLDER" }, "create_discussion", 1);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Missing boundary test for exactly 64 characters. Consider adding:

it("should accept body that is exactly minLength", async () => {
  const { validateItem } = await import("./safe_output_type_validator.cjs");
  
  const exactlyMinLength = "a".repeat(64);
  const result = validateItem(
    { type: "create_discussion", title: "Test", body: exactlyMinLength },
    "create_discussion",
    1
  );
  
  expect(result.isValid).toBe(true);
});

Boundary conditions are common sources of off-by-one errors and should always be tested explicitly.

}

return { isValid: true, normalizedValue: value };
// Check minimum length after any sanitization (trim before checking to reject whitespace-padded placeholders)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] The minLength check uses .trim().length but maxLength validation (in sanitizeContent) doesn't trim before checking. This inconsistency means:

  • A 70-char body with leading/trailing spaces passes maxLength (70 chars)
  • But a 63-char body with spaces fails minLength after trim (60 chars)

Consider applying trim uniformly, or document why minLength should reject whitespace-padded content while maxLength allows it.

// If whitespace-padding is intentionally rejected:
if (validation.minLength && value.trim().length < validation.minLength) {
  // Current approach - explicit about rejecting padding
}

// OR apply trim consistently to both checks
const trimmedValue = finalValue.trim();
if (validation.minLength && trimmedValue.length < validation.minLength) { ... }


return { isValid: true, normalizedValue: value };
// Check minimum length after any sanitization (trim before checking to reject whitespace-padded placeholders)
if (validation.minLength && finalValue.trim().length < validation.minLength) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] The error message doesn't explain why this minimum exists, making it less actionable for AI agents. Consider:

return {
  isValid: false,
  error: `Line ${lineNum}: ${itemType} '${fieldName}' is too short (minimum ${validation.minLength} characters). Placeholder-only content like "PLACEHOLDER" or "TODO" is not allowed — provide substantive analysis.`,
};

Error messages are more helpful when they explain the context and provide guidance on what's expected, not just what's wrong.

MaxBodyLength = 65000
MaxGitHubUsernameLength = 39
MaxGitHubTeamSlugLength = 100
MinDiscussionBodyLength = 64 // Minimum body length for create_discussion to prevent placeholder-only submissions
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/zoom-out] Why 64 characters specifically? This magic number should have a comment explaining the rationale:

// MinDiscussionBodyLength prevents placeholder-only submissions.
// 64 chars is enough for a minimal substantive sentence (e.g.,
// "No safe output job failures detected in the last 24 hours.")
// but rejects single-word placeholders like "PLACEHOLDER" or "TODO".
MinDiscussionBodyLength = 64

Future maintainers need to understand why this threshold was chosen so they can adjust it appropriately if requirements change.

const { validateItem } = await import("./safe_output_type_validator.cjs");

const result = validateItem({ type: "create_discussion", title: "Test Discussion", body: "TODO" }, "create_discussion", 1);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Missing test for multi-byte characters (Unicode/emoji). JavaScript's .length counts UTF-16 code units, not characters, which can cause unexpected results:

it("should handle Unicode characters correctly in minLength", async () => {
  const { validateItem } = await import("./safe_output_type_validator.cjs");
  
  // Emoji like 👍 count as 2 in .length but display as 1 character
  const emojiBody = "👍".repeat(32); // 64 UTF-16 units, 32 visual chars
  const result = validateItem(
    { type: "create_discussion", title: "Test", body: emojiBody },
    "create_discussion",
    1
  );
  
  expect(result.isValid).toBe(true); // Should pass if using .length
});

This clarifies whether the 64-char minimum is intentionally based on UTF-16 units or if it should use a different length calculation.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 73/100. Test quality is acceptable — 16.7% of new tests are implementation tests (threshold: 30%).

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

🧪 Test Quality Sentinel completed test quality analysis.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

PR Code Quality Reviewer completed the code quality review.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnose and /tdd based on the bug fix + regression testing approach in this PR.


Summary

This PR addresses a critical production issue where the safe-output-health workflow submitted a discussion with the body "PLACEHOLDER", consuming 30.4M tokens for a useless report. The fix is well-architected with defense-in-depth: validation-layer enforcement (minLength field) + prompt-level guidance (workflow markdown instructions).

Root cause correctly identified: No minimum body length validation for create_discussion.

Solution is sound:

  • ✅ Go FieldValidation struct extended with MinLength field
  • ✅ JavaScript validateField() enforces minimum length with .trim() to prevent whitespace padding
  • ✅ Constant MinDiscussionBodyLength = 64 is well-named and documented
  • ✅ Comprehensive test coverage (4 new JS tests + Go marshaling test)
  • ✅ Prompt-level guidance makes the 64-char requirement explicit to AI agents

Key Themes from /diagnose

  1. Regression prevention: The exact failure mode ("PLACEHOLDER" submission) is now covered by tests. Whitespace-padding exploits are prevented via .trim().
  2. Defense-in-depth appreciated: Both validation layer AND prompt-level guard ensure the issue won't recur.
  3. Minor observation: No logging when minLength validation triggers — would help diagnose future bypass attempts in workflow logs.

Key Themes from /tdd

  1. Test coverage is strong: Edge cases well-covered (short body, single-word placeholder, whitespace padding, exact minimum).
  2. Test quality is high: Descriptive names, clear Arrange/Act/Assert structure, actionable assertions.
  3. Boundary condition gap: Missing test for exactly 64 characters (see inline comment).
  4. Semantic placeholder detection: Current implementation relies solely on length. Longer placeholders like "TODO: Fill this in later" (25 chars) would slip through (see inline comment).

Positive Highlights

  • Excellent root cause diagnosis: Both layers of the problem (validation + prompt) addressed
  • Clean refactor: finalValue variable eliminates repeated sanitization logic
  • Backward compatibility: omitempty on MinLength JSON field ensures old configs still parse
  • Error messages are actionable: "too short (minimum 64 characters)" tells the user exactly what's wrong
  • Constant over magic number: MinDiscussionBodyLength is self-documenting

Verdict

Approved with suggestions. The core fix is solid and well-tested. The inline comments suggest:

  1. Adding a boundary test (exactly 64 chars)
  2. Considering explicit placeholder pattern detection for defense-in-depth
  3. Condensing the verbose prompt guidance for better AI agent compliance

None of these block the merge — they're opportunities to strengthen an already-strong fix.


Learn more about these skills:

  • /diagnose — Disciplined debugging: reproduce → minimise → hypothesise → instrument → fix → regression-test
  • /tdd — Test-driven development: red-green-refactor loop

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 5.5M

expect(result.error).toContain("too short");
});

it("should accept body that meets minLength", async () => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Missing boundary condition test: exactly 64 characters. The current tests check <64 and >64, but not the exact minimum threshold.

Consider adding:

it("should accept body at exactly minLength (64 chars)", async () => {
  const { validateItem } = await import("./safe_output_type_validator.cjs");
  
  const exactlyMinLength = "A".repeat(64); // Exactly 64 characters
  const result = validateItem({ 
    type: "create_discussion", 
    title: "Test", 
    body: exactlyMinLength 
  }, "create_discussion", 1);
  
  expect(result.isValid).toBe(true);
});

Boundary conditions are common sources of off-by-one errors in validation logic.

}

return { isValid: true, normalizedValue: value };
// Check minimum length after any sanitization (trim before checking to reject whitespace-padded placeholders)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] Consider adding explicit detection for common placeholder strings beyond just length checking. The current implementation catches "PLACEHOLDER" via length, but would miss longer placeholders like "TODO: Fill this in later" (25 chars) or "PLACEHOLDER CONTENT HERE" (24 chars).

Suggestion:

// After trim, check for common placeholder patterns
const trimmed = finalValue.trim();
const placeholderPatterns = [
  /^(PLACEHOLDER|TODO|TBD|FIXME|XXX)(\b|$)/i,
  /^\[.*\]$/,  // Template tokens like [DATE], [NUMBER]
];

for (const pattern of placeholderPatterns) {
  if (pattern.test(trimmed)) {
    return {
      isValid: false,
      error: `Line ${lineNum}: ${itemType} '${fieldName}' contains placeholder content that must be replaced with actual data`,
    };
  }
}

This provides defense-in-depth beyond the 64-character minimum, catching semantic placeholders that slip through length checks.


## Important Guidelines

### Analysis Integrity
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnose] The "Analysis Integrity" guidelines are thorough but quite verbose (13 lines). AI agents may skip or skim long instruction blocks, especially when under token pressure.

Consider condensing to the critical points:

### Analysis Integrity

- **Never submit placeholder content**: The body must contain actual analysis, not "PLACEHOLDER", "TODO", or unfilled template tokens like `[DATE]` or `[NUMBER]`.
- **Minimum 64 characters required**: Validation will reject bodies shorter than this. If no runs were found, state that explicitly; if logs failed, explain why.
- **Real findings only**: No single-word stubs, no trivial content.

This retains all the enforcement points (no placeholders, no templates, 64-char minimum, substantive content) while being easier to parse quickly. Brevity increases compliance.

@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 70/100

⚠️ Acceptable, with suggestions

Metric Value
New/modified tests analyzed 6
✅ Design tests (behavioral contracts) 6 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 3 (50%)
Duplicate test clusters 1
Test inflation detected Yes ⚠️
🚨 Coding-guideline violations 0

Test Classification Details

Test File Classification Issues Detected
should reject body shorter than minLength actions/setup/js/safe_output_type_validator.test.cjs:547 ✅ Design None
should reject single-word placeholder body actions/setup/js/safe_output_type_validator.test.cjs:554 ✅ Design Partial duplication with test #1
should accept body that meets minLength actions/setup/js/safe_output_type_validator.test.cjs:561 ✅ Design None
should reject body that is only whitespace below minLength actions/setup/js/safe_output_type_validator.test.cjs:568 ✅ Design None
TestCreateDiscussionBodyMinLength pkg/workflow/safe_output_validation_config_test.go:239 ✅ Design None
TestFieldValidationMinLengthMarshaling pkg/workflow/safe_output_validation_config_test.go:254 ✅ Design None

Test Inflation Analysis

⚠️ Test files grew disproportionately compared to production code:

File Pair Test Lines Production Lines Ratio Status
safe_output_type_validator.test.cjs / safe_output_type_validator.cjs +47 +12 3.92:1 ⚠️ Exceeds 2:1
safe_output_validation_config_test.go / safe_outputs_validation_config.go +41 +3 13.67:1 ⚠️ Exceeds 2:1

Context: The production changes were minimal (adding minLength field support and one validation config entry), while the test files added comprehensive validation coverage. This inflation is defensible given that:

  • The feature is security-critical (prevents placeholder-only submissions)
  • Tests cover multiple edge cases (boundary values, whitespace padding, common placeholders)
  • Tests verify both runtime validation (JavaScript) and config correctness (Go)

However, the 13.67:1 ratio in Go suggests potential over-testing. Consider whether both config and marshaling tests are necessary, or if they could be consolidated.


Flagged Tests — Requires Review

⚠️ Minor Duplication: should reject single-word placeholder body (actions/setup/js/safe_output_type_validator.test.cjs:554)

Classification: Design test

Issue: Partially duplicates should reject body shorter than minLength — both test the same validation path (minLength rejection), just with different input strings ("TODO" vs. "PLACEHOLDER").

What design invariant does this test enforce? The validator rejects bodies below the minimum length threshold.

What would break if deleted? Limited impact — the first test already covers the minLength rejection path. However, testing common placeholder patterns ("TODO", "PLACEHOLDER") has documentation value.

Suggested improvement: Consider consolidating both tests into a single table-driven test that covers multiple placeholder patterns:

it.each([
  { body: "PLACEHOLDER", name: "all-caps placeholder" },
  { body: "TODO", name: "single-word placeholder" },
  { body: "TBD", name: "abbreviation placeholder" },
])("should reject placeholder body: $name", ({ body }) => {
  const result = validateItem({ type: "create_discussion", title: "Test", body }, "create_discussion", 1);
  expect(result.isValid).toBe(false);
  expect(result.error).toContain("too short");
});

This reduces duplication while maintaining coverage of multiple placeholder patterns.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 2 tests — unit (//go:build !integration)
  • 🟨 JavaScript (*.test.cjs): 4 tests (vitest)

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All tests verify behavioral contracts and add genuine protection against regressions. The test inflation ratio is high but defensible given the security-critical nature of the feature.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.


References:

🧪 Test quality analysis by Test Quality Sentinel · ● 5.7M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 70/100. Test quality is acceptable — 0% of new tests are implementation tests (threshold: 30%). All tests verify behavioral contracts and add genuine protection against regressions.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Approved — This PR implements a clean, defense-in-depth solution to prevent placeholder content submissions.

What was reviewed:

  • Workflow prompt improvements (safe-output-health.md)
  • JavaScript validation layer with minLength support
  • Go configuration structs and constants
  • Comprehensive test coverage

Key strengths:

  • Two-layer protection: prompt guidance + validation enforcement
  • Proper sanitization order (sanitize → trim → check length)
  • Excellent test coverage including edge cases (whitespace-only, exact minimum)
  • Clean constant definition (MinDiscussionBodyLength = 64)
  • Consistent implementation across Go and JavaScript layers

No issues found — The implementation is robust, well-tested, and follows repository conventions.

Note

🔒 Integrity filter blocked 1 item

The following item was blocked because it doesn't meet the GitHub integrity level.

  • #33595 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

🔎 Code quality review by PR Code Quality Reviewer · ● 3M

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

@pelikhan pelikhan merged commit a829b68 into main May 20, 2026
148 of 149 checks passed
@pelikhan pelikhan deleted the copilot/fix-safe-output-health-monitor branch May 20, 2026 20:42
Copilot stopped work on behalf of pelikhan due to an error May 20, 2026 20:43
Copilot AI requested a review from pelikhan May 20, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Safe Output Health Monitor emitted literal "PLACEHOLDER" body

4 participants