-
Notifications
You must be signed in to change notification settings - Fork 22
[failproofai-483] Add hook-sync container for daily agent-CLI drift detection #483
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
10d3a2f
37b443b
9b0e10b
0ee6d7b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| name: build-image | ||
|
|
||
| # Builds and pushes the failproofai hook-sync container to GHCR. | ||
| # | ||
| # The hook-sync container (docker-hook-sync/) runs Claude Code in --effort ultracode mode on a | ||
| # daily schedule (a k8s CronJob managed in our infra, not this repo) to detect drift | ||
| # between every integrated agent CLI and this repo, and opens auto-sync PRs. See | ||
| # docker-hook-sync/Dockerfile for the image and scripts/sync-agent-cli-harnesses-prompt.md for | ||
| # what the agent does. | ||
| # | ||
| # The daily rebuild refreshes the @latest-pinned claude-code + failproofai npm globals. | ||
|
|
||
| on: | ||
| push: | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Warning — the new image and entrypoint get zero automated validation on PRs.
Fix: add a build-only PR trigger so the image is smoke-tested before merge, e.g. on:
pull_request:
paths: ['docker-hook-sync/**', '.github/workflows/build-image.yml']and gate publishing so PR runs build-without-push: push: ${{ github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.push_to_ghcr) }}At minimum, run Note: I could not build the image during this review — no Docker daemon in the sandbox, non-root user, and |
||
| branches: [main] | ||
| paths: | ||
| - 'docker-hook-sync/**' | ||
| - '.github/workflows/build-image.yml' | ||
| schedule: | ||
| - cron: '0 8 * * *' # daily at 08:00 UTC — refresh claude-code@latest + failproofai@latest | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag_suffix: | ||
| description: 'Extra tag to publish alongside :latest and :sha-<short> (e.g. dev, hotfix). Allowed chars: [A-Za-z0-9._-], max 128. Leave empty for none.' | ||
| default: '' | ||
| required: false | ||
| push_to_ghcr: | ||
| description: 'Push the built image to GHCR. Uncheck to build-only (validate the Dockerfile without publishing).' | ||
| type: boolean | ||
| default: true | ||
| required: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3 | ||
|
|
||
| - name: Log in to GHCR | ||
| uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
Comment on lines
+49
to
+54
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why do we need this? we can login with the repo itself?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This step is logging in with the repo itself — |
||
|
|
||
| - name: Compute tags | ||
| id: tags | ||
| env: | ||
| TAG_SUFFIX: ${{ inputs.tag_suffix }} | ||
| run: | | ||
| short_sha="${GITHUB_SHA::7}" | ||
| if [ -n "$TAG_SUFFIX" ]; then | ||
| if ! printf '%s' "$TAG_SUFFIX" | grep -qE '^[A-Za-z0-9_.-]{1,128}$'; then | ||
| echo "::error::tag_suffix '$TAG_SUFFIX' has invalid chars; allowed: [A-Za-z0-9._-], max 128" | ||
| exit 1 | ||
| fi | ||
| fi | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "ghcr.io/failproofai/hook-sync:latest" | ||
| echo "ghcr.io/failproofai/hook-sync:sha-${short_sha}" | ||
| if [ -n "$TAG_SUFFIX" ]; then | ||
| echo "ghcr.io/failproofai/hook-sync:${TAG_SUFFIX}" | ||
| fi | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Build and push | ||
| uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6 | ||
| with: | ||
| context: docker-hook-sync | ||
| file: docker-hook-sync/Dockerfile | ||
| push: ${{ github.event_name != 'workflow_dispatch' || inputs.push_to_ghcr }} | ||
| tags: ${{ steps.tags.outputs.tags }} | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
| provenance: false | ||
|
Comment on lines
+78
to
+87
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. also add latest tag
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,6 +2,9 @@ | |
|
|
||
| ## 0.0.12-beta.0 — 2026-07-09 | ||
|
|
||
| ### Features | ||
| - Add the `hook-sync` container (`docker-hook-sync/` + `.github/workflows/build-image.yml`, published to `ghcr.io/failproofai/hook-sync`): a single-shot daily job that clones failproofai and runs Claude Code in `--effort ultracode` mode to detect hook event-name, tool/payload-schema, and settings-file-shape drift across all seven integrated agent CLIs, then opens one auto-sync PR — the agent commits, pushes, and opens the PR itself, gated by failproofai's own `require-*-before-stop` hooks (dogfooding). Replaces the broken `sync-hook-events.yml` GitHub Action (removed, along with `scripts/sync-hook-events-prompt.md`). | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 Nit — CHANGELOG entry omits the PR number. The repo convention (CLAUDE.md → Changelog) is: "Each entry should be a single line: a short description followed by the PR number (e.g. Cosmetic only: it won't fail the |
||
|
|
||
| ### Fixes | ||
| - Codex hooks: drop the invalid top-level `version` field from `.codex/hooks.json` (Codex CLI v0.142+ rejects it with `unknown field 'version'`, refusing to start any session), and strip any leftover `version` on the next install/uninstall so previously-broken configs self-heal. Also correct the Codex `timeout` unit from `60000` to `60` — Codex reads `timeout` in seconds (its `timeout_sec` field), so the old value meant ~16.7h instead of 60s. Copilot and Cursor legitimately carry `version: 1` in their own schemas and are untouched. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| # Self-contained failproofai hook-sync container. | ||
| # | ||
| # A single-shot daily job (run by a k8s CronJob) that clones failproofai, runs | ||
| # Claude Code in `--effort ultracode` mode to detect drift between every agent-CLI | ||
| # harness we integrate with and this repo, and opens ONE auto-PR with the fix. | ||
| # | ||
| # Bundles: | ||
| # - Node 20 + npm | ||
| # - bun (REQUIRED: the cloned repo's .claude/settings.json fires hooks via | ||
| # `bun $CLAUDE_PROJECT_DIR/bin/failproofai.mjs --hook <Event>`, so the | ||
| # dogfooded require-*-before-stop policies enforce the commit/push/PR) | ||
| # - Claude Code CLI (`claude`) — run headless with --effort ultracode | ||
| # - failproofai@latest on PATH (belt-and-suspenders; the dogfood hooks actually | ||
| # run the clone's SOURCE via bun, not this global) | ||
| # - git + gh + jq + dumb-init + `script` (PTY wrap for stream-json) | ||
| # | ||
| # Build: docker build -t hook-sync:latest docker-hook-sync/ | ||
| # Run: docker run --rm -e CLAUDE_CODE_OAUTH_TOKEN=... -e GH_TOKEN=... hook-sync:latest | ||
| # | ||
| # Every dependency is installed at build time. The daily GitHub Actions rebuild | ||
| # (.github/workflows/build-image.yml) keeps claude-code@latest + failproofai@latest fresh. | ||
| FROM node:20-bookworm-slim | ||
|
|
||
| ENV DEBIAN_FRONTEND=noninteractive \ | ||
| HOME=/home/appuser \ | ||
| BUN_INSTALL=/home/appuser/.bun \ | ||
| FAILPROOFAI_TELEMETRY_DISABLED=1 \ | ||
| PATH="/home/appuser/.bun/bin:/home/appuser/.local/bin:${PATH}" | ||
|
|
||
| # Layer 1: apt deps. dumb-init for PID 1 signal handling; git+gh for the agent's | ||
| # clone/branch/push/PR flow AND the require-*-before-stop hooks; jq for the | ||
| # entrypoint's policy-config edit; bsdextrautils provides `script` (PTY wrap so | ||
| # Node line-flushes --output-format stream-json); curl/gnupg for the gh apt key | ||
| # and the bun installer; unzip is required by the bun installer. | ||
| RUN apt-get update \ | ||
| && apt-get install -y --no-install-recommends \ | ||
| ca-certificates curl gnupg dumb-init jq git bsdextrautils unzip \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Layer 2: GitHub CLI (gh). Used by the agent for `gh pr create/list` and by the | ||
| # require-pr-before-stop hook. | ||
| RUN install -d /etc/apt/keyrings \ | ||
| && curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \ | ||
| | gpg --dearmor -o /etc/apt/keyrings/githubcli-archive-keyring.gpg \ | ||
| && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \ | ||
| > /etc/apt/sources.list.d/github-cli.list \ | ||
| && apt-get update \ | ||
| && apt-get install -y --no-install-recommends gh \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Layer 3: npm globals. Claude Code CLI + failproofai, pinned to latest at build | ||
| # time; the daily workflow rebuild keeps them fresh. Root install so symlinks land | ||
| # in /usr/local/bin (executable by the non-root appuser). | ||
| RUN npm install -g --omit=dev \ | ||
| @anthropic-ai/claude-code@latest \ | ||
| failproofai@latest \ | ||
| && npm cache clean --force | ||
|
|
||
| # Layer 4: non-root user + chowned workspace. | ||
| RUN useradd --create-home --uid 10001 --shell /bin/bash appuser \ | ||
| && mkdir -p /workspace \ | ||
| && chown -R appuser:appuser /workspace /home/appuser | ||
|
|
||
| COPY entrypoint.sh /usr/local/bin/entrypoint.sh | ||
| RUN chmod 0755 /usr/local/bin/entrypoint.sh | ||
|
|
||
| USER appuser | ||
| WORKDIR /home/appuser | ||
|
|
||
| # Layer 5: bun (installs into $BUN_INSTALL=/home/appuser/.bun, on PATH per ENV). | ||
| # The dogfood hook fast-path runs TypeScript from src/ directly — bun executes TS | ||
| # natively, so no build step is needed for the hooks to work. | ||
| RUN curl -fsSL https://bun.sh/install | bash | ||
|
|
||
| ENTRYPOINT ["dumb-init", "--", "/usr/local/bin/entrypoint.sh"] |
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,109 @@ | ||||||||||||||||
| #!/usr/bin/env bash | ||||||||||||||||
| # | ||||||||||||||||
| # failproofai hook-sync entrypoint (runs as PID 1 under dumb-init). | ||||||||||||||||
| # | ||||||||||||||||
| # Lifecycle: | ||||||||||||||||
| # 1. Validate env (CLAUDE_CODE_OAUTH_TOKEN, GH_TOKEN). | ||||||||||||||||
| # 2. gh auth setup-git + git identity. | ||||||||||||||||
| # 3. Clone failproofai --depth=1; cut a fresh auto/sync-cli-harnesses-<UTC> branch. | ||||||||||||||||
| # 4. NEUTRALIZE two policies in the THROWAWAY clone's .failproofai config | ||||||||||||||||
| # (require-ci-green-before-stop, block-read-outside-cwd) via jq, then | ||||||||||||||||
| # `git update-index --skip-worktree` so the edit is invisible to | ||||||||||||||||
| # git status/add and never enters the agent's commit/PR — while failproofai | ||||||||||||||||
| # reads the edited WORKTREE file at runtime and sees them gone. The | ||||||||||||||||
| # require-commit/push/pr-before-stop gates stay ACTIVE (dogfooding). | ||||||||||||||||
| # 5. exec `claude --effort ultracode -p <prompt>` wrapped in a PTY so | ||||||||||||||||
| # --output-format stream-json line-flushes to `kubectl logs -f`. | ||||||||||||||||
| # 6. Exit with claude's return code (all output streams to stdout / the pod log). | ||||||||||||||||
| # | ||||||||||||||||
| # Exit codes: 0 = agent finished (claude's rc) · 64 = missing required env · | ||||||||||||||||
| # 65 = prompt file missing in clone · 66 = unsafe WORKSPACE · else = claude's rc. | ||||||||||||||||
| set -euo pipefail | ||||||||||||||||
|
|
||||||||||||||||
| CLAUDE_CODE_OAUTH_TOKEN="${CLAUDE_CODE_OAUTH_TOKEN:-}" | ||||||||||||||||
| GH_TOKEN="${GH_TOKEN:-}" | ||||||||||||||||
| REPO_URL="${REPO_URL:-https://github.com/failproofai/failproofai.git}" | ||||||||||||||||
| REPO_BRANCH_FROM="${REPO_BRANCH_FROM:-main}" | ||||||||||||||||
| WORKSPACE="${WORKSPACE:-/workspace}" | ||||||||||||||||
| PROMPT_PATH="${PROMPT_PATH:-scripts/sync-agent-cli-harnesses-prompt.md}" | ||||||||||||||||
| CLAUDE_MODEL="${CLAUDE_MODEL:-claude-opus-4-8}" | ||||||||||||||||
|
|
||||||||||||||||
| export HOME="${HOME:-/home/appuser}" | ||||||||||||||||
| export CLAUDE_CODE_OAUTH_TOKEN | ||||||||||||||||
| export GH_TOKEN | ||||||||||||||||
| export GITHUB_TOKEN="${GITHUB_TOKEN:-$GH_TOKEN}" | ||||||||||||||||
| export FAILPROOFAI_TELEMETRY_DISABLED=1 | ||||||||||||||||
| # Keep ultracode background subagents alive in headless -p mode (default is ~10 min). | ||||||||||||||||
| export CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS="${CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS:-1800000}" | ||||||||||||||||
|
|
||||||||||||||||
| log() { printf '[entrypoint %s] %s\n' "$(date -u +%FT%TZ)" "$*"; } | ||||||||||||||||
|
|
||||||||||||||||
| # ---------------- 1. env validation ---------------- | ||||||||||||||||
| if [ -z "$CLAUDE_CODE_OAUTH_TOKEN" ]; then log "ERROR: CLAUDE_CODE_OAUTH_TOKEN not set"; exit 64; fi | ||||||||||||||||
| if [ -z "$GH_TOKEN" ]; then log "ERROR: GH_TOKEN not set"; exit 64; fi | ||||||||||||||||
|
|
||||||||||||||||
| # ---------------- 2. gh + git auth ---------------- | ||||||||||||||||
| log "authenticating gh" | ||||||||||||||||
| gh auth setup-git >/dev/null | ||||||||||||||||
| git config --global user.name "failproofai-hook-sync" | ||||||||||||||||
| git config --global user.email "hook-sync-bot@exosphere.host" | ||||||||||||||||
| git config --global init.defaultBranch main | ||||||||||||||||
|
|
||||||||||||||||
| # ---------------- 3. clone + fresh branch ---------------- | ||||||||||||||||
| case "$WORKSPACE" in | ||||||||||||||||
| ""|"/"|"."|"..") log "ERROR: refusing unsafe WORKSPACE '$WORKSPACE'"; exit 66 ;; | ||||||||||||||||
| esac | ||||||||||||||||
|
Comment on lines
+53
to
+55
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win WORKSPACE safety check is too narrow for the The case statement only blocks exact matches of 🛡️ Proposed fix: add `$HOME` and common system dirs to the blocklist case "$WORKSPACE" in
- ""|"/"|"."|"..") log "ERROR: refusing unsafe WORKSPACE '$WORKSPACE'"; notify fail "hook-sync: unsafe WORKSPACE '$WORKSPACE'"; exit 66 ;;
+ ""|"/"|"."|".."|"$HOME"|"$HOME/"|"/home"|"/home/"|"/etc"|"/usr"|"/var"|"/bin"|"/sbin"|"/lib")
+ log "ERROR: refusing unsafe WORKSPACE '$WORKSPACE'"; notify fail "hook-sync: unsafe WORKSPACE '$WORKSPACE'"; exit 66 ;;
esac📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||
| mkdir -p "$WORKSPACE" | ||||||||||||||||
| if [ -n "$(ls -A "$WORKSPACE" 2>/dev/null)" ]; then | ||||||||||||||||
| find "$WORKSPACE" -mindepth 1 -maxdepth 1 -exec rm -rf {} + | ||||||||||||||||
| fi | ||||||||||||||||
| log "cloning $REPO_URL@$REPO_BRANCH_FROM -> $WORKSPACE" | ||||||||||||||||
| git clone --depth=1 --branch "$REPO_BRANCH_FROM" "$REPO_URL" "$WORKSPACE" | ||||||||||||||||
| cd "$WORKSPACE" | ||||||||||||||||
| # Fetch the base ref (shallow) so require-*-before-stop can diff against origin/<base>. | ||||||||||||||||
| git fetch --depth=1 origin "$REPO_BRANCH_FROM" >/dev/null 2>&1 || true | ||||||||||||||||
| BRANCH="auto/sync-cli-harnesses-$(date -u +%Y%m%dT%H%M%SZ)" | ||||||||||||||||
| git checkout -b "$BRANCH" | ||||||||||||||||
| log "working on branch $BRANCH" | ||||||||||||||||
|
|
||||||||||||||||
| # ---------------- 4. policy-config neutralization ---------------- | ||||||||||||||||
| # enabledPolicies is a UNION across scopes with no disable field, so a policy must | ||||||||||||||||
| # be removed from the file that lists it. --skip-worktree then hides that edit from | ||||||||||||||||
| # git status/add (so it never lands in the PR) while failproofai still reads the | ||||||||||||||||
| # edited worktree file at runtime. Keeps require-commit/push/pr-before-stop active. | ||||||||||||||||
| CFG=".failproofai/policies-config.json" | ||||||||||||||||
| if [ -f "$CFG" ]; then | ||||||||||||||||
| cfg_tmp="$(mktemp)" | ||||||||||||||||
| jq '.enabledPolicies |= map(select(. != "require-ci-green-before-stop" and . != "block-read-outside-cwd"))' "$CFG" > "$cfg_tmp" && mv "$cfg_tmp" "$CFG" | ||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 Info — the policy-neutralization step fails open if Under So if Fix: check if ! jq '.enabledPolicies |= map(select(. != "require-ci-green-before-stop" and . != "block-read-outside-cwd"))' "$CFG" > "$cfg_tmp"; then
log "ERROR: failed to neutralize policies in $CFG"; exit 67
fi
mv "$cfg_tmp" "$CFG" |
||||||||||||||||
| git update-index --skip-worktree "$CFG" | ||||||||||||||||
| log "neutralized require-ci-green-before-stop + block-read-outside-cwd for this run (skip-worktree; not committed)" | ||||||||||||||||
| else | ||||||||||||||||
| log "WARN: $CFG not found; running with the repo's default policy set" | ||||||||||||||||
| fi | ||||||||||||||||
|
|
||||||||||||||||
| # ---------------- 5. invoke claude (ULTRACODE, headless, PTY-wrapped) ---------------- | ||||||||||||||||
| PROMPT_FILE="$WORKSPACE/$PROMPT_PATH" | ||||||||||||||||
| if [ ! -f "$PROMPT_FILE" ]; then log "ERROR: prompt missing at $PROMPT_FILE"; exit 65; fi | ||||||||||||||||
|
|
||||||||||||||||
| # Node block-buffers stdout when it's a pipe; `script -qefc ... /dev/null` gives | ||||||||||||||||
| # claude a PTY so --output-format stream-json line-flushes to the pod logs. `-e` | ||||||||||||||||
| # ties script's exit code to claude's. | ||||||||||||||||
| RUN_SH="$(mktemp /tmp/run-claude.XXXXXX.sh)" | ||||||||||||||||
| cat >"$RUN_SH" <<EOF | ||||||||||||||||
| #!/usr/bin/env bash | ||||||||||||||||
| exec claude --effort ultracode --model ${CLAUDE_MODEL} \\ | ||||||||||||||||
| --verbose --output-format stream-json \\ | ||||||||||||||||
| --dangerously-skip-permissions \\ | ||||||||||||||||
| -p "\$(cat "$PROMPT_FILE")" 2>&1 | ||||||||||||||||
| EOF | ||||||||||||||||
| chmod +x "$RUN_SH" | ||||||||||||||||
|
|
||||||||||||||||
| log "invoking claude --effort ultracode --model ${CLAUDE_MODEL} (prompt: $PROMPT_PATH)" | ||||||||||||||||
| set +e | ||||||||||||||||
| script -qefc "$RUN_SH" /dev/null | ||||||||||||||||
| rc=$? | ||||||||||||||||
| set -e | ||||||||||||||||
| rm -f "$RUN_SH" | ||||||||||||||||
| log "claude exited $rc" | ||||||||||||||||
|
|
||||||||||||||||
| exit "$rc" | ||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: FailproofAI/failproofai
Length of output: 980
Add the PR number to this changelog entry
CHANGELOG.md:3-7is missing the required#NNNreference; the version already matchespackage.json.🤖 Prompt for AI Agents
Source: Coding guidelines