feat(hooks): add 'type: model' hook and integrate pre_tool_use into approval flow#2546
Merged
dgageot merged 1 commit intodocker:mainfrom Apr 28, 2026
Conversation
trungutt
previously approved these changes
Apr 28, 2026
bbdfe37 to
5f3db5b
Compare
rumpl
previously approved these changes
Apr 28, 2026
5f3db5b to
a445589
Compare
rumpl
previously approved these changes
Apr 28, 2026
…pproval flow
Add a first-class `type: model` hook handler so users can wire LLM-as-X hooks (judge, summarizer, classifier, ...) entirely from YAML, with no Go code.
Highlights:
- New `HookTypeModel` factory in pkg/hooks with a Go text/template prompt, an injectable ModelClient, and a pluggable ResponseShape registry. Tests cover validation, templating, structured-output threading, and lenient JSON parsing.
- Well-known `pre_tool_use_decision` shape ships with the package: the model is asked for {decision, reason} (strict JSON schema for providers that honor it; tolerant parser otherwise), and the result becomes a permission_decision verdict.
- Runtime model_hook.go wires a real provider-backed ModelClient from a model spec (provider/model).
- Move pre_tool_use dispatch from runTool into executeWithApproval. Allow now auto-approves (skips user prompt), Ask escalates to it, Deny blocks BEFORE prompting (no more wasted clicks). New Result.Decision / DecisionReason carries the most-restrictive verdict (Deny > Ask > Allow). Deterministic permissions: rules still win, so the hook only sees the ambiguous middle.
- Schema & docs: new model/prompt/schema fields on HookDefinition with conditional 'if type == model' validation; LLM-as-a-judge doc section rewritten around type: model; examples/llm_judge.yaml migrated and slimmed down.
- Removed: pkg/hooks/builtins/llm_judge.go (~370 LOC) and its tests, superseded by the generic framework. The judge use case is now ~30 LOC of YAML.
Behavior change to flag in changelog: pre-existing pre_tool_use shell-script hooks fire BEFORE the user prompt instead of after. This is strictly an improvement (Deny no longer wastes a user click) but is observable for any hook that relied on running post-approval.
Assisted-By: docker-agent
a445589 to
3a6990e
Compare
trungutt
approved these changes
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a first-class
type: modelhook handler so users can wire LLM-as-X hooks (judge, summarizer, classifier, …) entirely from YAML — no Go code, no shell glue, no jq+curl. Combined with the well-knownpre_tool_use_decisionschema and a small approval-flow change, it delivers a working LLM-as-a-judge that actually auto-approves tool calls.That's it — the framework owns provider construction, streaming, structured output, JSON parsing, error fallback, and verdict shaping.
What's in the PR
1. New
type: modelhook frameworkHookTypeModelfactory (pkg/hooks/model_handler.go) with a pluggableBackend/ModelClientseam, Gotext/templateprompt rendering exposing the hookInput, plustoJSONandtruncatetemplate helpers.ResponseShaperegistry: a shape interprets the model's reply as a hookOutput, with an optional pairedlatest.StructuredOutputschema. The default shape passes the reply through asadditional_context(turn_start summarizers, etc.).HookDefinition:model,prompt,schema. Conditional validation: whentype == "model", bothmodelandpromptare required (enforced in the Go validator andagent-schema.json).pkg/runtime/model_hook.goprovides a real provider-backedModelClientthat builds aprovider.Providerper call from a model spec, threadsWithStructuredOutputwhen a schema is supplied, and streams the reply.2. Well-known
pre_tool_use_decisionshapeShips with the package (auto-registered):
{decision: "allow"|"ask"|"deny", reason: string}) for providers that honor structured output (OpenAI, …).HookSpecificOutput.PermissionDecisionverdict the runtime's approval flow understands.3. Runtime: pre_tool_use becomes an active approval gate
The pre_tool_use hook used to fire inside
runTool, after the user had already been asked. So a Deny wasted a click and an Allow had no effect on the prompt. This PR moves it intoexecuteWithApproval:Result.Decision/Result.DecisionReasonfields, populated by a most-restrictive aggregator (Deny > Ask > Allow).ApprovalSourceconstants —pre_tool_use_hook_allowandpre_tool_use_hook_deny— so the existingon_tool_approval_decisionobserver event fires for hook verdicts too. Telemetry / audit gets full visibility for free.4. Tests (~30 new sub-tests)
pkg/hooks/aggregate_test.goDecisionempty.pkg/hooks/model_handler_test.gotruncatehelper, nil-client misuse.pkg/runtime/pre_tool_use_approval_test.gopkg/runtime/hooks_wiring_test.gomodelfactory is registered byNewLocalRuntime.5. Docs / examples
examples/llm_judge.yaml— full working configuration showing the 3-layer pattern (deterministic permissions → LLM judge → user prompt).docs/configuration/hooks/index.md— "LLM as a Judge" section rewritten aroundtype: model, plus a security-considerations subsection.agent-schema.json— new fields withif/thenconditional validation fortype: model.Behavior change to flag
Pre-existing
pre_tool_useshell-script hooks that returnedDenyused to fire after the user had been prompted. They now fire before. This is strictly an improvement (no more wasted clicks on hooks that were going to deny) but is observable for any audit log keyed on hook-vs-prompt timing. Allow-style verdicts gain new effect: they previously did nothing; they now skip the user prompt.Validation
golangci-lint run ./...→ 0 issuesgo test ./...→ all packages pass, no FAILpkg/config) validates the new YAML shape.go build ./...clean.Stats
18 files changed, 1519 insertions(+), 37 deletions(-)Built incrementally with
docker agentfrom a blank slate ("how do I implement an LLM judge?") through to a shippedtype: modelframework with reviewer feedback applied. See commit message for details.