Repository navigation
Replies: 1 comment
|
A guardian should call its judge where there is no adapter///guardian. I ran the same test on my Codex work, and the results back it up. I first tried this because I was wondering about #22's per-call performance implications and how they compound in a loop like this (also I thought the loop was recursive). They don't :). A hook only costs about 0.16 s per call on my machine. (the age old quote... 'it works on my machine!'). How fast the loop grows depends on the host and on how fast the judge answers. but why would the speed matter when we're using LLMs which don't exactly spit out answers or make tool calls very quickly? So rework would most likely be far too taxing is performance gain is the motive. Anyways. Setup. Codex CLI 0.142.5 with the Codex adapter from GenAI-Security-Project/agent-control-standard#162 · Add signed Codex PreToolUse…. Before answering a shell call, the Guardian starts a judge, which is a new codex exec under the same ACS hook, and waits for its answer. The judge’s own shell call asks the Guardian again. The model is scripted on localhost. The negotiated decision deadline is 5 s. For safety, the test stops at 30 judge runs, and after that the Guardian denies without asking.
Three things differ from Openclaw:
I guess... only the guardian itself can count? the adapter sees each judge run as a new session with nothing linking it to the step that actually started it? A guardian that does use an agent as a judge would also need its own limit on how many judges it waits on at once. Sooooo That's a guardian requirement as opposed to an adapter one. #162 has the updated codex material |
Uh oh!
There was an error while loading. Please reload this page.
While reviewing #502, @Lrd0036 ran a Guardian that asks an agent on the gateway it governs before it answers. Each question starts a new run, and that run asks the Guardian again. With the rate limit off, he counted 47-49 nested runs in 40 s. Every step failed closed, so nothing ran that the Guardian refused.
The spec doesn't settle this case:
My view: a Guardian that uses an LLM-as-a-judge should call it where there is no adapter and no Guardian. Otherwise the recursion never ends.
Should the spec prohibit it? Or should we count the recursion and stop it?
What do you think?
All reactions