Repository navigation
uses: github.com/mozership/probe-agent@4d0d6ccee6165624283b22fa0541b02d17e50ac5 # v0.2.0Tests can now run with models other than Claude. provider picks the provider: anthropic (the default, as before), openai, xai, gemini or sakura (the Sakura AI Engine). See Models.
with:
provider: openai
model: gpt-5.4- The other providers' models are offered the same browser tools as tools of the agent's own, so the judge, persona tests and recordings work alike for every model. A recording made with one model replays with another.
openaiis reached through the Responses API, keeping nothing on OpenAI's side; the others through their APIs compatible with Chat Completions.vision: false, the default forsakura, reads pages by their elements and text alone, without screenshots.base_urlpoints a provider at a proxy or another compatible API.- A request a provider is busy for is retried after the wait it asks for, unless that is longer than a minute.
- The reasons and summaries are written in English unless the task is written in another language.
The e2e workflow passed against the real APIs with claude-opus-5-5, gpt-5.4, gpt-6-luna, grok-4.7, gemini-3.5-flash and gpt-oss-120b (Sakura AI Engine). gpt-oss-120b does not call finish, and gemini-3.5-flash can carry on after an error until the test runs out of time; both are still judged correctly.
Changelog
- c376598 Merge pull request #1 from mozership/openai-responses
- b1a0602 Point action.yml at v0.1.0
- b61554d Reach OpenAI through the Responses API
- a2b8558 Run the tests with the models of OpenAI, xAI, Gemini and the Sakura AI Engine
- 7e0ec07 Write the reasons in English unless the task is in another language