Skip to content

Releases: mozership/probe-agent

Release list

v0.2.0

v0.2.0 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 24 Oct 02:18
v0.2.0
c376598
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode. 
uses: github.com/mozership/probe-agent@4d0d6ccee6165624283b22fa0541b02d17e50ac5 # v0.2.0

Tests can now run with models other than Claude. provider picks the provider: anthropic (the default, as before), openai, xai, gemini or sakura (the Sakura AI Engine). See Models.

with:
  provider: openai
  model: gpt-5.4
  • The other providers' models are offered the same browser tools as tools of the agent's own, so the judge, persona tests and recordings work alike for every model. A recording made with one model replays with another.
  • openai is reached through the Responses API, keeping nothing on OpenAI's side; the others through their APIs compatible with Chat Completions.
  • vision: false, the default for sakura, reads pages by their elements and text alone, without screenshots.
  • base_url points a provider at a proxy or another compatible API.
  • A request a provider is busy for is retried after the wait it asks for, unless that is longer than a minute.
  • The reasons and summaries are written in English unless the task is written in another language.

The e2e workflow passed against the real APIs with claude-opus-5-5, gpt-5.4, gpt-6-luna, grok-4.7, gemini-3.5-flash and gpt-oss-120b (Sakura AI Engine). gpt-oss-120b does not call finish, and gemini-3.5-flash can carry on after an error until the test runs out of time; both are still judged correctly.

Changelog

  • c376598 Merge pull request #1 from mozership/openai-responses
  • b1a0602 Point action.yml at v0.1.0
  • b61554d Reach OpenAI through the Responses API
  • a2b8558 Run the tests with the models of OpenAI, xAI, Gemini and the Sakura AI Engine
  • 7e0ec07 Write the reasons in English unless the task is in another language
Assets 7
Loading

v0.1.0

v0.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Oct 14:46
v0.1.0
4ba769d
This commit was signed with the committer’s verified signature.
linyows Tomohisa Oda
GPG key ID: 09C6A6237B0ACCB2
Verified
Learn about vigilant mode. 
uses: github.com/mozership/probe-agent@b1a060242b364be4e7371af96f790090e461f22f # v0.1.0

Changelog

  • 69d2f78 Add the agent action, which tests a web application from a task in words
  • b344ba8 Add the persona mode, which reports what is wrong on the way to a goal
  • 7cc3e4b Bump setup-go, setup-chrome and goreleaser-action to their Node 24 releases
  • 868f798 Mask the secrets in any case
  • 4ba769d Replay the steps of a scenario test that passed, without the model
Loading