0.52.0

Optimizing the CLI for Coding Agents

We have been exploring various ways to make Arbigent more accessible to coding agents, including prototyping a Model Context Protocol (MCP).

However, we recognized that fully implementing MCP at this stage would require users to prepare specific configuration files (like mcp.json), while also increasing our maintenance load by having to support both the CLI and MCP interfaces.

After consideration, we determined that optimizing the existing Command-Line Interface (CLI) is a more direct and beneficial first step for everyone. It provides immediate value to users without requiring them to adopt a new protocol, and allows us to focus our efforts.

Therefore, this release is focused on enhancing the CLI to be significantly more agent-friendly. We have introduced several updates, including some breaking changes, to achieve this goal.

Breaking Change: The arbigent command now defaults to --help instead of running tests

Previously, executing arbigent without any arguments would automatically run tests. This behavior was not ideal as it limited the CLI's extensibility. We have now introduced run, scenarios, and tags as subcommands. This change helps coding agents discover the available commands and options more effectively.

$ arbigent
Usage: arbigent [\<options\>] \<command\> [\<args\>]...

Options:
\-h, --help   Show this message and exit

Commands:
run
scenarios
tags

Support for arbigent.properties

(We have received feedback on this and may implement changes in the future, such as requiring the use of .arbigent/settings.local.yaml. However, the core concept will remain unchanged.)

Previously, running the arbigent command required passing configuration options and API keys as environment variables, like so:

arbigent --os=android --project-file=sample-test/src/main/resources/projects/e2e-test-android.yaml --ai-type=gemini --gemini-model-name=gemini-1.5-flash

However, this method is cumbersome for AI agents. To simplify the process, we have added support for an arbigent.properties file.

ai-type=azureopenai

azure-openai-endpoint=https://xxxxxxxxx.openai.azure.com/openai/deployments/xxxxx/
azure-openai-api-version=2025-xx-xx
azure-openai-model-name=gpt-4o
azure-openai-api-key=xxxxxxxxxxxxxxxxxx

project-file=tests/arbigent-project.yml
os=android

Now, when a coding agent calls arbigent run --help, it can see which parameters have already been set via the properties file, indicated by (source: already provided by property file). This means the agent does not need to add these parameters again.

Options for Azure OpenAI:
  --azure-openai-endpoint=<text>               Endpoint URL (source: already provided by property file)
  --azure-openai-api-version=<text>            API version (source: already provided by property file)
  --azure-openai-model-name=<text>             Model name (default: gpt-4o) (source: already provided by property file)
  --azure-openai-api-key, --azure-openai-key=<text> API key (source: already provided by property file)

Options:
  --ai-type=(openai|gemini|azureopenai)  Type of AI to use (source: already provided by property file)
  --ai-api-logging                       Enable AI API debug logging
  --os=(android|ios|web)                 Target operating system (source: already provided by property file)
  --project-file=<text>                  Path to the project YAML file (source: already provided by property file)

As a result, both you and your coding agents can now simply run:

arbigent run

Sophisticated CLI Output

We have reduced the verbosity of debug logs in the standard output. Additionally, the paths to the result artifacts are now clearly displayed at the beginning of a test run, making them easily accessible for coding agents.

 11:48:43 I: 📁 Results will be saved to:
 11:48:43 I:   • YAML Results: /Users/xxx/arbigent-result/result.yml
 11:48:43 I:   • Screenshots: /Users/xxx/arbigent-result/screenshots/
 11:48:43 I:   • API Logs: /Users/xxx/arbigent-result/jsonls/
 11:48:43 I:   • HTML Report: /Users/xxx/arbigent-result/report.html

Breaking Change: Removed API requests and responses from result.yaml

We found that result.yaml could become too large for coding agents to load efficiently due to the inclusion of full API request and response logs. Since these logs are already available in the jsonls/ directory, we have removed them from result.yaml to significantly reduce the file size.

What's Changed

• Add Maestro YAML Integration to README by @takahirom in #298
• Fix scenario generation button by @takahirom in #299
• Enable the CLI to be called by Vibe Coding Tools by loading secret properties from a file by @takahirom in #301
• Fix log level by @takahirom in #302
• Add CLAUDE.md by @takahirom in #303
• Add CLI result summary for coding agents by @takahirom in #304
• Redesign console output by @takahirom in #305
• Add arbigent run subcommand by @takahirom in #306
• Suppress SLF4J logs by @takahirom in #307
• Pass parameters by using just project-files property without run. prefix by @takahirom in #308
• Add defaultOption() which provides (source: already provided by property file) in help by @takahirom in #309
• Add tags command by @takahirom in #310
• Remove api request and response from result yaml by @takahirom in #311
• Add "source: already provided by property file" to all CLI properties by @takahirom in #312

Full Changelog: 0.51.0...0.52.0