Gua

Gua is a runtime UI automation protocol for games.

It exposes a semantic UI tree from a running game, so test runners and AI agents can inspect, query, click, and verify in-game UI without relying on fragile image recognition or coordinate-based input.

NuGet Packages

Gua.Core:
P/Invoke bindings for using the Gua C ABI runtime from .NET, including the Windows x64 native runtime.
Gua.Testing:
Adds Gua locators, waits, assertions, and adapter test loops to regular .NET tests.
Gua.Testing.Godot:
Starts a Godot process and provides helpers for controlling and verifying a running scene through the Gua bridge.

MCP and Inspector

gui-mcp:
A thin MCP server that exposes Gua runtime actions to AI agents through the same WebSocket bridge used by the Inspector.
Gua Inspector:
A browser and Windows desktop UI for inspecting the semantic UI tree, node state, screenshots, and logs, and for sending runtime commands.

await game.getByRole("button", { name: "Start Game" }).click()
await expect(game.getById("loading")).toBeVisible()

The current implementation exposes that shape for C++ and C# over the C ABI, adds Inspector and MCP consumers, and includes Godot 4.7 adapters and samples:

gua::testing::get_by_role(ctx, "button", "Start Game").click();
gua::testing::wait_for_text(ctx, "Loading...").to_be_visible();

GuaAssertions.GetByRole(ui, "button", "Start Game").Click();
GuaAssertions.WaitForText(ui, "Loading...").ToBeVisible();

Click() intentionally enqueues a click request; it does not directly mutate the game. A game adapter, such as the ImGui or Godot adapter, must consume the request on a later frame and emit the observed click event. Tests that do not run a real engine adapter can use GuaTestHost for the same loop:

using var ui = new GuaContext();
var host = new GuaTestHost(ui);
var loading = false;

host.Frame("title", frame =>
{
    frame.Button("start", "Start Game", new GuaBounds(100, 100, 240, 64));
});

GuaAssertions.GetByRole(ui, "button", "Start Game").Click();

host.Frame("title", frame =>
{
    frame.Button("start", "Start Game", new GuaBounds(100, 100, 240, 64));
});

host.DrainClickEvents(id => loading = id == "start");

host.Frame("loading", frame =>
{
    if (loading)
    {
        frame.Text("loading", "Loading...", new GuaBounds(100, 180, 240, 24));
    }
});

GuaAssertions.WaitForText(ui, "Loading...").ToBeVisible();

Real .NET tests should use an existing test runner such as NUnit. Gua.Testing does not try to become a Vitest-style runner; it provides Gua-specific locators, waits, assertions, and adapter test loops inside normal NUnit tests. One C# file can contain multiple [Test] methods:

using Gua.Core;
using Gua.Testing;
using NUnit.Framework;

[TestFixture]
public sealed class TitleScreenTests
{
    [Test]
    public void StartClickShowsLoadingText()
    {
        using var _ = GuaAssertionScope.UseNUnit(Assert.Fail);
        using var ui = new GuaContext();
        var host = new GuaTestHost(ui);

        host.Frame("title", frame =>
        {
            frame.Button("start", "Start Game", new GuaBounds(100, 100, 240, 64));
        });

        GuaAssertions.GetByRole(ui, "button", "Start Game").Click();
    }
}

The repository includes a runnable NUnit sample in examples/dotnet-nunit.

Godot scene tests can use the Gua.Testing.Godot package to start a Godot process and assert against the live Gua bridge:

using var host = GodotSceneTestHost.Load("game/scenes/title_screen.tscn");

GuaAssertions.GetByRole(host.Context, "button", "開始").ToBeVisible();
host.Click("CenterPanel/Content/ButtonBox/StartButton", nextScene: "game/scenes/village_list.tscn");
GuaAssertions.GetByRole(host.Context, "button", "Create").ToBeVisible();

For v0.2.0 and later, repository-local GuaLiveAssertions helpers can migrate directly to WaitForId / WaitForText / WaitForVisible / WaitForEnabled, locator WaitForCount*, and correlated node actions such as ClickAsync. Wait-returned expectations retain the successful completed-frame snapshot; use Refresh() or WaitUntil* when a later frame is required. Repository GodotHostFactory wrappers should keep only game-specific process and fixture policy while GodotSceneTestHost owns executable/project discovery, automatic loopback ports, rendered/headless startup, bridge diagnostics, and reset policy. All APIs are additive over the existing C ABI and WebSocket protocol.

Failure diagnostics can be coordinated through the framework-independent GuaDiagnosticsSession. Assertions and completed actions use the same artifact layout, return typed absolute paths/media types/secondary capture errors, and can attach through a callback. Godot sessions can include live process metadata, stdout/stderr, runtime version, and an opt-in on-demand screenshot; successful tests do not create artifacts unless capture is explicitly requested.

Want to try that in GitHub Actions without wiring every setup step by hand? link1345/gua-tester provides reusable Actions for Godot GDScript projects. It downloads Godot on the runner, links the released Gua Godot addon into your project, sets GODOT_EXECUTABLE, and runs your .NET tests that use Gua.Testing.Godot.

For a typical consumer repository, the workflow can be as small as:

- uses: link1345/gua-tester@v1
  with:
    project-path: game
    test-project: tests/GuaTester.Tests.csproj
    godot-version: "4.7"
    godot-status: stable

If you have a Godot project that starts the Gua bridge, give it a spin in CI and see whether the semantic UI tree catches regressions before they land.

context.log(gua::LogLevel::info, "title screen opened");
context.set_screenshot("data:image/png;base64,...", 1280, 720);

std::cout << context.ui_tree_json() << '\n';
std::cout << context.logs_json() << '\n';
std::cout << context.screenshot_json() << '\n';

while (ui.TryPollEvent(out var e))
{
    if (e.Type == GuaEventType.Click && e.NodeId == "start")
    {
        ShowLoading();
    }
}

Gua is not a game engine.
Gua is not an editor MCP.
Gua is not an image-recognition QA bot.

Gua is a small bridge between a game runtime and automation tools.

Scope

The first implementation focuses on a small, stable core:

Protocol specification and JSON schemas
C ABI runtime core
Thin C++ wrapper
ImGui adapter
C++ and C# testing helpers
.NET P/Invoke binding over the C ABI
Inspector for tree, node detail, screenshot, logs, and runtime commands
MCP server for AI-agent access to the runtime bridge
Godot 4.7 C# sample using the shared native runtime bridge
Godot 4.7 GDScript sample using the same runtime bridge through GDExtension

Engine-specific integrations such as Unity, Unreal Engine, Godot, and MonoGame are expected to be adapters built on top of the protocol, not the center of the project.

Native Toolchain

Runtime compatibility can be inspected through the additive C ABI gua_copy_version_json, the WebSocket get_version command, or .NET GetVersion(). The response follows protocol/schema/version.schema.json and lists stable capability IDs. Non-Godot runtimes report godotPluginVersion as null; release workflows inject the release version and commit build ID.

Windows native development uses MSVC as the primary toolchain:

cmake --preset windows-msvc-debug
cmake --build --preset windows-msvc-debug

The portable boundary remains the C ABI. macOS and iOS should use Apple Clang, and Android should use Android NDK Clang when those targets are added.

.NET Testing

The .NET packages are published on NuGet and can also be packed locally:

dotnet pack bindings/dotnet/src/Gua.Core/Gua.Core.csproj --configuration Release
dotnet pack bindings/dotnet/src/Gua.Testing/Gua.Testing.csproj --configuration Release

The packages are written to artifacts/packages. Gua.Testing declares a NuGet dependency on the matching Gua.Core package, so a test project only needs the testing package:

<PackageReference Include="Gua.Testing" Version="0.5.0-preview.3" />

Gua.Core is also delivered as a NuGet package and Gua.Testing depends on the matching version. The Windows x64 native runtime is included in Gua.Core under runtimes/win-x64/native/gua.dll, so a normal package restore/build copies it to the consuming app or test output. GUA_NATIVE_DIR remains as an override for locally built native runtimes. A missing or wrong architecture native library is reported with the exact checked paths.

Run the standalone testing sample:

cmake --preset windows-msvc-debug
cmake --build --preset windows-msvc-debug --target gua
$env:GUA_NATIVE_DIR = "$PWD\build\windows-msvc-debug\native\gua-core\Debug"
dotnet run --project examples/dotnet-testing/GuaDotNetTestingSample.csproj

Run the NUnit testing sample:

cmake --preset windows-msvc-release
cmake --build --preset windows-msvc-release --target gua
dotnet pack bindings/dotnet/src/Gua.Core/Gua.Core.csproj --configuration Release
dotnet pack bindings/dotnet/src/Gua.Testing/Gua.Testing.csproj --configuration Release
dotnet test examples/dotnet-nunit/GuaDotNetNUnitSample.csproj

Inspector

The Inspector is a React application that consumes Gua protocol snapshots. It is not tied to MCP. The UI talks to a GuaInspectorClient abstraction so transport implementations can be added for mock data, WebSocket bridges, HTTP bridges, MCP, saved files, or the native runtime bridge.

Run the browser version:

bun run --filter @gua/inspector dev

Run the sample WebSocket bridge in another terminal:

bun run bridge:ws

Or build and run the native C++ bridge that serves a real gua::Context:

cmake --preset windows-msvc-debug
cmake --build --preset windows-msvc-debug --target gua-native-bridge-example
.\build\windows-msvc-debug\examples\native-bridge\Debug\gua-native-bridge-example.exe

The ImGui example also hosts the same WebSocket bridge while the UI is running:

cmake --preset windows-msvc-debug
cmake --build --preset windows-msvc-debug --target gua-cpp-imgui-example
.\build\windows-msvc-debug\examples\cpp-imgui\Debug\gua-cpp-imgui-example.exe

Then connect the Inspector to:

ws://127.0.0.1:8765

The ImGui bridge pushes snapshot notifications while the UI is running, so the Inspector updates without polling. The Poll toggle in the Inspector remains as a fallback for bridges that only implement request/response.

The bridge speaks the same JSON command shape used by Gua runtime adapters:

{ "id": 1, "type": "get_ui_tree" }
{ "id": 2, "type": "click_node", "nodeId": "start" }

Build the static Inspector:

bun run --filter @gua/inspector build

Run the Tauri desktop shell during development:

bun run --filter @gua/inspector tauri:dev

Tauri requires a Rust toolchain in addition to the JavaScript dependencies.

MCP

The MCP server is a thin protocol consumer over the same bridge used by the Inspector. Start a runtime bridge first:

bun run bridge:ws

Then run the MCP server over stdio:

bun run mcp

The MCP server is published to npm as gui-mcp. MCP clients can start it with:

bunx gui-mcp@latest mcp

By default it connects to ws://127.0.0.1:8765. Override that for another runtime adapter:

$env:GUA_BRIDGE_URL = "ws://127.0.0.1:8765"
bunx gui-mcp@latest mcp

The MCP tool surface is:

get_ui_tree
click_node
press_key
wait_for_node
get_screenshot
get_logs
run_test

Release automation

GitHub Actions publishes runtime artifacts from main when a related protocol consumer changes. The Inspector, Godot plugin, and ImGui plugin are built together and attached to one versioned gua-v* GitHub Release. MCP and .NET packages continue to use their independent npm and NuGet publishing workflows.

The MCP workflow uses npm trusted publishing through GitHub Actions OIDC. The gui-mcp package must be configured on npm with this repository, the .github/workflows/mcp-publish.yml workflow, and the release environment.

Godot 4.7 C# Sample

The v0.5 C# runtime sample lives in examples/dotnet-godot. It is a minimal Godot 4.7 project using Godot.NET.Sdk/4.7.0, net10.0, project references to Gua.Core and Gua.Testing, and a runtime addon in addons/gua. The addon P/Invokes gua_runtime.dll, which owns both the Gua context and the Inspector WebSocket bridge.

Build it with:

dotnet build examples/dotnet-godot/GuaGodotSample.csproj -v:minimal

The sample uses Gua.Godot.GuaGodotRuntime from the addon, attaches it to the root Godot Control, and starts an Inspector bridge on ws://127.0.0.1:8765. The adapter collects standard controls into the semantic tree and dispatches external click requests through normal Godot button signals. examples/dotnet-monogame is still present as a future placeholder.

Godot 4.7 GDScript Addon

The GDScript-facing adapter lives in native/gua-godot and builds a thin GDExtension wrapper over native/gua-runtime. It exposes GuaContext to GDScript without reimplementing the runtime core or the Inspector bridge:

var ui := GuaContext.new()
ui.begin_frame("title")
ui.register_node("start", "button", "Start Game", Rect2(512, 312, 256, 56), true, true)
ui.end_frame()
ui.start_inspector_bridge(8765)
ui.enqueue_click("start")

while true:
    var event := ui.poll_event()
    if event.is_empty():
        break

Build the Windows debug GDExtension:

cmake --preset windows-msvc-debug
cmake --build --preset windows-msvc-debug --target gua-godot

The build writes the debug DLL into examples/godot-gdscript/addons/gua/bin. Open examples/godot-gdscript with Godot 4.7 and run the project. The running game process starts an Inspector bridge on ws://127.0.0.1:8765, which Gua Inspector can connect to while the game is open. The addon includes plugin.cfg only for standard Godot addon packaging; Gua's runtime API is provided by gua.gdextension, not by an editor MCP.

For game scripts, instantiate the GDScript adapter through an explicit preload instead of relying on class_name registration order:

const GuaAutoAdapterScript := preload("res://addons/gua/gua_auto_adapter.gd")

var ui := GuaAutoAdapterScript.new()

The adapter resolves the native GuaContext class through ClassDB on first use and verifies that required methods such as consume_click_request exist before dispatching Inspector click requests. If that check fails, rebuild gua-godot; the stale vendored DLL is the problem, not the game script.

Run the GDScript smoke check with:

C:\Users\testk\.local\bin\Godot_v4.7-stable_win64_console.exe --headless --path examples/godot-gdscript --script res://scripts/gua_smoke.gd

Repository Layout

protocol/             Protocol specs and JSON schemas
native/gua-core/      C ABI runtime core and C++ reference implementation
native/gua-runtime/   Shared native runtime bridge for Godot C# and GDScript
native/gua-imgui/     ImGui adapter layer
native/gua-testing/   C++ testing helpers over the C ABI
native/gua-godot/     Godot GDExtension adapter for GDScript
bindings/dotnet/      .NET P/Invoke binding and C# testing helpers
bindings/dotnet/src/Gua.Testing.Godot/
                      Godot process test helpers over Gua.Testing
packages/mcp/         Published MCP server package
packages/inspector/   Browser and Tauri desktop Inspector UI
examples/             Minimal demos and samples, including the Godot C# sample
docs/                 Native toolchain guidance

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 99 Commits
.codex/skills		.codex/skills
.github		.github
bindings/dotnet		bindings/dotnet
docs		docs
examples		examples
native		native
packages		packages
protocol		protocol
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CMakeLists.txt		CMakeLists.txt
CMakePresets.json		CMakePresets.json
CONTRIBUTING.md		CONTRIBUTING.md
Directory.Build.props		Directory.Build.props
LICENSE		LICENSE
README.ja.md		README.ja.md
README.md		README.md
bun.lock		bun.lock
package.json		package.json
tsconfig.base.json		tsconfig.base.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Gua

NuGet Packages

MCP and Inspector

Scope

Native Toolchain

.NET Testing

Inspector

MCP

Release automation

Godot 4.7 C# Sample

Godot 4.7 GDScript Addon

Repository Layout

License

About

Uh oh!

Releases 21

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Gua

NuGet Packages

MCP and Inspector

Scope

Native Toolchain

.NET Testing

Inspector

MCP

Release automation

Godot 4.7 C# Sample

Godot 4.7 GDScript Addon

Repository Layout

License

About

Topics

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 21

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages