ForgeClaw

ForgeClaw is a supervised visual automation runtime for bounded browser and desktop workflows. It gives an agent a bounded way to observe a screen, plan a single action, act, verify the result, and write evidence for human review.

ForgeClaw can also operate on a native desktop when explicitly enabled, but native mode is elevated risk and is disabled by default.

Current public label: v0.1.0-alpha.5.

Latest alpha.5 changes:

• added a live VM capability campaign that is separate from the public contract benchmark
• added a shared VM case manifest across browser, visual, OCR, replay, hardware, failure, and comparative lanes
• added standardized evidence packets, weighted scoring, and repair-agenda output
• tightened VM browser proof so cdp_eval can target the exact requested URL/host and preserve raw VM payloads
• kept the contract benchmark, but labeled it clearly as contract-only proof rather than live VM proof

Current measured live-proof behavior:

• the new campaign runner writes vm_capability_score.json, VM_CAPABILITY.md, and per-case evidence artifacts
• the smoke run passed the blank-OCR honesty lane and failed the browser lane honestly
• the current live blocker exposed by the browser smoke is VM-side unauthorized, not a hidden false success

What It Does

ForgeClaw helps agents work inside controlled visual environments:

• observe a VM desktop or approved native desktop window
• capture screenshots and OCR-visible text
• run one bounded browser, IDE, keyboard, mouse, or command action
• verify the result against expected state
• classify failures with concrete failure classes
• write artifacts that a human can inspect later

ForgeClaw is not a fully autonomous production-control system. It is designed for supervised automation, smoke tests, visual workflows, and agent research.

Competitive Capability Lanes

ForgeClaw uses stronger executors and parsers as optional lanes while keeping ForgeClaw in charge of policy, artifacts, and verification:

• deterministic Playwright for simple browser evidence and local page checks
• optional exploratory web execution for ambiguous multi-step browser workflows
• normalized screen parsing that merges OCR, layout, and semantic signals into one element map
• bounded record/replay workflows that validate anchors before every action
• public capability benchmarks that produce capability_score.json and CAPABILITY.md

These lanes are optional. The base package remains lightweight and must still degrade to CPU/OCR-only behavior when GPU, browser, or model dependencies are not installed.

Safety Model

ForgeClaw uses explicit execution surfaces and fails closed when it cannot prove where it is operating.

Default behavior:

• surface=vm_guest
• guest_only=true
• host_input=false
• native desktop control blocked
• unknown surface blocked

ForgeClaw fails closed when it cannot prove the execution surface.

Execution Surfaces

ForgeClaw uses explicit execution surfaces:

Surface	Meaning	Default Risk
vm_guest	controlled VM guest input path	controlled
native	host desktop input path	elevated
contract_only	no-mutating dry run/test mode	none
unknown	untrusted or unrecognized surface	blocked

Every run should write the active surface into its metadata and report.

Native Desktop Mode

Native desktop mode exists for workflows that cannot practically run in a VM:

• testing installed desktop applications
• validating OS-specific workflows
• interacting with local IDEs or developer tools
• inspecting unexpected local popups
• diagnosing native automation failures while the operator is present

Native mode must be explicitly enabled:

$env:FORGECLAW_SURFACE = "native"
$env:FORGECLAW_ALLOW_NATIVE = "1"

Equivalent CLI-style intent:

forgeclaw run --surface native --allow-native

Native mode should never be used for unsupervised live-account automation, credential entry, financial actions, private messaging, destructive operations without confirmation, or any workflow where an accidental click can cause real harm.

Native mode runtime proof:

{
  "surface": "native",
  "native_mode_enabled": true,
  "vm_mode_enabled": false,
  "operator_approved_native": true,
  "host_name": "redacted-or-configured",
  "display_target": "primary_desktop",
  "risk_level": "elevated"
}

Quick Start

Install test dependencies in your environment, then run the focused safety contract tests:

py -3 -m pytest tests\test_forgeclaw_vm_guest_act.py tests\test_forgeclaw_controller.py

Run the no-mutation lifecycle demo:

py -3 examples\forgeclaw\contract_only_lifecycle_demo.py

Run the VM browser plan demo:

py -3 examples\forgeclaw\vm_guest_browser_plan_demo.py

Run the public-release audit:

py -3 tools\forgeclaw_public_release_audit.py

Run the public capability benchmark:

py -3 tools\forgeclaw_capability_benchmark.py --output-dir reports\capability

Run the live VM capability campaign:

py -3 tools\forgeclaw_vm_capability_campaign.py --output-dir reports\vm_capability

The public contract benchmark and the live VM campaign are intentionally separate:

• forgeclaw_capability_benchmark.py checks public contracts and evidence shapes without needing a live VM.
• forgeclaw_vm_capability_campaign.py attempts real VM/browser/OCR/replay proof and is expected to fail honestly when the VM path or evidence is weak.

Live VM Capability Campaign

The live VM campaign is the real-world proof surface for ForgeClaw's public claims.

Its rules are stricter than the contract benchmark:

• browser navigation is not enough by itself
• CDP proof must come back from the exact requested URL or host
• a VM observation run cannot claim success without visible-state evidence
• OCR/document cases must extract the required fields, not just any text
• replay must prove anchor match before any action
• blocked/login/transport failures must be reported honestly instead of being turned into generic success

Campaign outputs:

• vm_capability_score.json
• VM_CAPABILITY.md
• per-case evidence_packet.json
• per-case case_result.json

If the live campaign exposes weak areas, the report emits a repair agenda instead of hiding the failure.

Example Workflows

Good public examples:

• open a VM browser to a test URL
• capture a VM screenshot
• OCR visible text from a screenshot
• click a visible VM UI element
• open VS Code inside a VM
• verify that a page or prompt changed
• classify a target-not-ready failure

Examples that should stay private:

• real account logins
• private staff portals
• live marketplace or messaging automation
• private dashboards
• local auth tokens
• personal browser profiles
• customer data

Failure Classes

ForgeClaw should return concrete failure classes:

Failure class	Meaning
vm_unavailable	VM transport or display is unavailable
display_unavailable	screenshot/display source could not be captured
target_not_visible	requested UI target was not visible
ocr_no_text_found	OCR ran but found no useful text
browser_not_ready	browser surface was unavailable
vscode_not_ready	VS Code surface was unavailable
native_desktop_blocked	native mode was requested without explicit approval
surface_unknown	ForgeClaw could not prove VM vs native execution
operator_approval_missing	high-risk action lacks human approval
target_window_not_allowed	target window is not allowlisted
sensitive_window_detected	blocked or sensitive window detected
credential_prompt_detected	credential entry is visible and must pause
risky_action_requires_review	submit/post/pay/send/delete requires review
click_target_unverified	click target could not be proven
route_not_found	expected control route is missing
timeout_waiting_for_state	expected state did not appear in time
human_review_required	automation paused for operator decision
abort_requested	operator or stop file requested abort
native_action_timeout	native desktop action exceeded its timeout

Public Repo Boundary

Keep in a public ForgeClaw repo:

• src/forgeclaw
• ForgeClaw tests
• generic examples
• safety docs
• failure-class docs
• smoke-check scripts
• release audit tooling

Keep out of the public repo:

• credentials or tokens
• .env files
• private agent identities
• customer or employee data
• local workstation paths
• generated screenshots and logs
• browser profiles
• live account automation payloads
• Precision Arts Lab private dashboards and workflows

Security Notes

• Use a VM or disposable environment by default.
• Do not store credentials in the repo.
• Do not enter credentials through automation.
• Capture evidence before acting.
• Require confirmation for destructive or irreversible actions.
• Treat OCR and visual model output as fallible.
• Keep native desktop automation supervised.

Known Limitations

• This is an alpha/developer-preview package.
• Native desktop mode is intentionally conservative.
• Some VM transports require a compatible control service.
• Visual targeting can be wrong and must be verified.
• OCR can miss or misread text.
• Optional browser and vision dependencies are not required for the base package.

Relationship To Precision Arts Lab

ForgeClaw is a reusable subproduct that can be open-sourced as infrastructure. Precision Arts Lab-specific integrations, private workflows, live credentials, business dashboards, customer data, and agent identities should remain private.