Introduce cua-bench-ui submodules (webview API, configurable ports, improved python RPC)

[ddupont808]

This pull request introduces a new lightweight web UI window controller library for CUA bench environments using pywebview. The main features include launching and controlling browser-based desktop windows, querying element positions, and executing JavaScript remotely. The implementation provides a Python API, a child process for window management, and example and test scripts. Additionally, there are improvements in type annotations and API flexibility in the computer package.

Example Usage

def launch_window(
        computer,
        url: Optional[str] = None,
        *,
        html: Optional[str] = None,
        title: str = "Window",
        x: Optional[int] = None,
        y: Optional[int] = None,
        width: int = 600,
        height: int = 400,
        icon: Optional[str] = None,
        use_inner_size: bool = False,
        title_bar_style: str = "default",
    ) -> int:

        @computer.python_command(use_system_python=True)
        def _open(url, html, title, x, y, width, height, icon, use_inner_size, title_bar_style):
            from bench_ui import launch_window
            return launch_window(
                url=url,
                html=html,
                title=title,
                x=x,
                y=y,
                width=width,
                height=height,
                icon=icon,
                use_inner_size=use_inner_size,
                title_bar_style=title_bar_style,
            )
        
        pid = asyncio.run(
            _open(url, html_content, title, x, y, width, height, icon, use_inner_size, title_bar_style)
        )
        return pid

Changes Included

Bench UI library implementation and usage:

• Added new bench_ui library with API functions launch_window, get_element_rect, and execute_javascript for controlling pywebview windows. [1] [2]
• Implemented child process module bench_ui/child.py to manage window lifecycle, serve HTTP control endpoints, and handle incoming requests for element rects and JS execution.
• Provided a comprehensive README and a usage example (examples/simple_example.py) demonstrating launching a window, querying element positions, and executing JavaScript. [1] [2]
• Added a test to verify port detection and remote JS execution after clearing cached PID-port mapping.

Improvements in computer package (type annotations and API flexibility):

• Enhanced type safety in computer/helpers.py by introducing ParamSpec, improving the sandboxed decorator, and adding explicit return types. [1] [2] [3] [4]
• Extended create_interface_for_os in computer/interface/factory.py to accept an optional api_port argument for more flexible interface creation, and renamed the existing port argument to provider_port to increase clarity. [1] [2]

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
docs	Ready	Preview	Comment	Dec 9, 2025 9:25pm

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 9.84456% with 174 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
libs/python/computer/computer/computer.py	6.25%	165 Missing ⚠️
libs/python/computer/computer/helpers.py	70.00%	3 Missing ⚠️
libs/python/computer/computer/interface/factory.py	0.00%	3 Missing ⚠️
libs/python/computer/computer/interface/generic.py	0.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!