Skip to content

list_windows MCP response format on Linux (wlroots) #2041

Description

@NyllRE

Environment

  • cua-driver: 0.6.8
  • OS: NixOS 26.11 (Zokor), x86_64
  • Display server: Wayland (niri — wlroots-based)
  • Architecture: x86_64

Summary

On this Linux/wlroots system, computer_use capture calls fail in an MCP client (Hermes Agent) with:

{"error": "capture failed: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'"}

The client's capture flow calls list_windows first, then get_window_state. Tracing the data path, the issue seems to be in how list_windows delivers its response over MCP, or in the window fields it includes on wlroots.

Findings

1. list_windows via CLI works

$ cua-driver call list_windows '{"on_screen_only": true}'
{
  "windows": [
    {
      "height": 10,
      "pid": null,
      "title": "Chromium clipboard",
      "width": 10,
      "window_id": 4194304,
      "x": -100,
      "y": -100
    },
    {
      "height": 200,
      "pid": 300523,
      "title": "spotify",
      "width": 200,
      "window_id": 6291459,
      "x": 0,
      "y": 0
    }
  ]
}

2. get_window_state via CLI works

Calling on a visible Spotify window returns valid data with screenshot, elements, and tree_markdown.

3. The MCP client fails to find windows

The client (Hermes Agent v0.17.0) calls list_windows over MCP and expects the window array in the MCP structuredContent field of CallToolResult:

# Hermes wrapper, cua_backend.py line 1030
raw_windows = (lw_out.get("structuredContent") or {}).get("windows") or []

When structuredContent is absent, it finds zero windows and returns an empty capture, which then hits an int(None) downstream. This suggests list_windows over MCP may deliver its data as a text content part rather than structuredContent.

Alternatively, if list_windows does deliver via structuredContent on this platform, the window objects lack several fields the client expects:

Field Present in CLI output Expected by Hermes wrapper
height
pid ✅ (but can be null)
title
width
window_id
x
y
app_name yes
z_index yes (sort key)
is_on_screen yes

Questions

  1. Does the MCP server for list_windows intend to deliver the window array as structuredContent or as a text content part? If structuredContent, is there a known issue with it not being set on Linux/wlroots?
  2. The pid field can be null — is this expected for certain windows on wlroots (e.g., the "Chromium clipboard" utility window), and should the MCP response handle this explicitly?
  3. Could list_windows include app_name, z_index, and is_on_screen fields on wlroots for consistency with other platforms?

Reproduction

# Start the MCP server
cua-driver mcp

# In another terminal, inspect what list_windows returns over MCP
# Or run capture via any MCP client that uses the structuredContent path

Let me know if you need more details or if I should test with a specific patch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions