Skip to content

v0.1.28

Choose a tag to compare

@github-actions github-actions released this 17 Oct 22:34
· 39 commits to main since this release
1d5afbd

The assistant can look at screenshots from your MCP tools

When a tool such as windows-commander's capture_screen returns a screenshot,
the model now actually sees it. Ask "what is on my screen?" in a voice call or
in the Prompt section and it can read and describe it. Before, the picture was
passed on as a long block of encoded text that the model could not use, and a
large screenshot could break the request.

This works with OpenAI and Gemini models. The older
gemini-2.5-flash-native-audio voice model sometimes starts guessing before
the picture arrives and then corrects itself; the 3.x Live models wait for it.

Full changelog: v0.1.27...v0.1.28