Repository navigation
v0.27.1
v0.27.1 — 深度工具强化:AI 现在能「看见」界面并精确点击
UI 自动化质变(核心)
此前 AI 只能盲猜坐标点击,不知道按钮在哪里。现在形成完整闭环:
ua_find_text查看界面结构:列出屏幕上所有文字元素的坐标与可点击状态(可留空列出全部可点项)ua_tap_text按文字点击:「点登录按钮」——基于无障碍节点查找,比坐标点击可靠得多screen_ocr默认返回每行文字坐标文字 @(x,y):AI 读到内容就知道位置,配合坐标点击精确操作- 全流程:看界面 → 按文字点 → OCR 验证结果,任意 App 自动化真正可用
其他深度强化
calculator科学计算:sin/cos/tan/sqrt/log/ln/floor/ceil/round(角度制)、常量 pi/e、百分比(200*15%= 30)get_time日期计算:days_offset回答「3 天后是几号星期几」write_document追加模式:分多次写长文档不再覆盖media_control查询当前曲目:歌名/艺术家/专辑/播放进度read_notifications过滤:按时间范围(最近 N 小时)+ 应用名筛选fetch_rss条目摘要:订阅条目附带内容摘要
工具总数 63。需要无障碍 + 截屏权限(设置 → 权限)。
SHA-256:
- full:
2762820e2825d8eae8490a66c384e61d72a5ae2274da8ff6ceb3756c39b9b41e - lite:
03aaba5f2c39be92acc9ae59d110b24e10b85d0a3d6cc14b82925d7ac92a19dd
v0.27.1 — Deep tool upgrades: the AI can now SEE the UI and tap precisely
UI automation breakthrough
The AI used to guess coordinates blindly. Now it's a closed loop:
ua_find_text: lists every text element with coordinates and clickability (leave text empty to list all tappable items)ua_tap_text: taps an element by its text ("tap the Login button") via the accessibility tree - far more reliable than coordinatesscreen_ocrnow returns per-line coordinatestext @(x,y)by default, so the AI knows where things are and can tap precisely- Workflow: inspect UI -> tap by text -> verify with OCR. Real automation of any app.
Other deep upgrades
calculator: scientific functions (sin/cos/tan/sqrt/log/ln/floor/ceil/round, degrees), constants pi/e, percentages (200*15%= 30)get_time:days_offsetanswers "what weekday is it 3 days from now"write_document: append mode for multi-part long documentsmedia_control: status action returns the current track (title/artist/album/progress)read_notifications: filter by time range (last N hours) and app namefetch_rss: entries now include content summaries
63 built-in tools. Requires Accessibility + screen-capture permissions (Settings -> Permissions).
SHA-256:
- full:
2762820e2825d8eae8490a66c384e61d72a5ae2274da8ff6ceb3756c39b9b41e - lite:
03aaba5f2c39be92acc9ae59d110b24e10b85d0a3d6cc14b82925d7ac92a19dd