Skip to content

dsh-android v0.1.0-rc.2

Pre-release
Pre-release

Choose a tag to compare

View all tags
@Kayshen-X Kayshen-X released this 21 Aug 15:29
· 6 commits to main since this release
v0.1.0-rc.2
46c5ba8

dsh-android v0.1.0-rc.2

原生多模态版：支持图像输入的模型直接"看"安卓屏幕。/ The native-multimodal release: image-capable models now SEE the Android screen.

✨ 新增 / New

  • 截图本身交给模型 / The screenshot itself reaches the model — 五个截图类工具（android_screenshot、android_interact、android_tap_element、android_tap_text、android_tap_row）在路由模型声明 image 输入时，把落定截图作为 {type:'image'} 块附在 JSON 摘要旁——识别界面、读像素里的文字不再经过 OCR 转述。门控与落盘照 DSH 内置 read_image 的官方模式（附件服务持久化 + resolveModelInfo 模态检查）。/ On a route whose resolved model declares image input, all five capture-producing tools attach the settle screenshot as an image block beside their JSON summary, following the in-tree read_image pattern (durable attachment store + modality gate).
  • 降级不拒绝 / Degrade, never refuse — 纯文本模型、未挂附件服务、路由解析失败、附件准入失败：结果与 rc.1 逐字节一致，绝不新增错误。/ A text-only model, an absent attachment store, an unresolvable route, or a failed admission all keep the rc.1 result byte-for-byte.
  • OCR 三件套保留其不可替代的用途：像素级精确的按文字点按，以及纯文本路由/非 macOS 宿主的读屏。skill 剧本已教模型"先用眼睛看"。/ The OCR trio keeps its irreplaceable jobs — pixel-precise text taps, and reading on text-only routes; the skill playbook now says "look first".

🐛 修复 / Fixed

  • 永不空闲的前台不再拖 34 秒 / A never-idle foreground fails fast with a real answer — 浏览器网页这类持续动画的前台会让 uiautomator 永远等不到 idle。现在瞬时动画重试一次；仍失败则跳过注定同样失败的 /sdcard 兜底，并明确指路："改用 android_find_text / android_tap_text（OCR 读像素，不需要 idle）"。在真机 x.com 页面上复现并验证。/ A continuously animating foreground (a web page in Chrome) starves uiautomator's idle wait forever. One retry for transient animations, then the equally doomed /sdcard fallback is skipped and the error routes the model to OCR. Reproduced live against x.com.
  • CI 假 adb 的 stdout 截断 / The fake adb's stdout truncation on Linux — 写完就 process.exit() 会在 Linux 管道上丢弃未刷缓冲（900 行日志只到 251 行，macOS 侥幸全绿）。成功路径改为自然退出。/ process.exit() right after writes discards buffered stdout on Linux pipes (900 chatty lines arrived as 251; macOS happened to swallow it all). Success paths now drain naturally.

📦 安装 / Install

dsh plugin --profile web add @zseven-w/dsh-android@next
dsh web

⚠️ 说明 / Notes

  • npm dist-tag: next。/ Published under next.
  • 新增 18 步 dev-vision-smoke（门控/落盘/双降级/全链路），随 pnpm test 运行。/ An 18-step vision smoke joins pnpm test.
  • 15 个语言 README 均加入"🖼️ 原生多模态"亮点行。/ All 15 locale READMEs gained the Native-multimodal highlight row.

Full Changelog: v0.1.0-rc.1...v0.1.0-rc.2

Assets 2
Loading