feat: ESP32 BLE companion device — port mascot animations to a real desk pet (with bilingual notes)#131
Merged
Conversation
…itHub Actions workflow for ARM64 DMG
…n, StepFun, Trae, and WorkBuddy - Introduced Hermes with body, sleep, work, and alert animations. - Added Kimi with rounded cube design and corresponding animations. - Implemented OpenCode featuring a square block with bracket face. - Created Qoder as a chat bubble with Q face and animations. - Developed Qwen as a 6-pointed star with unique animations. - Added StepFun with a blocky rectangle and step accent. - Implemented Trae as a rounded rectangle with a screen. - Created WorkBuddy featuring a circular body with antenna.
- Updated draw functions for Qwen, Stepfun, Trae, Workbuddy to include scaling and rotation parameters for improved visual effects. - Refactored body drawing functions to support squash effects, enhancing the character's responsiveness during animations. - Improved eye blinking logic and added dynamic eye height adjustments based on animation states. - Enhanced shadow effects and viewport adjustments for smoother transitions and better visual depth. - Adjusted color dimming for various elements to improve overall aesthetics during animations.
- Renamed references from "real-buddy" to "Buddy" across the codebase for consistency. - Updated documentation and comments to reflect the new Buddy terminology. - Enhanced ESP32StatePublisher to include brightness control for Buddy. - Added localization strings for Buddy's features, including connection status and brightness settings. - Modified Settings and SettingsView to accommodate Buddy settings. - Implemented brightness configuration frame in ESP32Protocol and corresponding tests. - Updated hardware firmware to support brightness adjustments and renamed relevant identifiers.
Contributor
Author
|
实机演示效果： |
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Contributor
Author
|
连接配对页演示（图中二维码为我fork项目的链接，在最新commit中已更新为主项目） 
|
- Buddy companion keys (PR 131) and autoApproveTools key (PR 126) both kept
|
That's great.非常期待可以复刻它 |
Contributor
Author
我已经编写好了一版上手文档，如果你感兴趣的话，可以参考尝试一下。https://github.com/Lakphy/CodeIsland/blob/feat/append-buddy-practice/hardware/README.md 期待你的反馈 |
senshinya
added a commit
to senshinya/CodeIsland
that referenced
this pull request
Apr 26, 2026
Ports upstream's Buddy companion-device feature (wxtsky#131, upstream 9c1920e) with the dedup follow-up (upstream 79787e9) folded in: - New `Sources/CodeIslandCore/ESP32Protocol.swift` defines the BLE wire contract: 16 mascot slots, 5 status codes, ≤20-byte downlink frame with UTF-8 truncated tool name, brightness/orientation config frames, uplink button payload. `MascotID(sourceName:)` folds source strings through `SessionSnapshot.normalizedSupportedSource`, so in this fork the only reachable slots are .claude / .codex — but the full 16 enum cases ship intact since the protocol is the on-wire contract with the firmware regardless of which mascots the host actually sends. - `ESP32BridgeManager` (CoreBluetooth central + write/notify chars, reconnect/discovery state machine, selected-buddy persistence), `ESP32StatePublisher` (heartbeat + dirty-notify push that mirrors `NotchPanelView.CompactLeftWing.displaySession`), and `ESP32FocusCoordinator` (button-press → best-session-for-mascot → TerminalActivator) live in Sources/CodeIsland. - TerminalActivator's `sourceToNativeAppBundleId` is dropped from `private` to internal so ESP32FocusCoordinator can reuse the same map for its desktop-app focus fallback. This is the upstream 79787e9 refactor; in upstream it removed a stale duplicate already carried by ESP32FocusCoordinator. We never carried that duplicate — the new file references the shared map directly. - AppDelegate wires the bridge into the app at launch (publisher attach, focus-request callback, configure() with persisted settings). AppState.refreshDerivedState() and rotateToNextSession() call `ESP32StatePublisher.shared.notifyDirty()` so the device sees state changes without waiting for the heartbeat. - Settings: 6 new keys (esp32BridgeEnabled, esp32HeartbeatSeconds, buddyScreenBrightnessPercent, buddyScreenOrientation, selectedBuddyIdentifier, selectedBuddyName). - SettingsView: new `.buddy` page with discovery/select UI, connection status indicator, sync-interval slider, brightness slider, screen orientation picker, and reconnect/forget controls. - L10n: 36 buddy strings (en + zh only — fork's L10n.swift exposes only those two; upstream's ja/ko/tr translations are dropped). - Entitlements: `com.apple.security.device.bluetooth = true`. Info.plist: `NSBluetoothAlwaysUsageDescription` ("CodeIsland uses Bluetooth to mirror the island onto Buddy."). - `hardware/` directory imports upstream's ESP32 firmware verbatim: hardware.ino, 17 mascot headers, mascot_common.h, HARDWARE_NOTES.md, RENDER_OPTIMIZATION.md. The firmware is target-agnostic — it renders whichever mascot the host sends, so the full mascot set ships even though the macOS side only emits claude/codex. - ESP32ProtocolTests (14 tests) covers status mapping, frame encoding (incl. UTF-8 truncation at 17 bytes), brightness clamping, orientation round-trip, and the 16×5 mascot/status combination encode-bounds. `testMascotIDFoldsAllCanonicalSources` is narrowed to `testMascotIDFoldsSupportedSources` (only claude/codex pass through this fork's `supportedSources` gate). All 178 tests pass. - Version bumped to 1.0.23.1-shinya, tracking upstream's v1.0.23 release with this fork's `-shinya` suffix. Skipped from the upstream commit: `.github/workflows/build-macos-arm-dmg.yml` and `scripts/build-dmg.sh` BUILD_ARCH env-var changes (this fork has its own release flow). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
刚刚买了一个 M5stack stickc plus @Lakphy |
Contributor
Author
这个听说不错，我也买一个，看看能不能蒸馏一些灵感🤪 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
This PR ports CodeIsland's mascot animations from the macOS Dynamic Island to a real, palm-sized physical companion device powered by an ESP32-C6 + 1.47" LCD, and adds the Mac-side BLE bridge that drives it. The little screen mirrors whatever your menu-bar island is showing — same mascots, same status states, same vibe — but sitting on your desk instead of pinned under your notch.
I'd love to explore landing this upstream (or at least making the protocol officially "blessed") and would be happy to collaborate on the design and on more board variants.
What's in this PR
1. macOS-side: BLE bridge + settings
Sources/CodeIslandCore/ESP32Protocol.swift— a small, versioned binary protocol (≤ 20 byte BLE writes) defining the contract between the app and the device. Service UUID0000beef-…, downlink frames (sourceId, statusId, toolName), brightness/orientation config frames (0xFE/0xFD), and a 1-byte uplink for the device button.ESP32BridgeManager.swift/ESP32StatePublisher.swift/ESP32FocusCoordinator.swift— Core Bluetooth central that scans for a peripheral namedBuddy, manages reconnect with backoff, debounces frames, and forwards button presses back to the existing terminal-focus pipeline (TerminalActivator).SettingsView.swift/Settings.swift/L10n.swift(EN + zh-CN): enable bridge, sync interval, screen brightness, screen orientation (180° flip).CodeIsland.entitlements/Info.plist: addsNSBluetoothAlwaysUsageDescriptionand the bluetooth entitlement.Tests/CodeIslandCoreTests/ESP32ProtocolTests.swift— unit tests for the encoder/decoder (truncation, max length, marker frames).2. Firmware (
hardware/)Arduino sketch for Waveshare ESP32-C6-LCD-1.47 (ST7789, 172×320). It advertises as
Buddy, exposes the BLE service above, and renders the mascot the host tells it to.hardware.ino— main sketch: BLE GATT server, frame parser, scene scheduler, button handling (short = next mascot, long = demo mode), backlight PWM with idle dimming, orientation flip.mascot_*.h— per-mascot animation modules:claude(Buddy),codex,cursor,copilot,gemini,droid,dex,antigrav,clawd, plus the newhermes,kimi,opencode,qoder,qwen,stepfun,trae,workbuddy. Same status states as the app: idle / thinking / tool-use / done / error.HARDWARE_NOTES.md— board pinout, library list, upload tips.RENDER_OPTIMIZATION.md— notes on usingGFXcanvas16double-buffering to kill the black-flicker.3. Build / CI
.github/workflows/build-macos-arm-dmg.yml— Apple-Silicon DMG build workflow.scripts/build-dmg.shupdated to take an architecture flag.Hardware needed to actually use it
Minimum bill of materials for one Buddy:
HARDWARE_NOTES.mdassume this exact board.GPIO9↔GND. Used for "next mascot" / "demo mode" / focus-request.Any other ESP32 + ST7789 combo should work with minor pin changes; the firmware does not assume Wi-Fi.
Quick start
hardware/hardware.inoin Arduino IDE; installAdafruit GFX+Adafruit ST7735 and ST7789 Library.ESP32C6 Dev Module,USB CDC On Boot = Enabled. Flash.On the animation code — credit & licensing
I want to be very upfront about this: the canvas/mascot animations on the ESP32 are direct re-implementations of the same animation logic used by your macOS Dynamic Island views in this project (e.g.
BuddyView.swift,CursorView.swift, etc.). The motion curves, color palettes and per-state choreography are intentionally identical so that the physical device feels like the same character that lives in the island — that's the whole point of the project.Why I'd love to collaborate
I'm a long-time CodeIsland user and the menu-bar mascots genuinely make my coding day better. Putting one on my desk as a tiny pet that reacts to Claude / Codex / Cursor / Gemini / etc. has been delightful, and a few friends have already asked where to buy one. I'd like to:
Totally fine if the answer is "thanks, but I'd prefer to keep this as a fork-only experiment" — I just wanted to offer it back first.
Testing done
build.sh) on Apple Silicon; new unit tests pass (swift test).Happy to record a short demo video if useful.
Thanks for building CodeIsland — it's a lovely piece of software. 🙇
中文版本
简介
这个 PR 把 CodeIsland 灵动岛上的吉祥物动画移植到了一台真实的、巴掌大的实体伴侣设备上：硬件是 ESP32-C6 + 1.47 寸 LCD，并配套加入了 Mac 端通过蓝牙驱动它的桥接代码。小屏幕上显示的就是你菜单栏灵动岛此刻显示的内容 —— 同样的吉祥物、同样的状态切换、同样的气质 —— 只不过它现在摆在你桌子上，而不是挂在你的刘海下面。
我希望能把这部分能力合入上游（或者至少把协议变成"官方认可"的），也很想跟你一起继续把这件事做下去。
这个 PR 包含什么
1. macOS 端：蓝牙桥接 + 设置
Sources/CodeIslandCore/ESP32Protocol.swift：一个小而稳的二进制协议（每帧 ≤ 20 字节），定义 App 与设备之间的契约。Service UUID0000beef-…、下行帧（sourceId / statusId / toolName）、亮度与方向配置帧（0xFE/0xFD），以及设备按键的 1 字节上行帧。ESP32BridgeManager.swift/ESP32StatePublisher.swift/ESP32FocusCoordinator.swift：基于 Core Bluetooth 的中心端，扫描名为Buddy的外设，处理带退避的自动重连、帧去抖，并把设备上的按钮事件接回原项目的终端聚焦链路（TerminalActivator）。SettingsView.swift/Settings.swift/L10n.swift（中英双语）中新增 "Buddy" 设置分组：开关蓝牙桥、同步间隔、屏幕亮度、屏幕方向（180° 翻转）。CodeIsland.entitlements/Info.plist：补上NSBluetoothAlwaysUsageDescription和蓝牙能力。Tests/CodeIslandCoreTests/ESP32ProtocolTests.swift：协议编解码的单元测试。2. 固件（
hardware/）为 Waveshare ESP32-C6-LCD-1.47（ST7789，172×320）写的 Arduino 工程。设备以
Buddy名称广播，对外暴露上述 BLE 服务，并按 Mac 下发的内容渲染对应吉祥物。hardware.ino：主 sketch，包括 BLE GATT 服务端、帧解析、场景调度、按键处理（短按切换吉祥物 / 长按进入演示模式）、背光 PWM 与空闲降亮、屏幕方向翻转。mascot_*.h：每个吉祥物一个动画模块：claude(Buddy)、codex、cursor、copilot、gemini、droid、dex、antigrav、clawd，以及新增的hermes、kimi、opencode、qoder、qwen、stepfun、trae、workbuddy。状态机与 App 对齐：idle / thinking / tool-use / done / error。HARDWARE_NOTES.md：开发板引脚、库依赖、上传踩坑。RENDER_OPTIMIZATION.md：使用GFXcanvas16离屏双缓冲解决黑屏闪烁的笔记。3. 构建 / CI
.github/workflows/build-macos-arm-dmg.yml：Apple Silicon DMG 构建工作流。scripts/build-dmg.sh增加架构参数。跑起来需要的硬件
最小物料清单：
HARDWARE_NOTES.md中的引脚就是基于这块板GPIO9↔GND，用于"下一个吉祥物 / 演示模式 / 请求聚焦终端"固件不依赖 Wi-Fi，只用 BLE，因此其它 ESP32 + ST7789 组合稍改引脚也可以跑。
快速上手
hardware/hardware.ino，安装Adafruit GFX与Adafruit ST7735 and ST7789 Library。ESP32C6 Dev Module，开启USB CDC On Boot，烧录。关于动画代码 —— 署名与版权说明
这一点我想说得明确一点：ESP32 上的 canvas 动画，是对你这个项目里 macOS 灵动岛视图（如
BuddyView.swift/CursorView.swift等）所使用的同一套动画逻辑的直接重新实现。运动曲线、配色和各状态下的演出节奏都是有意保持一致的 —— 因为这个项目的全部意义就在于：让桌面上这只小家伙看起来"就是"灵动岛里那只小家伙。为什么想跟你合作
我自己是 CodeIsland 的长期用户，菜单栏上的小家伙真的让我每天写代码的心情更好一些。把它做成一个会对 Claude / Codex / Cursor / Gemini 等 Agent 状态做出反应的桌面小宠物之后，已经有几个朋友问我哪里能买到一个。所以我希望：
如果你的答案是"谢谢，但我还是希望它只作为一个 fork 上的实验"，完全没问题 —— 我只是想先把它送回来给你看看。
已做的测试
build.sh通过；新增单测swift test通过。如果有需要可以录一段演示视频。
最后再次感谢你做了 CodeIsland —— 它是个非常可爱也非常用心的作品。🙇