Skip to content

v1.6.0: Skill System v2 and the Ghost Skill Hub

Latest

Choose a tag to compare

@C-K-Loan C-K-Loan released this 14 Nov 19:30
· 2 commits to main since this release
651bb8d

Ghost in the Droid v1.6.0: Skill System v2 and the Ghost Skill Hub

Ghost 1.6 replaces the v1 skill runtime with a declarative, content-pinned Skill System v2 and ships the Ghost Skill Hub: 117 registry entries covering 113 Android app packages, each with a score, signed run records, demo clips and a compatibility record. It also adds typed human-in-the-loop steps, supply-chain checks for community skill submissions, an AndroidWorld replay integration, and a rebuilt skills site. All evidence in this release was recorded by the maintainers; none of it is independent yet, and the notes below say so where it matters.

Trailer: ghost-1-6-trailer.mp4 (also attached below). Launch post: Ghost 1.6 blog.

Highlights

  • Skill System v2. Skills are YAML bundles (skill.yaml, SOFT.md, optional file-bound helpers.py) with 26 step kinds and 7 validator kinds, a device/app fingerprint, popup handling, subroutines and typed ask steps. ghost skill now exposes 29 subcommands for the whole lifecycle.
  • Ghost Skill Hub. ghost skill search | info | install | update | outdated | fork | deprecate, with every downloaded file pinned by sha256 and a version commit. The registry lists 117 skills, all installable and all hard, across 113 distinct app packages.
  • Evidence you can read. Each skill has a 0 to 100 score computed from signed run records, split by evidence class (self, maintainer, CI, independent). Downloads and likes earn nothing. Today every run in the hub is maintainer-run, so no skill has independent evidence.
  • Token savings, with the caveat. Across 99 recorded before/after pairs (same prompt, same phone, claude-haiku-5-5), the median token reduction is 90% (10.2x). In 28 of those 99 pairs the run without the skill failed the task, so the all-pairs figure partly compares a failed manual run with a successful replay. On the 71 pairs where both runs succeeded, the median reduction is 86% and the median speedup is 1.09x.
  • Human in the loop. checkpoint pauses for a person (captcha, login, SMS code, confirmation); ask collects a typed answer from a human, the calling agent (run_skill_resume or ghost skill resume) or a bounded sub-agent. Demonstrated on four public captcha demo pages.
  • MCP: find_skill first. New tools find_skill, run_skill, run_skill_resume and get_skill_notes; four private CRM/lead tools are removed, so the server still registers 66 tools. GHOST_MCP_PROFILE selects the skills-first, soft-only or hand-only tool set.
  • Safer submissions. Skill pull requests go through static analysis, a trust gate, fork-safe emulator runs and signed-evidence checks, with seven named supply-chain gates covered by a frozen 22-case local corpus.
  • Opt-in telemetry and guard rails. ghost telemetry on|off|status (default off, DO_NOT_TRACK=1 wins), GHOST_DENY_PACKAGES to keep an agent session out of named apps, and a fingerprint-distance check that refuses runs on a device far from what the skill was recorded on.

Side by side: calculator-comparison.mp4 (attached below).

Skill system

  • Schema v2 (gitd/skills/engine/, spec in docs/skill-schema-v2.md). A skill is skill.yaml plus SOFT.md; skill_class: hard replays without an LLM, skill_class: soft is a guide the agent follows by hand.
  • 26 step kinds: ask, checkpoint, compare, dismiss_popups, dump_xml, for_each, if_visible, launch_app, launch_intent, long_press, open_url, press_key, python_action, read_text, retry, run_skill, screenshot, sleep, subroutine, swipe, tap, tap_element, tap_nth, type_text, wait_for_element, wait_for_text.
  • 7 validator kinds: custom_python, element_present, ocr_text_present, pixel_at, screen_hash, text_visible, url_contains. ghost skill list-registered prints both lists from the code.
  • Fingerprints. Each skill declares OS, OS version range, hardware, app package and app version range. ghost skill run and MCP run_skill compute a distance of 0 to 5: 0 runs, 1 to 2 warns and runs, 3 or more refuses (--force-fingerprint overrides on the CLI). --strict-fingerprint or GHOST_STRICT_FINGERPRINT=1 requires distance 0. An "adapt" tier that rewrites a skill for a nearby device is planned for a later release.
  • Popups. Per-skill popup_detectors, an OS popup catalog, and an unknown-popup safety net. Skills with declared popups no longer get generic popup taps.
  • Python helpers are file-bound. python_action: helpers:<fn> loads from the skill's own helpers.py; evidence for skills that ship Python must carry a skill_sha256 content digest or it earns nothing.
  • Subroutines de-duplicate step lists inside a skill; launch_intent supports URI extras.
  • Authoring commands: ghost skill scaffold, validate, lint, analyze (static Python analyzer), test (--live, --record), run (--param k=v, --device, --secret-env, --record), probe, doctor, diff, keys, import (Maestro and structured Automate flows, with TODO provenance), demo (per-action demo recorder), bench (AndroidWorld replay).
  • Migration of v1 skills: ghost skill migrate <path>... [--author HANDLE] [--dry-run] wraps a v1 skill as v2 in place and keeps all v1 files, so the v1 runtime keeps working.
  • Auto-login state runner (beta). run_skill accepts state_params; a missing state returns needs_state instead of guessing credentials. Marked beta in the tool docstring.
  • Built-in skills moved. The five v1 skills that lived under gitd/skills/ (tiktok, tiktok_ios, safari, play_store and an email-sending skill) are removed from there and now ship as hub skills under skillhub/ghost/. The legacy loaders still scan skillhub/ghost/.
  • Catalog. 116 official skills under skillhub/ghost/ plus one AndroidWorld benchmark skill (androidworld/pro-expense). New official skills include Calculator, Contacts, Clock, Tasks, Markor, Settings, NewPipe, Unitto, Feeder, arXiv search, Open Food Facts and World Clock.
  • Privacy-bounded skills. Settings 1.1.0 returns only aggregate reads (device model/Android/build, Wi-Fi enabled/connected booleans, battery level/charging, storage totals); identifier-bearing paths are removed with no opt-in replacement. NewPipe 1.1.0 exposes search metadata only and never opens a video.

Skill Hub and site

  • Client (gitd/skills/hub.py): fetches skillhub/registry.json from the hub repository, then downloads every file of the chosen version pinned to its commit (jsDelivr at the pinned revision first, the GitHub release tarball second, never raw URLs), verifies sha256, and installs atomically to ~/.ghost/skills/<author>/<slug>/<version>/. GHOST_SKILLS_DIR relocates the install root; GHOST_HUB_REPO and GHOST_HUB_REGISTRY point the client at another hub.
  • Trust labels (separate from quality): official, benchmark, verified, unverified, quarantined. In this tree 116 skills are official and 1 is benchmark; the skillhub/REVIEWED list that grants verified is empty.
  • Quality tiers: draft, runs, proven, hardened, reference (0, 10, 35, 60, 90 points). Scoring table v1.3 (skillhub/scoring/v1.json). Current registry: 28 proven, 75 runs, 14 draft, none hardened or reference. In v1.3 Proven is reachable on your own passing runs; Hardened and Reference still require independent runs.
  • Status: active, blocked, deprecated in the loader, registry and installer.
  • Generated artifacts: skillhub/registry.json, per-skill skillhub/registry/index/*.json (117 files) and SKILL.md wrappers, built by tools/build_registry.py; signed run records under each skill's runs/ and verifications/ (8,984 files under runs/ in this tree).
  • Site pages: /skills/ browser (rank, skill, description, token savings, score), /skills/<author>/<slug>/, /skills/<author>/, /skills/leaderboard/, /skills/scoring/, /skills/showcase/, /skills/compatibility/, /skills/validate/ (client-side manifest validator), /skills/install/<ref> (install QR links), /skills/schema/v2.json, a /.well-known/assetlinks.json placeholder, and a "Ghost Skill" page with agent guides (skillhub/_guides/).
  • Per-skill pages show actions with sample prompts, demo tabs (one clip per action), before/after clips and paired metrics, a reliability card, "Runs by device" tables and recorded-run splits by evidence class.
  • Compatibility matrix (/skills/compatibility/): every skill against every phone and Android version it was run on, with passed/total runs, including screen size, density and locale for new runs. Coverage means tested, not passed.
  • Demos: demo clips for 99 skills and before/after pairs for 99 skills. These show recorded behavior, not independent benchmark coverage.

CLI

  • ghost skill now runs the v2 CLI. It was previously delegated to the legacy parser. The legacy parser is still reachable as android-agent skill ... (and gitd skill ...) with its GitHub-URL and local-path installs.
  • 29 registered subcommands: scaffold, validate, test, run, probe, demo, list-registered, submit, analyze, verify, import, lint, doctor, keys, diff, install, uninstall (remove is an alias), update (--all), outdated, list, info, search, find, fork, deprecate, migrate, bench, resume.
  • Submission: ghost skill submit <dir> [--push] and ghost skill verify <ref> [--push] open PRs; without --push they print the plan. The default target repository is the public Android Agent repository; --repo or GHOST_HUB_REPO overrides it.
  • Resume: ghost skill resume <token> --answer '<json>'.
  • Telemetry: ghost telemetry on|off|status.
  • Device scoping: GHOST_DEVICE (or ANDROID_SERIAL) limits discovery to one connected ref; pasted device ids with quotes, backticks or display suffixes are normalized.
  • Misc: per-tool emoji on the ghost tool-call line; a Makefile with make check (tests, mutants, flake runs).

MCP server

Total registered: 66 tools, the same count as in 1.5.1.

  • New (4): find_skill(intent, top=5) (BM25 over installed skill actions, returns skill, action, score, prompt, params), run_skill(name, action, params, device, state_params), run_skill_resume(token, answer, secrets), get_skill_notes(name) (returns SOFT.md layout notes for driving an app by hand).
  • Removed (4, breaking): lookup_lead, list_unread_leads, crm_lookup_contact, crm_list_unread_messages. The backing modules gitd/services/crm_lookup.py and gitd/services/marketing_lookup.py are deleted. The same four tools are removed from the in-app agent tool list, which gains find_skill and get_skill_notes.
  • Changed: list_devices output is now a header line plus <id> model=<model> rows (iOS rows: model=... platform=iOS via=Appium/WDA); parsers of the old serial (model) form need updating. run_action forwards author/slug names to run_skill. list_skills and the skill loaders include hub skills.
  • Every device argument is normalized (quotes, backticks and display suffixes stripped, validated against the session scope) and passes the package-policy guard.
  • Profiles (GHOST_MCP_PROFILE): skills-first keeps 17 tools (find_skill, list_skills, run_skill, run_skill_resume, create_skill, explore_app, get_skill_notes, list_devices, screenshot, get_elements, tap, type_text, swipe, press_back, press_home, launch_app, wait); soft-only is the same minus run_skill and run_skill_resume (15); hand-only keeps the 10 device tools. An unknown or empty value serves all 66. GHOST_MCP_HIDE_TOOLS=a,b drops named tools.
  • Server instructions tell the agent to call find_skill before driving an app by hand when the skill tools are served.
  • GHOST_DENY_PACKAGES=pkg1,pkg2 makes tool calls refuse denied packages and stops a call when the foreground app cannot be verified. It is an accident guard inside one process, not an OS sandbox.

Human in the loop

  • checkpoint pauses a run (reason: captcha, sms, email, login, llm_decision, generic) until a person resumes or aborts it from the dashboard or POST /api/skills/runs/{id}/resume, or until an optional success: validator passes. Default timeout 600 s; timeout and abort do not trigger the automatic retry.
  • ask collects one typed answer (schema: str, int, enum, json) from who: human | parent_agent | sub_agent and stores it with store_as. For parent_agent, run_skill returns status: needs_input with a resume_token; answer with run_skill_resume or ghost skill resume. Invalid answers are rejected and the token stays usable. Paused state is stored privately under ~/.ghost/runs/; traces note who answered and how long it took, never screenshots.
  • Demos: four public captcha demo pages (reCAPTCHA, hCaptcha, BotDetect, MTCaptcha), each with a fresh agent filling the form, calling a human, and finishing after the human solves it. Clips are sped up. The captchas are solved by a person; nothing here bypasses them.
  • Guide: site/src/content/docs/guides/human-in-the-loop.md.

Demos: four captchas solved by a human mid-run (reCAPTCHA, hCaptcha, BotDetect, MTCaptcha) in the human-in-the-loop guide. The hCaptcha clip is attached below.

Benchmarks and evidence

All of the following is maintainer-run, not independent.

  • Before/after pairs: 99 skills, one recorded pair each (same prompt and phone, with vs without the skill), model claude-haiku-5-5. Median token ratio 0.098 (90% fewer, 10.2x), median time speedup 1.24x, median tool calls 71% fewer. Four pairs used more tokens with the skill. In 28 pairs the no-skill run did not complete the task; restricted to the 71 pairs where both succeeded, the medians are 86% fewer tokens and 1.09x faster.
  • Native showcase pool (skillhub/showcase.yml, tools/bench/): 6 declared tasks across Calculator, Contacts, Clock and three Pro Expense tasks. Model claude-haiku-5-5, 24 paired episodes: 19/24 passed without the skill and 23/24 with it, counted under the declared intention-to-treat rule (capped post-launch failures count as failures). Exact paired McNemar: 5 pairs favor the skill, 1 against, p = 0.21875. This is exploratory, not significant.
  • AndroidWorld integration: ghost skill bench <ref> --androidworld ... replays a skill and lets AndroidWorld's own evaluator score it (tools/bench/aw_bridge.py); agent-arm tools (tools/bench/agent_arm.py, native_ab.py, ab_report.py) are source-checkout only. One AndroidWorld skill is public (androidworld/pro-expense); further AndroidWorld drafts stay out of the public tree until they have replayable steps.
  • Run records: 8,984 signed JSON records under skillhub/*/*/runs/, classified by signing key, not by the label inside the file. Invalid signatures and stale or unbound artifacts are excluded from scores.
  • Compatibility record: raw artifacts cover a Samsung Android 13 phone (4,909 records), a Samsung Android 16 phone (3,656), and emulators on Android 13 and 14. The site builder only counts verified current-version artifacts, so its totals can be lower than raw file counts.
  • Per-action cards expose model, date, seeds, paired statistics, void results and historical findings; actions without a benchmark show as pending.

Security and supply chain

  • Workflow design (docs/skillhub/security.md): PR jobs run on ephemeral hosted runners with no secrets, a read-only token and pull_request (never pull_request_target); reporting runs separately from default-branch code; signing and registry commits happen only after a maintainer merge. Actions are pinned to commit SHAs and linted with zizmor and actionlint.
  • Seven analyzer or evidence gates: network-exfil (undeclared network use; destinations must be listed in data_access), aliased-exec (exec/eval/shell via aliases), soft-injection (agent-override text, hidden characters, encoded blobs in SOFT.md), typosquat-slug, unbound-helper, wrong-app and stale-app evidence rejection.
  • Frozen local corpus (tests/supplychain/): 22 cases, 20 expected catches and 2 benign controls; all 20 are caught and both controls accepted on this tree. This is fixture coverage, not a general security guarantee, and the "13 of 20 before the fixes" comparison cannot be reproduced from the current tree.
  • Trust gate: emulator runs start only when the app is on an allowlist and the skill ships no submitter Python, or a maintainer applies an approved:<full commit sha> label; approval is bound to that commit. CI and trust-policy changes are flagged in the PR comment. The check scripts and skillhub/trusted-vendors.yml are read from the base branch.
  • Signed evidence: run records are signed with SSH Ed25519 keys (skillhub/SIGNERS, skillhub/verifier-keys.yml). A signature shows a record was not altered; it is not proof of provenance.
  • Privacy tooling: forbidden-content scanner (scripts/privacy/scan_forbidden.py, covering the whole tree) and a release gate (scripts/release_gate.sh) that runs it before the release commit.
  • Telemetry design (docs/telemetry.md, infra/telemetry-worker/): opt-in, rotating random install id, enumerated fields only, no retries, never affects scores. The collection endpoint is staged, not deployed.
  • Portal: the on-device Portal accessibility service now preserves node state and includes transient application windows (GhostAccessibilityService.kt, with a new snapshot unit test).

Docs

  • New: Human in the Loop guide, Ghost Skill agent guide, skill creation guide, docs/skill-schema-v2.md, docs/skill-state-model.md, docs/telemetry.md, docs/skillhub/security.md, docs/CI_PHONE_SETUP.md, docs/contributing-skills.md, docs/benchmark-protocol.md, and the 1.6 blog post. Updated: publishing, contributing and using-skills guides, the MCP server and MCP clients pages (profiles), and the skill-system pages.
  • README: MCP tool count badge and text updated to 66, hub quick start (ghost skill search | info | install | run), and a link to the scoring page. CONTRIBUTING.md was cut substantially and now points to the hub guides.
  • Pages for planned features (ghost skill mine, ghost skill try) are marked planned and not part of 1.6.0.

Developer and CI

  • New workflows (9): skill-submission, skill-submission-report, skill-emulator-run, skill-emulator-report, skillhub-registry, skillhub-security, skillhub-stale (daily), skill-drift (weekly fingerprint drift), phone-smoke (throwaway emulator, one job per tests/phone/test_<skill>.py). Updated: ci.yml, demos.yml, privacy.yml, publish.yml.
  • Repo plumbing: .github/CODEOWNERS (generated by tools/build_codeowners.py), skill issue templates (request, bug, first-timer, verify request) and a skill-submission PR template, Makefile (make check), tools/ci/ helpers.
  • Tests: 2,456 tests collected by pytest at this commit across 184 test files (79 test files before); plus site unit and Playwright suites and an Android unit test for Portal. Collected is not passed; the CI run is the authority.
  • Packaging: skillhub/ is excluded from the wheel (exclude = ["skillhub*"]); skills install through ghost skill install, not pip. pexpect joins the test extra.
  • Release tooling: scripts/release.sh and scripts/release_gate.sh strip internal paths before the release commit.

Fixes

  • Device and UI: stale UI dumps are never returned when uiautomator cannot reach idle; the Portal tree is read before uiautomator; Portal "no active window" gaps no longer disable it; find_bounds matches labels with HTML entities; screenshots use LANCZOS for the half-resolution downscale.
  • Skills: Pro Expense accepts AndroidWorld row keys and rejects missing amount or category; Google Clock alarms target the right app; Tasks search query is stable while rows are read; Contacts compatibility; Unitto length choices declared as enum parameters; first-run dialogs handled in Notes.
  • Runner and engine: popup diff initialization, Android package fallback from v2 fingerprints, benchmark import-time directory creation, skills load by reference for agent benchmark calls, run_skill accepts dict-valued params.
  • Hub and site: mobile filter controls, install QR links, compact browse table.

Breaking changes and migration

  1. Four MCP tools removed: lookup_lead, list_unread_leads, crm_lookup_contact, crm_list_unread_messages (also gone from the dashboard agent's tool list; modules gitd/services/crm_lookup.py and gitd/services/marketing_lookup.py deleted). No replacement; they depended on a private lead database.
  2. ghost skill ... is now the v2 CLI. ghost skill install takes hub refs (author/slug[@version]) only. For GitHub-URL, local-path or bare-name installs use android-agent skill install ... (the legacy parser).
  3. v1 skills: run ghost skill migrate <path> [--author HANDLE] [--dry-run]. It appends a generated v2 block to skill.yaml and a generated helpers.py and keeps every v1 file; re-running refreshes the generated block.
  4. Bundled skills relocated: the five v1 skill directories under gitd/skills/ (TikTok, TikTok iOS, Safari, Play Store and an email-sending skill) no longer exist; code importing those packages must use the hub copies under skillhub/ghost/ or an installed copy.
  5. list_devices text format changed (<id> model=<model>); scripts parsing serial (model) must be updated. Pass only the first column as device.
  6. New behavior to know about: runs at fingerprint distance 3 or more are refused (--force-fingerprint on the CLI; no force flag on the MCP tool; GHOST_STRICT_FINGERPRINT=1 refuses at any distance).
  7. Renamed config or environment variables: none found. New variables are additive: GHOST_MCP_PROFILE, GHOST_MCP_HIDE_TOOLS, GHOST_DENY_PACKAGES, GHOST_STRICT_FINGERPRINT, GHOST_SKILLS_DIR, GHOST_HUB_REPO, GHOST_HUB_REGISTRY, GHOST_TELEMETRY_ENDPOINT, GHOST_TELEMETRY_DEBUG, plus the DO_NOT_TRACK opt-out. GHOST_DEVICE already existed and now also scopes discovery.

Upgrade

pipx upgrade ghost-in-the-droid        # or: pip install -U ghost-in-the-droid
ghost skill search calculator
ghost skill info ghost/calculator
ghost skill install ghost/calculator
ghost skill run ghost/calculator calculate --param expression='1234*5' --device <device-id>
ghost skill migrate <path-to-v1-skill> --dry-run   # then again without --dry-run

MCP clients: restart the server after upgrading; optionally set GHOST_MCP_PROFILE=skills-first. Hub installs read the registry from the public repository's main branch, so they work once this release is published there.

Known limitations

  • All recorded runs are maintainer-run. Zero independent runs exist, so no skill can reach Hardened or Reference, the device grid does not establish cross-device reliability, and no skill is marked verified.
  • Proven means the author's own passing runs on every action plus a demo and validators, not independent proof.
  • The token and speed medians come from one pair per skill with one model; 28 of 99 no-skill baselines failed. The native pool result (23/24 vs 19/24) is not statistically significant.
  • Planned, not in 1.6.0: ghost skill mine, ghost skill try, the LLM "adapt" tier, automatic Tier 3 rejection, APK source pinning for CI installs, and recording the CI tier on the registry entry.
  • Hosted release signing, attestation and the tarball fallback need validation in the release environment. The telemetry endpoint is staged, not deployed.
  • The package policy guard is not a sandbox. The auto-login state runner is beta.
  • Skills are pinned to the OS, device and app version they were recorded on; a blank compatibility cell means not tested, not broken.

Discord

Join the Ghost Discord to share the skills and things you build with Ghost, report failures, and get support.