Skip to content

Tej-Sharma/selfloop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

selfloop - automatically test your web + mobile apps

image

Point it at your iOS app or your web app. It drives the simulator or a real browser like a human tester would — tapping around, filling forms, trying the things a QA person tries — and comes back with an HTML report of every bug it hit, screenshot attached.

  • Finds real bugs, not lint warnings: typos, broken flows, dead placeholders, crashes.
  • Never forgets a screen. A coverage map persists across runs, so it knows what it hasn't tried yet.
  • Grounded, not vibes. Every flaw and every "flow completed" claim is logged with the exact screenshot that proves it.
  • One brain, two hands. The same exploration loop drives both platforms — only the "hands" (mobile-mcp vs. Playwright) differ.

Installation

Two ways to run it — pick based on what you have available:

Option 1 — Most Powerful + Fastest (v2-engine, direct)

Runs the new single-brain explore loop directly against a model over OpenRouter (default google/gemini-3.5-flash) — no Claude Code CLI in the loop, so each step is one fast API call instead of a Claude Code turn.

  1. Clone the repo and install the engine's dependencies:

    git clone https://github.com/Tej-Sharma/selfloop.git
    cd selfloop/v2-engine
    npm install
  2. Web only — one-time browser install:

    npm run setup:web
  3. Get an OpenRouter API key and export it:

    export OPENROUTER_API_KEY="sk-or-..."
  4. Describe your target app in an inputs file (gitignored — this is where credentials live).

    Webweb/inputs/my-app.json:

    mkdir -p web/inputs
    cat > web/inputs/my-app.json <<'EOF'
    {
      "url": "https://my-app.example.com",
      "credentials": { "username": "demo", "password": "demo" },
      "about": "One line description of what the app does."
    }
    EOF

    iOS — build and install your app on a booted Simulator first (same as you'd do from Xcode), grab its UDID, then write mobile/inputs/<bundle-id>.json:

    xcrun simctl list devices | grep Booted   # copy the UDID
    
    mkdir -p mobile/inputs
    cat > mobile/inputs/com.your.bundleid.json <<'EOF'
    {
      "credentials": { "username": "demo", "password": "demo" },
      "notes": "Log in, then explore every tab.",
      "deviceUdid": "YOUR-SIMULATOR-UDID"
    }
    EOF
  5. Run it — explore, critique, design-check, and report in one shot:

    npm run test-app:web
    # or:
    DEVICE=YOUR-SIMULATOR-UDID npm run test-app:mobile com.your.bundleid
  6. Open the report:

    open web/runs/my-app__explore__*/report.html

Option 2 — Powerful but Slower (Claude Code CLI)

Drives the app through your existing Claude Code subscription instead of an API key — no OpenRouter account needed, but slower per step than Option 1.

iOS tester (selfloop):

curl -fsSL https://raw.githubusercontent.com/Tej-Sharma/selfloop-ai-test-your-apps/development/install.sh | bash

Web tester (webbot):

curl -fsSL https://raw.githubusercontent.com/Tej-Sharma/selfloop-ai-test-your-apps/development/web/install.sh | bash

Then, in Claude Code:

test my ios app using the selfloop skill
test my web app using the webbot skill

📦 Prefer a hosted, always-on tester instead of running it locally? Get it here


See it in action

Real output from a run against a fitness-tracker app (planted bugs, used for validation):

$ npm run test-app:mobile

explore: google/gemini-3.5-flash · FitTrack (ai.beemo.fittrack)

step 0  [Welcome/Login Screen]  → tap 'Log In'
   Landed on the FitTrack welcome screen with an email/password form and a Log In CTA.
   flaw F-001 [low content] Spelling mistake in welcome message

step 4  [Home Dashboard]  → tap 'Start Workout'
   ✓ flow FL-001 done: "Start a workout, exercise, finish it, and verify on Home
     Dashboard" (steps 4–8)
   flaw F-002 [medium content] Placeholder 'TODO: load workouts' visible in UI
   flaw F-003 [low copy] Spelling mistake: 'Caloires'

================ EXPLORE SUMMARY ================
steps: 12/40   flaws: 3   screen: Home Dashboard
coverage: 9/14 controls across 6 screens   UNTRIED: Settings (2)
flows completed: "Start a workout, exercise, finish it, and verify on Home Dashboard" (5 steps)
run: v2-engine/mobile/runs/ai.beemo.fittrack/2026-07-07T15-54-59-053Z
================================================

Every one of those lines is backed by a saved screenshot and a JSON record on disk — nothing here is a description written after the fact.


What it does

Capability What it means
Single-brain explore loop One model call per turn both judges what just happened and picks the next action — not a separate "planner" and "actor"
Never-pruned memory Every prior step (action, expectation, result, reasoning) is fed back into every future turn — no lossy summarization
Coverage map, not vibes A deterministic state graph tracks every screen and control seen, so the tester knows what's still untried and won't quit early
Flow tracking The model declares a completed user journey ("signed up → onboarded → saw dashboard"); code enforces the rules (no overlaps, no padding a flow to hit a minimum)
Grounded flaws Every bug is logged with severity, a bounding box, and the screenshot that shows it — the HTML report is built directly from this file, nothing is re-summarized
Design QA (gated) When a Figma link is configured, screens are matched to design frames and checked against the frame's real element data — never a raw pixel diff
One core, two drivers The loop, memory format, and report pipeline are identical for iOS and web; only the platform "hands" differ

How it works

                    ┌─────────────────────────────────────────────┐
                    │        core/explore.mjs  (one brain)         │
                    │                                               │
   memory.jsonl ───▶│  every turn: judge last action + pick next   │◀─── state-graph.json
   (never pruned)   │  1 model call, full screenshot in the prompt │     (coverage map,
                    │                                               │      persists across runs)
                    └───────────────────┬───────────────────────────┘
                                        │ driver contract
                       ┌────────────────┴────────────────┐
                       ▼                                  ▼
              mobile/driver.mjs                    web/driver.mjs
              (mobile-mcp → simulator)        (Playwright MCP → browser)
              tap / type / swipe /            click / type / navigate(url) /
              relaunch app to backtrack        browser back is free
                       │                                  │
                       └────────────────┬─────────────────┘
                                        ▼
                    journal.jsonl · flaws.jsonl · flows.jsonl
                                        │
                                        ▼
                     critique pass → design pass (optional) → report.html

Everything platform-specific — how to observe a screen, how to execute a tap vs. a click, how to recover from a crash, how to backtrack — is isolated behind a driver contract in mobile/driver.mjs / web/driver.mjs. The exploration doctrine, prompts, memory format, and reporting all live once in core/ and are shared byte-for-byte between platforms.

The memory system: everything written to disk as it happens

Nothing relies on long context. Every artifact is appended incrementally — kill the run mid-way and the record up to the last step is still complete:

  • journal.jsonl — the step trace: screen, action, expectation, result, screenshot ref, crash flag.
  • memory.jsonl — the full reasoning trace fed back into the model every turn (the "never-pruned memory").
  • flaws.jsonl — every bug found, deduped, with severity and the screenshot(s) that prove it. The HTML report is built from this file.
  • flows.jsonl — completed user journeys the model declared and the code validated (no overlapping, no padding to hit a minimum length).
  • state-graph.json — the coverage map, persisted across runs: every screen seen, its controls, and which ones were actually tried.

Platform differences (same core, different hands)

iOS (selfloop) Web (webbot)
Driver mobile-mcp → iOS Simulator Playwright MCP → real browser
Actions tap, type, swipe, relaunch, speak click, type, navigate(url), back
Backtracking relaunch the app + re-walk a screen's saved tap-path (no URLs on iOS) navigate(url) — one call, instant
Error signal crash + visible-error detection (no console/network on native) crash + console errors + failed network requests

Prerequisites

iOS:

  • macOS with Xcode + command-line tools
  • An iPhone simulator runtime installed
  • Node.js 20+ (mobile-mcp runs via npx)
  • claude CLI (Claude Code) authenticated and on PATH

Web:

  • Node.js 20+ (Playwright MCP runs via npx)
  • claude CLI (Claude Code) authenticated and on PATH

Developing against the engine directly

The exploration engine (what actually powers both testers) is a standalone Node project — useful if you're iterating on the loop itself rather than going through the CLI or Claude Code:

cd v2-engine
npm install
npm run setup:web            # one-time: installs the Chromium build Playwright uses

npm run test-app:mobile      # explore → critique → design → annotate → report, iOS
npm run test-app:web         # same pipeline, web

# or run one stage at a time:
npm run explore:mobile
npm run critique:mobile
npm run report:mobile

npm run parity               # proves core/ + both drivers stay output-identical

Project layout

ios-tester/
├── v2-engine/                 ← the shared explore/critique/report engine
│   ├── core/                   explore loop, prompts, schemas, state graph — written once
│   ├── mobile/                 iOS driver (mobile-mcp) + platform files
│   └── web/                    web driver (Playwright MCP) + platform files
├── bin/selfloop                ← iOS CLI entry (bash) — installs, boots the sim, invokes Claude Code
├── web/bin/webbot              ← web CLI entry (bash), same shape as bin/selfloop
├── src/{lib,templates}         ← simctl/xcodebuild helpers, prompt templates (iOS CLI)
├── web/src/{lib,templates}     ← Playwright MCP plumbing, prompt templates (web CLI)
└── install.sh / web/install.sh ← curl-target installers

A run against a target app writes a runs/<bundle-or-target>/<timestamp>/ folder containing journal.jsonl, flaws.jsonl, flows.jsonl, memory.jsonl, state-graph.json, screenshots/, and report.html — open the last one.

Status

v2 engine (v2-engine/) is the actively developed core, already validated against apps with planted bugs. The bash CLIs (bin/selfloop, web/bin/webbot) are the current install path for Claude Code users; a hosted version is at selfloop.it.com.

About

Visual AI that automatically tests your apps' core flows and finds bugs. No need to torture yourself manually testing, just keep vibe coding forwards.

Resources

Stars

51 stars

Watchers

1 watching

Forks

Releases

No releases published

Packages