Skip to content

qodetest/qstack

BranchesTags

Repository files navigation

QStack

A TypeScript test automation framework that unifies Playwright for web testing and WebdriverIO/Appium for native mobile testing (iOS and Android) under a single codebase. Tests run locally or on BrowserStack real devices and browsers. An AI-powered self-healing engine (OpenAI, Anthropic, Google Gemini, or a local LLM) automatically fixes broken selectors at runtime so that UI changes don't immediately break your entire suite.

Table of Contents

Architecture

QStack uses a dual-runner architecture: each runner is best-in-class for its domain, and both share common infrastructure.

Layer Tool Why
Web testing Playwright Test Auto-wait, multi-browser, built-in assertions, trace viewer
Mobile testing WebdriverIO + Appium First-class TypeScript client, rich driver ecosystem, platform-aware selectors
Cloud execution BrowserStack Real browsers and physical devices, SDK integration for both runners
Self-healing OpenAI / Anthropic / Gemini / Local LLM LLM-driven selector repair with caching to avoid repeat API calls 
                        ┌──────────────────────────────────┐
                        │         QStack Framework         │
                        │                                  │
                        │  ┌────────┐  ┌────────────────┐  │
                        │  │ Common │  │   AI Healing    │  │
                        │  │ Utils  │  │ Engine + Cache  │  │
                        │  └────────┘  └────────────────┘  │
                        └──────┬───────────────┬───────────┘
                               │               │
                ┌──────────────┴──┐     ┌──────┴──────────────┐
                │  Web (Playwright)│     │ Mobile (WebdriverIO) │
                │                 │     │                      │
                │  Page Objects   │     │  Screen Objects       │
                │  Healing Fixture│     │  Healing Hooks        │
                │  *.spec.ts      │     │  Gesture Helpers      │
                └────────┬────────┘     │  *.test.ts            │
                         │              └──────────┬───────────┘
                         │                         │
              ┌──────────┴─────────────────────────┴──────────┐
              │              Execution Targets                 │
              │                                                │
              │   Local Browsers / Emulators    BrowserStack   │
              └────────────────────────────────────────────────┘

Prerequisites

Global (required for all test types)

  • Node.js >= 20 and npm (download)
  • Git
  • Clone the repo, run npm install, and copy .env.example to .env

Web Testing -- Local

Everything above, plus:

npx playwright install --with-deps

This downloads Chromium, Firefox, and WebKit binaries plus OS-level dependencies.

Web Testing -- BrowserStack

Everything in Global, plus:

  • A BrowserStack Automate account
  • Set BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY in .env
  • The browser/OS matrix is configured in browserstack.yml (already included)

Mobile Testing -- Local

Everything in Global, plus:

  • Appium installed globally: 
    npm install -g appium
  • Platform drivers: 
    # Android
appium driver install uiautomator2

# iOS
appium driver install xcuitest
  • Android: Android SDK, ANDROID_HOME set, an emulator or device connected (adb devices shows it)
  • iOS: Xcode with command-line tools, a booted simulator (xcrun simctl list devices | grep Booted)
  • Your app (.apk / .ipa) installed on the emulator/simulator, or the appium:app capability set in config/wdio.local.config.ts

Mobile Testing -- BrowserStack

Everything in Global, plus:

  • A BrowserStack App Automate account
  • Upload your app to BrowserStack: 
    curl -u "$BROWSERSTACK_USERNAME:$BROWSERSTACK_ACCESS_KEY" \
  -X POST "https://api-cloud.browserstack.com/app-automate/upload" \
  -F "file=@/path/to/app.apk"
  • Set the returned bs:// hash in .env as BS_ANDROID_APP and/or BS_IOS_APP
  • Set BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY in .env

Self-Healing

Everything needed for the test type you're running (web or mobile), plus:

  • Set SELF_HEALING_ENABLED=true in .env (or pass it as an env var inline)
  • Set AI_PROVIDER to one of: openai, anthropic, gemini, or local
  • Set the corresponding credentials:
    • OpenAI: OPENAI_API_KEY
    • Anthropic: ANTHROPIC_API_KEY
    • Gemini: GEMINI_API_KEY
    • Local LLM: LOCAL_LLM_BASE_URL, LOCAL_LLM_MODEL, and optionally LOCAL_LLM_API_KEY (see below)

Quick Start

# 1. Clone and install
git clone <repo-url> qstack && cd qstack
npm install

# 2. Configure environment
cp .env.example .env
# Edit .env with your keys (only BASE_URL is needed for local web tests)

# 3. Install Playwright browsers
npx playwright install --with-deps

# 4. Run web tests locally
npm run test:web

Project Structure

qstack/
  package.json                         # Scripts, dependencies
  tsconfig.json                        # TypeScript config (strict, ES2022)
  .env.example                         # Environment variable template
  browserstack.yml                     # BrowserStack SDK browser/OS matrix

  config/
    playwright.config.ts               # Playwright local: 3 browsers, HTML + JUnit reporters
    playwright.browserstack.config.ts  # Playwright on BrowserStack via SDK wrapper
    wdio.local.config.ts               # WDIO local: Appium service auto-start, Mocha framework
    wdio.browserstack.config.ts        # WDIO on BrowserStack: real devices, Galaxy S24 + iPhone 15 Pro

  src/
    web/
      pages/                           # Page Object Model (extend BasePage)
        BasePage.ts                    #   Abstract base: navigate, locator, getByRole, getByText, etc.
        LoginPage.ts                   #   Example: login form interactions
      tests/                           # Playwright test specs (*.spec.ts)
        login.spec.ts                  #   Example: valid login, invalid login, logout
      fixtures/
        healing.fixture.ts             #   Self-healing Playwright fixture (healingPage)

    mobile/
      screens/                         # Screen Object Model (extend BaseScreen)
        BaseScreen.ts                  #   Abstract base: platform-aware selectors, tap, type, getText
        LoginScreen.ts                 #   Example: login form for Android + iOS
      tests/                           # WDIO test specs (*.test.ts, Mocha syntax)
        login.test.ts                  #   Example: valid login, invalid login
      helpers/
        gestures.ts                    #   Touch helpers: swipeUp, swipeDown, scrollToElement, tap
        healing.hooks.ts               #   Self-healing WDIO hooks + healingFind() helper

    common/
      utils/
        env.ts                         # Loads .env, exports typed config object
        logger.ts                      # Structured logging via pino
        retry.ts                       # Generic retry with exponential backoff
      types/
        index.ts                       # Shared types: Platform, ExecutionTarget, capabilities

    healing/
      healer.ts                        # Core orchestrator: cache check -> AI fallback -> record results
      cache.ts                         # JSON file cache (healing-cache.json) for healed selectors
      analyzer.ts                      # Selector type inference, HTML/XML cleaning, context truncation
      types.ts                         # HealingContext, SelectorSuggestion, HealingResult, etc.
      providers/
        base.ts                        # Abstract provider: prompt builder, JSON response parser
        openai.ts                      # OpenAI GPT-4o adapter
        anthropic.ts                   # Anthropic Claude adapter
        gemini.ts                      # Google Gemini adapter
        local.ts                       # Local/self-hosted LLM adapter (OpenAI-compatible API)

  .github/workflows/
    web-tests.yml                      # Web CI: push/PR/dispatch/repository_dispatch
    mobile-tests.yml                   # Mobile CI: dispatch/repository_dispatch (BrowserStack only)
    reusable-test.yml                  # Reusable workflow: callable from other repos via workflow_call

  reports/                             # .gitignored -- local test reports, JUnit XML, healing JSON

Writing Tests

1. Web Tests (Playwright)

Web tests follow the Page Object Model pattern. Every page you test gets a class that extends BasePage.

Step 1: Create a Page Object

Create a new file in src/web/pages/. Your class must extend BasePage and define a path property (the URL path relative to BASE_URL).

BasePage API (methods available to all page objects):

Method Description
navigate() Navigates to this.path
waitForPageLoad() Waits for domcontentloaded
getTitle() Returns the page title
screenshot(name) Saves a full-page screenshot to reports/
locator(selector) Returns a Playwright Locator
getByRole(role, options?) Finds by ARIA role
getByText(text, options?) Finds by visible text
getByTestId(testId) Finds by data-testid attribute

Example -- src/web/pages/LoginPage.ts:

import type { Page } from "@playwright/test";
import { BasePage } from "./BasePage.js";

export class LoginPage extends BasePage {
  protected readonly path = "/login";

  private readonly usernameInput = "#username";
  private readonly passwordInput = "#password";
  private readonly loginButton = 'button[type="submit"]';
  private readonly flashMessage = "#flash";
  private readonly logoutButton = 'a[href="/logout"]';

  constructor(page: Page) {
    super(page);
  }

  async login(username: string, password: string): Promise<void> {
    await this.locator(this.usernameInput).fill(username);
    await this.locator(this.passwordInput).fill(password);
    await this.locator(this.loginButton).click();
  }

  async getFlashMessage(): Promise<string> {
    const text = await this.locator(this.flashMessage).innerText();
    return text.trim();
  }

  async isLoggedIn(): Promise<boolean> {
    return this.locator(this.logoutButton).isVisible();
  }

  async logout(): Promise<void> {
    await this.locator(this.logoutButton).click();
  }
}

Key points:

  • Store selectors as private readonly properties at the top of the class.
  • Each public method represents a user-facing action or assertion query.
  • The constructor accepts a Playwright Page and passes it to BasePage.

Step 2: Write a Test Spec

Create a .spec.ts file in src/web/tests/. Import test and expect from @playwright/test, and import your page object.

Example -- src/web/tests/login.spec.ts:

import { test, expect } from "@playwright/test";
import { LoginPage } from "../pages/LoginPage.js";

test.describe("Login Page", () => {
  let loginPage: LoginPage;

  test.beforeEach(async ({ page }) => {
    loginPage = new LoginPage(page);
    await loginPage.navigate();
  });

  test("should login with valid credentials", async () => {
    await loginPage.login("tomsmith", "SuperSecretPassword!");
    const flash = await loginPage.getFlashMessage();
    expect(flash).toContain("You logged into a secure area!");
    expect(await loginPage.isLoggedIn()).toBe(true);
  });

  test("should show error for invalid credentials", async () => {
    await loginPage.login("invaliduser", "wrongpassword");
    const flash = await loginPage.getFlashMessage();
    expect(flash).toContain("Your username is invalid!");
  });

  test("should logout successfully", async () => {
    await loginPage.login("tomsmith", "SuperSecretPassword!");
    await loginPage.logout();
    const flash = await loginPage.getFlashMessage();
    expect(flash).toContain("You logged out of the secure area!");
  });
});

Pattern summary:

  • test.describe groups related tests.
  • test.beforeEach instantiates the page object with the Playwright page fixture and navigates.
  • Each test() calls page object methods and asserts with expect.

Step 3: File Placement

Place specs in src/web/tests/ with a .spec.ts extension. The Playwright config (config/playwright.config.ts) automatically picks up all *.spec.ts files in that directory. No additional registration is needed.

2. Web Tests with Self-Healing

Self-healing web tests use a custom Playwright fixture that intercepts locator actions. If a selector fails, the framework captures the page DOM, asks the AI for alternative selectors, and retries transparently.

Step 1: Import from the Healing Fixture

Replace your @playwright/test import with the healing fixture:

// Standard test (no healing):
// import { test, expect } from "@playwright/test";

// Self-healing test:
import { test, expect } from "../fixtures/healing.fixture.js";

Step 2: Use healingPage Instead of page

The healing fixture provides a healingPage fixture that wraps the standard Playwright page with a Proxy. Use it exactly like page:

import { test, expect } from "../fixtures/healing.fixture.js";

test("login with self-healing", async ({ healingPage }) => {
  await healingPage.goto("/login");

  // If "#username" breaks (e.g., renamed to "#user-email"), the AI will:
  //   1. Capture the current DOM
  //   2. Ask OpenAI/Anthropic for alternative selectors
  //   3. Try each suggestion until one works
  //   4. Cache the working selector for future runs
  await healingPage.locator("#username").fill("tomsmith");
  await healingPage.locator("#password").fill("SuperSecretPassword!");
  await healingPage.locator('button[type="submit"]').click();

  await expect(healingPage.locator("#flash")).toContainText("You logged into a secure area!");
});

Using Page Objects with Healing

Page objects that accept a Page in their constructor work with healingPage since the proxy is fully Page-compatible:

import { test, expect } from "../fixtures/healing.fixture.js";
import { LoginPage } from "../pages/LoginPage.js";

test("page object with healing", async ({ healingPage }) => {
  const loginPage = new LoginPage(healingPage);
  await loginPage.navigate();
  await loginPage.login("tomsmith", "SuperSecretPassword!");
  expect(await loginPage.isLoggedIn()).toBe(true);
});

The healing is transparent -- BasePage.locator() calls this.page.locator(), which is intercepted by the proxy.

What Gets Healed

The following locator actions are wrapped with healing: click, fill, type, check, uncheck, selectOption, innerText, textContent, isVisible, isEnabled. If any of these fail because the element cannot be found, the healing engine activates.

3. Mobile Tests (WebdriverIO/Appium)

Mobile tests follow the Screen Object Model pattern. Each screen of your app gets a class that extends BaseScreen. Tests use Mocha syntax (describe/it).

Step 1: Create a Screen Object

Create a new file in src/mobile/screens/. Your class extends BaseScreen, which provides platform-aware element interaction.

BaseScreen API (methods available to all screen objects):

Method Description
selector(android, ios) Returns the correct selector for the current platform
findElement(android, ios) Finds and waits for an element (10s timeout)
tapElement(android, ios) Taps an element
typeText(android, ios, text) Clears and types text into an element
getElementText(android, ios) Returns an element's text content
isElementDisplayed(android, ios) Returns true/false without throwing
waitForScreenReady(android, ios) Waits up to 15s for an element to appear
screenshot(name) Saves a screenshot to reports/
isAndroid / isIOS Platform detection getters

Every interaction method takes two selectors -- one for Android, one for iOS. This lets a single test file run on both platforms. When selectors are identical across platforms (common with accessibility IDs), you pass the same value twice.

Example -- src/mobile/screens/LoginScreen.ts:

import { BaseScreen } from "./BaseScreen.js";

export class LoginScreen extends BaseScreen {
  private readonly usernameField = {
    android: '~username_input',
    ios: '~username_input',
  };

  private readonly passwordField = {
    android: '~password_input',
    ios: '~password_input',
  };

  private readonly loginButton = {
    android: '~login_button',
    ios: '~login_button',
  };

  private readonly errorMessage = {
    android: '~error_message',
    ios: '~error_message',
  };

  private readonly welcomeMessage = {
    android: '~welcome_message',
    ios: '~welcome_message',
  };

  async waitForScreen(): Promise<void> {
    await this.waitForScreenReady(this.usernameField.android, this.usernameField.ios);
  }

  async login(username: string, password: string): Promise<void> {
    await this.typeText(this.usernameField.android, this.usernameField.ios, username);
    await this.typeText(this.passwordField.android, this.passwordField.ios, password);
    await this.tapElement(this.loginButton.android, this.loginButton.ios);
  }

  async getErrorMessage(): Promise<string> {
    return this.getElementText(this.errorMessage.android, this.errorMessage.ios);
  }

  async isWelcomeDisplayed(): Promise<boolean> {
    return this.isElementDisplayed(this.welcomeMessage.android, this.welcomeMessage.ios);
  }
}

Key points:

  • Store selectors as objects with android and ios keys.
  • The ~ prefix is Appium shorthand for accessibility ID selectors.
  • For platform-specific XPath or resource-id selectors, provide different values per key.

Step 2: Write a Test

Create a .test.ts file in src/mobile/tests/. Use Mocha syntax (describe, it, before, beforeEach). WebdriverIO globals (browser, $, expect) are available automatically.

Example -- src/mobile/tests/login.test.ts:

import { LoginScreen } from "../screens/LoginScreen.js";

describe("Login Screen", () => {
  let loginScreen: LoginScreen;

  before(async () => {
    loginScreen = new LoginScreen();
  });

  beforeEach(async () => {
    await loginScreen.waitForScreen();
  });

  it("should login with valid credentials", async () => {
    await loginScreen.login("testuser", "password123");
    const isWelcome = await loginScreen.isWelcomeDisplayed();
    expect(isWelcome).toBe(true);
  });

  it("should show error for invalid credentials", async () => {
    await loginScreen.login("invalid", "wrong");
    const error = await loginScreen.getErrorMessage();
    expect(error).toContain("Invalid");
  });
});

Pattern summary:

  • before (runs once): instantiate screen objects.
  • beforeEach (runs per test): wait for the screen to be ready.
  • it: interact via screen object methods, assert with expect.

Step 3: Gesture Helpers

Import touch gesture utilities from src/mobile/helpers/gestures.ts when you need scrolling or custom taps:

import { swipeUp, swipeDown, scrollToElement, tapCoordinates } from "../helpers/gestures.js";

it("should scroll to find the submit button", async () => {
  const element = await scrollToElement("~submit_button", 5); // max 5 swipes
  await element.click();
});

it("should swipe through a carousel", async () => {
  await swipeUp();                           // Default: center of screen, 80% -> 20% Y
  await swipeDown({ duration: 500 });        // Slower swipe
  await tapCoordinates(200, 400);            // Tap exact coordinates
});
Function Description
swipeUp(options?) Swipe from bottom to top. Configurable start/end coordinates and duration.
swipeDown(options?) Swipe from top to bottom.
scrollToElement(selector, maxScrolls?) Repeatedly swipes up until the element is visible, returns the element.
tapCoordinates(x, y) Taps at exact screen coordinates.

Step 4: File Placement

Place specs in src/mobile/tests/ with a .test.ts extension. Both WDIO configs (wdio.local.config.ts and wdio.browserstack.config.ts) automatically pick up all *.test.ts files in that directory.

4. Mobile Tests with Self-Healing

Mobile self-healing uses a helper function, healingFind, that wraps element interactions with the AI healing engine.

Using healingFind

Import healingFind from the healing hooks module. It replaces direct $() calls with a healing-aware wrapper:

import { healingFind } from "../helpers/healing.hooks.js";

describe("Login Screen (self-healing)", () => {
  const testFile = "login.test.ts";

  it("should tap login button with healing", async () => {
    // If "~login_button" breaks, the AI will analyze the page source
    // and suggest alternative selectors
    await healingFind("~login_button", "click", "should tap login button", testFile);
  });

  it("should type username with healing", async () => {
    await healingFind("~username_input", "setValue", "should type username", testFile, ["testuser"]);
  });

  it("should read welcome text with healing", async () => {
    const text = await healingFind("~welcome_message", "getText", "should read welcome", testFile);
    expect(text).toContain("Welcome");
  });
});

healingFind API:

Parameter Type Description
selector string The selector to find (e.g., ~accessibility_id, CSS, XPath)
action "click" | "setValue" | "getText" | "isDisplayed" The action to perform on the element
testName string Current test name (used for logging and cache keys)
testFile string Current test file name
actionArgs? string[] Arguments for the action (e.g., text to type for setValue)

Wiring Hooks into WDIO Config

To enable automatic failure capture and healing reports, add the hooks from healing.hooks.ts to your WDIO config:

// In config/wdio.local.config.ts or config/wdio.browserstack.config.ts
import { afterTest, onComplete } from "../src/mobile/helpers/healing.hooks.js";

export const config: WebdriverIO.Config = {
  // ... existing config ...
  afterTest,
  onComplete,
};

The afterTest hook captures page source when a test fails due to an element error. The onComplete hook writes the healing report JSON after all tests finish.

Running Tests

Quick Reference

Command What it does Prerequisites
npm run test:web Web tests, all browsers, locally Playwright browsers
npm run test:web:bs Web tests on BrowserStack BS credentials
npm run test:web:heal Web tests with AI self-healing Playwright browsers + AI key
npm run test:mobile:android Android tests on local emulator Appium + Android SDK + emulator
npm run test:mobile:ios iOS tests on local simulator Appium + Xcode + simulator
npm run test:mobile:bs Mobile tests on BrowserStack BS credentials + uploaded app
npm run test:mobile:heal Mobile tests with AI self-healing Appium setup + AI key
npm run test:heal Web + mobile with healing All of the above
npm run typecheck TypeScript compilation check None
npm run lint ESLint None

Local Web Tests

Run all browsers (Chromium, Firefox, WebKit):

npm run test:web

Run a single browser:

npx playwright test --config=config/playwright.config.ts --project=chromium
npx playwright test --config=config/playwright.config.ts --project=firefox
npx playwright test --config=config/playwright.config.ts --project=webkit

Run a single test file:

npx playwright test --config=config/playwright.config.ts src/web/tests/login.spec.ts

Run tests matching a name pattern:

npx playwright test --config=config/playwright.config.ts --grep "should login"

Run in headed mode (see the browser):

npx playwright test --config=config/playwright.config.ts --headed

Run in debug mode (step through with Playwright Inspector):

npx playwright test --config=config/playwright.config.ts --debug

Override the base URL:

BASE_URL=https://staging.example.com npm run test:web

BrowserStack Web Tests

npm run test:web:bs

This uses browserstack-node-sdk to run Playwright tests across the browser/OS matrix defined in browserstack.yml. The SDK handles connecting to BrowserStack, launching browsers, and collecting results.

To customize the matrix, edit browserstack.yml:

platforms:
  - os: Windows
    osVersion: 11
    browserName: chrome
    browserVersion: latest
  - os: OS X
    osVersion: Sonoma
    browserName: safari
    browserVersion: latest

Local Mobile Tests

Android (requires running emulator):

npm run test:mobile:android

iOS (requires booted simulator):

npm run test:mobile:ios

Both commands use config/wdio.local.config.ts, which auto-starts an Appium server via @wdio/appium-service. You need to configure device capabilities in the config file before running. Add capabilities to the capabilities array:

capabilities: [
  {
    platformName: "Android",
    "appium:deviceName": "Pixel_7_API_34",
    "appium:platformVersion": "14",
    "appium:automationName": "UiAutomator2",
    "appium:app": "/path/to/your/app.apk",
  },
],

BrowserStack Mobile Tests

npm run test:mobile:bs

This runs on real devices via @wdio/browserstack-service. The default config targets Samsung Galaxy S24 (Android 14) and iPhone 15 Pro (iOS 17). Device capabilities are configured in config/wdio.browserstack.config.ts.

Self-Healing Mode

Web tests with healing:

npm run test:web:heal
# or inline:
SELF_HEALING_ENABLED=true npm run test:web

Mobile tests with healing:

npm run test:mobile:heal
# or inline:
SELF_HEALING_ENABLED=true npm run test:mobile:android

Both web and mobile with healing:

npm run test:heal

Utility Commands

Type-check all TypeScript without running tests:

npm run typecheck

Lint all source and config files:

npm run lint

AI Self-Healing Deep Dive

How It Works

The self-healing engine activates when a locator action fails (element not found). It follows this flow:

  Test action fails (selector not found)
           │
           ▼
  ┌─────────────────────┐
  │ Check healing cache  │
  │ (healing-cache.json) │
  └────────┬────────────┘
           │
     ┌─────┴──────┐
     │ Cache hit?  │
     └─────┬──────┘
       Yes │        No
           │         │
           ▼         ▼
  ┌──────────┐  ┌──────────────────────┐
  │ Try      │  │ Capture page context │
  │ cached   │  │ (DOM / page source)  │
  │ selector │  └──────────┬───────────┘
  └────┬─────┘             │
       │                   ▼
       │         ┌───────────────────┐
       │         │ Send to AI provider│
       │         │                   │
       │         └────────┬──────────┘
       │                  │
       │                  ▼
       │         ┌──────────────────────┐
       │         │ Receive 3-5 ranked   │
       │         │ selector suggestions │
       │         └────────┬─────────────┘
       │                  │
       ▼                  ▼
  ┌──────────────────────────┐
  │ Try each suggestion      │
  │ until one works          │
  └─────────┬────────────────┘
            │
      ┌─────┴──────┐
      │ Healed?    │
      └─────┬──────┘
        Yes │       No
            │        │
            ▼        ▼
     ┌──────────┐  ┌────────────────┐
     │ Cache +  │  │ Fail test with │
     │ Log +    │  │ detailed error │
     │ Continue │  └────────────────┘
     └──────────┘

What the AI Receives

The prompt sent to the AI includes:

  • The failed selector and its type (CSS, XPath, accessibility ID, etc.)
  • The action that was attempted (click, fill, getText, etc.)
  • The test name and file
  • The platform (web, android, ios)
  • The error message
  • A cleaned and truncated page context (~6,000 characters of DOM HTML or native XML page source)

The AI responds with a JSON array of 3-5 alternative selectors, each with a confidence score and reasoning.

Caching

Successful healings are persisted to healing-cache.json at the project root. On subsequent runs, the cache is checked first to avoid redundant API calls. Cache entries are keyed by {testFile, originalSelector} and include a hit counter.

Supported Providers

Provider Model Config
OpenAI GPT-4o AI_PROVIDER=openai + OPENAI_API_KEY
Anthropic Claude Sonnet AI_PROVIDER=anthropic + ANTHROPIC_API_KEY
Google Gemini Gemini 2.5 Flash AI_PROVIDER=gemini + GEMINI_API_KEY
Local LLM Any (configurable) AI_PROVIDER=local + LOCAL_LLM_BASE_URL + LOCAL_LLM_MODEL

Local LLM Setup

The local provider works with any server that exposes an OpenAI-compatible /v1/chat/completions endpoint. Common options:

Ollama (default):

# Install and start Ollama, then pull a model
ollama pull llama3

# QStack defaults point to Ollama already:
# LOCAL_LLM_BASE_URL=http://localhost:11434
# LOCAL_LLM_MODEL=llama3

LM Studio:

# Start LM Studio's local server, then set:
# LOCAL_LLM_BASE_URL=http://localhost:1234
# LOCAL_LLM_MODEL=<model-name-from-lm-studio>

vLLM / llama.cpp server / LocalAI:

# Point to your server's address and model name:
# LOCAL_LLM_BASE_URL=http://localhost:8000
# LOCAL_LLM_MODEL=<your-model>
# LOCAL_LLM_API_KEY=<if required>

The local provider sends no data to external services -- all inference runs on your machine.

CI/CD (GitHub Actions)

Workflows

Workflow File Triggers Description
Web Tests .github/workflows/web-tests.yml push, pull_request, workflow_dispatch, repository_dispatch Runs web tests locally (with Playwright browsers) or on BrowserStack
Mobile Tests .github/workflows/mobile-tests.yml workflow_dispatch, repository_dispatch Runs mobile tests on BrowserStack (local mobile in CI is impractical)
Reusable .github/workflows/reusable-test.yml workflow_call Callable from other repos; supports both web and mobile

Required Repository Secrets

Configure these in Settings > Secrets and variables > Actions in your GitHub repo:

Secret Required for
BROWSERSTACK_USERNAME BrowserStack runs
BROWSERSTACK_ACCESS_KEY BrowserStack runs
OPENAI_API_KEY Self-healing with OpenAI
ANTHROPIC_API_KEY Self-healing with Anthropic
GEMINI_API_KEY Self-healing with Gemini
AI_PROVIDER Self-healing (value: openai, anthropic, gemini, or local)
BS_ANDROID_APP BrowserStack mobile (Android bs:// hash)
BS_IOS_APP BrowserStack mobile (iOS bs:// hash)

Manual Dispatch (from GitHub UI)

Both web-tests.yml and mobile-tests.yml support workflow_dispatch with input parameters. Go to Actions > [workflow name] > Run workflow and select:

  • Web: environment (local/browserstack), browser (chromium/firefox/webkit), self-healing toggle
  • Mobile: platform (android/ios/both), self-healing toggle

Triggering from Another Repo -- Repository Dispatch

Send an HTTP request to trigger tests from any system (CI pipeline, script, another repo):

Trigger web tests:

curl -X POST https://api.github.com/repos/OWNER/qstack/dispatches \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  -H "Accept: application/vnd.github.v3+json" \
  -d '{
    "event_type": "run-web-tests",
    "client_payload": {
      "environment": "browserstack",
      "self_healing": "true",
      "base_url": "https://staging.myapp.com"
    }
  }'

Trigger mobile tests:

curl -X POST https://api.github.com/repos/OWNER/qstack/dispatches \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  -H "Accept: application/vnd.github.v3+json" \
  -d '{
    "event_type": "run-mobile-tests",
    "client_payload": {
      "self_healing": "false"
    }
  }'

client_payload options:

Key Values Default Used by
environment local, browserstack local web-tests
self_healing true, false false both
base_url Any URL https://the-internet.herokuapp.com web-tests

Triggering from Another Repo -- Reusable Workflow

In another repository's workflow file, call QStack's reusable workflow:

name: Run QStack Tests
on:
  push:
    branches: [main]

jobs:
  web-tests:
    uses: OWNER/qstack/.github/workflows/reusable-test.yml@main
    with:
      test_type: web
      environment: browserstack
      self_healing: "true"
      base_url: "https://staging.myapp.com"
      browser: chromium
    secrets:
      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}

  mobile-tests:
    uses: OWNER/qstack/.github/workflows/reusable-test.yml@main
    with:
      test_type: mobile
    secrets:
      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
      BS_ANDROID_APP: ${{ secrets.BS_ANDROID_APP }}
      BS_IOS_APP: ${{ secrets.BS_IOS_APP }}

Reusable workflow inputs:

Input Type Required Default Description
test_type string Yes -- web or mobile
environment string No local local or browserstack
self_healing string No false true or false
browser string No chromium chromium, firefox, or webkit (web only)
base_url string No https://the-internet.herokuapp.com Base URL for web tests

Environment Variables Reference

Variable Required Default Used By
BROWSERSTACK_USERNAME For BrowserStack runs -- Web BS, Mobile BS, CI
BROWSERSTACK_ACCESS_KEY For BrowserStack runs -- Web BS, Mobile BS, CI
BS_ANDROID_APP For BS mobile (Android) -- Mobile BS
BS_IOS_APP For BS mobile (iOS) -- Mobile BS
AI_PROVIDER For self-healing openai Healing engine
OPENAI_API_KEY If AI_PROVIDER=openai -- Healing engine
ANTHROPIC_API_KEY If AI_PROVIDER=anthropic -- Healing engine
GEMINI_API_KEY If AI_PROVIDER=gemini -- Healing engine
LOCAL_LLM_BASE_URL If AI_PROVIDER=local http://localhost:11434 Healing engine
LOCAL_LLM_MODEL If AI_PROVIDER=local llama3 Healing engine
LOCAL_LLM_API_KEY No not-needed Healing engine (if server requires auth)
SELF_HEALING_ENABLED No false Healing engine
BASE_URL No https://the-internet.herokuapp.com Web tests
APPIUM_HOST No 127.0.0.1 Local mobile tests
APPIUM_PORT No 4723 Local mobile tests
LOG_LEVEL No info All (pino logger)

Reports and Artifacts

Playwright HTML Report

After web tests run, an interactive HTML report is generated:

# Open the report in your browser
npx playwright show-report playwright-report

Location: playwright-report/index.html

JUnit XML Reports

For CI integration (GitHub Actions, Jenkins, etc.), JUnit XML files are generated:

Report Location
Web tests (local) reports/web-results.xml
Web tests (BrowserStack) reports/web-bs-results.xml
Mobile tests (local) reports/mobile-local-results.xml
Mobile tests (BrowserStack) reports/mobile-bs-results.xml

Healing Report

When self-healing is enabled, a JSON report is written after each run:

Location: reports/healing-report.json

Structure:

{
  "runTimestamp": "2026-04-15T10:30:00.000Z",
  "totalHealed": 2,
  "totalFailed": 1,
  "entries": [
    {
      "originalSelector": "#old-button",
      "healedSelector": "[data-action='submit']",
      "selectorType": "css",
      "confidence": 0.92,
      "reasoning": "Button with submit action attribute matches the original intent",
      "timestamp": "2026-04-15T10:30:05.000Z",
      "testFile": "src/web/tests/login.spec.ts",
      "testName": "should login with valid credentials",
      "platform": "web"
    }
  ]
}

BrowserStack Dashboard

BrowserStack test results, videos, logs, and screenshots are available on the BrowserStack dashboard:

Session links are printed in the test output when running with BrowserStack services.

CI Artifacts

In GitHub Actions, the following artifacts are uploaded after every run:

Artifact Contents
web-test-report playwright-report/ + reports/
mobile-test-report reports/ (JUnit XML)
healing-report-web reports/healing-report.json (if healing was enabled)
healing-report-mobile reports/healing-report.json (if healing was enabled)

Artifacts are retained for 14 days and downloadable from the Actions run page.

License

MIT

About

Test automation framework with Playwright, Appium, BrowserStack, and AI self-healing

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages