🧪 Agentic Tester

AI-driven automated E2E test generation & execution platform powered by Gemini, Playwright, and Firebase.

Fetch test cases from Firebase Firestore, select which ones to run through an interactive TUI, and execute them with a Playwright-based AI agent — all from your terminal.

---

📋 Table of Contents

• Features
• Architecture
• Prerequisites
• Installation
• Configuration
• Usage
  • Interactive TUI (Recommended)
  • CLI — Execute from Firebase
  • CLI — Execute from Local File
  • API Server
• Firebase Setup
• Project Structure
• API Reference

---

✨ Features

• Firebase Integration — Fetch test cases from Firestore, persist results back
• Interactive TUI — Visual test case selection with checkboxes, live execution progress
• AI-Powered Execution — Gemini 2.5 Flash-Lite + Playwright MCP for intelligent browser automation
• Multiple Modes — TUI, CLI, and REST API
• Live Feedback — Real-time pass/fail status during execution
• Screenshot Capture — Automatic screenshots uploaded to Firebase Storage

---

🏗 Architecture

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│  TUI (Rich/  │  │  CLI         │  │  REST API    │
│  Textual)    │  │  (argparse)  │  │  (FastAPI)   │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                  │
       └─────────────────┼──────────────────┘
                         │
              ┌──────────┴──────────┐
              │  Test Executor      │
              │  Agent (LangGraph + │
              │  Gemini + Playwright│
              │  MCP)               │
              └──────────┬──────────┘
                         │
              ┌──────────┴──────────┐
              │  Firebase (Firestore│
              │  + Storage)         │
              └─────────────────────┘

---

📦 Prerequisites

• Python 3.12+
• Node.js 18+ (for Playwright MCP server)
• Firebase Project with Firestore enabled
• Google API Key for Gemini

---

🔧 Installation

# Clone the repo
git clone https://github.com/your-username/Agentic_Tester.git
cd Agentic_Tester

# Create virtual environment (recommended)
python -m venv venv
venv\Scripts\activate       # Windows
# source venv/bin/activate  # macOS/Linux

# Install dependencies
pip install -r requirements.txt

---

⚙ Configuration

1. Create .env file

Copy the example and fill in your values:

cp .env.example .env

Edit .env:

# Gemini
GOOGLE_API_KEY=your-gemini-api-key

# Firebase
FIREBASE_PROJECT_ID=agentic-tester-ded1d
FIREBASE_API_KEY=your-firebase-api-key
FIREBASE_CREDENTIALS_PATH=./firebase-credentials.json

# Target
TARGET_URL=https://example.com

2. Add Firebase Credentials

1. Go to Firebase Console → Project Settings → Service Accounts
2. Click Generate new private key
3. Save the JSON file as firebase-credentials.json in the project root

---

🚀 Usage

1. Interactive TUI (Recommended)

Launch the visual terminal interface:

python main.py tui

How it works:

1. Enter your Project ID and Target URL
2. Click Fetch (or press F) to load test cases from Firebase
3. A loading spinner shows while fetching
4. Test cases appear as a checkbox list — select/deselect what to run
5. Click Execute Selected (or press R) to run the tests
6. Watch live results as each test completes

Keyboard shortcuts:

Key	Action
F	Fetch test cases from Firebase
A	Select all test cases
D	Deselect all test cases
R	Run selected tests
Q	Quit

---

2. CLI — Execute from Firebase

Fetch and run all test cases for a project:

python main.py execute --from-firebase --project demo-project

With a specific suite:

python main.py execute --from-firebase --project demo-project --suite-id suite-001

Override target URL:

python main.py execute --from-firebase --project demo-project --url https://mysite.com

---

3. CLI — Execute from Local File

Run tests from a local JSON file:

python main.py execute --test-file sample_test_cases.json --url https://example.com

---

4. API Server

Start the FastAPI server:

python main.py serve --port 8000

Then use the API endpoints (see API Reference below).

---

🔥 Firebase Setup

Firestore Collections

Your Firestore database should have a test_cases collection. Each document should follow this schema:

{
  "id": "tc-001",
  "project_id": "demo-project",
  "title": "Verify Homepage Loads",
  "description": "Navigate to homepage and verify title",
  "steps": [
    {
      "action": "navigate",
      "value": "https://example.com",
      "description": "Navigate to the homepage"
    },
    {
      "action": "assert",
      "expected": "Example Domain",
      "description": "Verify the page title"
    }
  ],
  "expected_result": "Homepage loads correctly",
  "priority": "high",
  "category": "functional"
}

Supported fields:

Field	Type	Required	Description
id	string	Yes	Unique test case ID
project_id	string	Yes	Project identifier (used for filtering)
title	string	Yes	Test case name
description	string	No	Detailed description
steps	array	No	Ordered list of test steps
expected_result	string	No	Expected outcome
priority	string	No	high, medium, or low
category	string	No	functional, boundary, negative, ui, or api

Test Step Schema

Field	Type	Description
action	string	navigate, click, fill, assert, snapshot
selector	string	CSS/XPath selector for target element
value	string	Input value or navigation URL
expected	string	Expected value for assertions
description	string	Human-readable description

---

📁 Project Structure

Agentic_Tester/
├── main.py                        # Entry point (CLI, TUI, Server)
├── requirements.txt               # Python dependencies
├── .env                           # Environment variables (gitignored)
├── .env.example                   # Environment template
├── firebase-credentials.json      # Firebase service account (gitignored)
├── sample_test_cases.json         # Example test cases (local file)
│
├── src/
│   ├── config.py                  # Settings from .env
│   │
│   ├── tui/
│   │   └── tui.py                 # Interactive Textual TUI
│   │
│   ├── executor/
│   │   ├── agent.py               # Test executor (LangGraph + Gemini)
│   │   ├── graph.py               # LangGraph execution graph
│   │   ├── prompts.py             # Execution prompts
│   │   ├── mcp_config.py          # Playwright MCP config
│   │   ├── state.py               # Graph state model
│   │   └── tools/                 # Custom assertion tools
│   │
│   ├── models/
│   │   ├── test_case.py           # TestCase, TestSuite models
│   │   └── execution_result.py    # ExecutionResult, Summary models
│   │
│   ├── storage/
│   │   └── firebase_client.py     # Firebase read/write operations
│   │
│   └── api/
│       ├── app.py                 # FastAPI app setup
│       └── routes/
│           └── execution.py       # API endpoints
│
└── tests/                         # Test suite

---

📡 API Reference

Method	Endpoint	Description
GET	/api/health	Health check
GET	/api/projects/{id}/test-cases	Fetch test cases from Firebase
POST	/api/projects/{id}/execute-test	Execute a single test case
POST	/api/projects/{id}/execute-tests	Execute a test suite
POST	/api/projects/{id}/execute-from-firebase	Fetch from Firebase & execute
GET	/api/projects/{id}/results	Get execution results
GET	/api/projects/{id}/results/{eid}	Get specific result

Example: Execute from Firebase via API

curl -X POST http://localhost:8000/api/projects/demo-project/execute-from-firebase \
  -H "Content-Type: application/json" \
  -d '{"target_url": "https://example.com"}'

Example: Fetch test cases

curl http://localhost:8000/api/projects/demo-project/test-cases

---

📄 License

MIT