🤖 Cognitive Process Automation (CPA) tool that automates Test Strategy creation using AI agents powered by Large Language Models (LLMs).
- What is ATS Generator?
- Functional Requirements
- Key Features
- How It Works
- Quick Start
- Run Locally with Free Resources
- LLM Provider Setup
- Usage Guide
- Architecture Deep Dive
- API Reference
- Project Structure
- Customization
- Troubleshooting
- Contributing
The Autonomous Test Strategy Generator is a tool that transforms raw software requirements (user stories, epics, tickets) into comprehensive Test Strategy documents using AI.
Creating a Test Strategy document manually requires:
- Hours of analysis to identify risks
- Deep expertise in testing methodologies
- Knowledge of BDD/Gherkin syntax for test scenarios
- Experience in resource estimation
ATS Generator automates all of this in seconds.
- QA Engineers - Generate test strategies quickly
- Test Leads - Get a starting point for test planning
- Developers - Understand testing requirements for features
- Product Managers - Review testing scope and coverage
- Students - Learn about test strategy components
This section describes what the system does from a user's perspective.
| Aspect | Description |
|---|---|
| What | Accept raw, unstructured requirements as input |
| Input | User stories, epics, feature descriptions, tickets - any text format |
| How | Single text area where users paste their requirements |
| Example Input | "As a user, I want to login with email so I can access my account" |
| Aspect | Description |
|---|---|
| What | Analyze requirements to identify testing risks |
| Output | Risk assessment report |
| Includes | • High-risk features that need extensive testing |
| • Complexity assessment (Low/Medium/High) | |
| • Potential failure modes and their impact | |
| • Dependencies and integration points | |
| • Risk mitigation recommendations |
| Aspect | Description |
|---|---|
| What | Generate comprehensive test scenarios and recommendations |
| Output | Test design document with BDD scenarios |
| Includes | • BDD/Gherkin scenarios (Given/When/Then format) |
| • Test type recommendations (UI, API, Performance, Security, etc.) | |
| • Test coverage matrix | |
| • Edge cases and boundary conditions | |
| • Positive and negative test scenarios |
| Aspect | Description |
|---|---|
| What | Compile all analyses into a formal Test Strategy document |
| Output | Executive-ready Test Strategy |
| Includes | • Executive Summary |
| • Scope and Objectives | |
| • Testing Approach and Methodology | |
| • Test Types and Levels | |
| • Entry and Exit Criteria | |
| • Resource Requirements | |
| • Timeline and Milestones | |
| • Risk Management Plan | |
| • Deliverables |
| Aspect | Description |
|---|---|
| What | Present generated strategy in a readable format |
| Features | • Tabbed interface (Strategy / Risk Analysis / Test Design) |
| • Scrollable content areas | |
| • Copy to clipboard functionality | |
| • Download as Markdown file | |
| • Duration metrics for each agent |
| Aspect | Description |
|---|---|
| What | Show real-time progress during generation |
| Features | • Visual agent pipeline showing current stage |
| • Progress percentage | |
| • Status messages for each agent | |
| • Non-blocking UI during processing |
Input (Raw Requirements):
User Story 1: As a user, I want to register with my email address
so that I can create an account.
User Story 2: As a user, I want to login with my credentials
so that I can access my dashboard.
User Story 3: As a user, I want to reset my password via email
so that I can regain access if I forget it.
Output (Generated by System):
| Agent | Output |
|---|---|
| Risk Analyst | Identifies authentication as high-risk, flags security concerns (brute force, session hijacking), assesses complexity as Medium-High |
| Test Architect | Generates 15+ BDD scenarios covering login success, login failure, password reset flow, edge cases; Recommends UI, API, Security, and Performance testing |
| QA Manager | Produces formal Test Strategy document with executive summary, scope, approach, resources, timeline, and deliverables |
| Feature | Description |
|---|---|
| 🔍 Risk Analysis | Automatically identifies high-risk features, complexity, and failure modes |
| 📝 BDD Scenarios | Generates Gherkin-style test scenarios (Given/When/Then) |
| 🎯 Test Type Recommendations | Suggests UI, API, Performance, Security testing needs |
| 📄 Strategy Document | Creates formal, executive-ready Test Strategy |
| 🔄 Multiple LLM Support | Works with Ollama (free/local), LM Studio, or Gemini |
| ⚡ Real-time Progress | Shows agent progress during generation |
| 📥 Export Options | Download as Markdown document |
The system uses a Three-Agent Pipeline where each agent is a specialized AI persona:
┌─────────────────────────────────────────────────────────────────────────┐
│ YOUR REQUIREMENTS │
│ "As a user, I want to login with email so I can access my account" │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ AGENT 1: RISK ANALYST │
│ • Identifies authentication as high-risk │
│ • Flags security concerns (brute force, session hijacking) │
│ • Assesses complexity: Medium-High │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ AGENT 2: TEST ARCHITECT │
│ • Creates BDD scenarios: │
│ Given a registered user │
│ When they enter valid credentials │
│ Then they should be logged in │
│ • Recommends: UI Testing, API Testing, Security Testing │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ AGENT 3: QA MANAGER │
│ • Compiles formal Test Strategy document │
│ • Includes: Scope, Approach, Resources, Timeline, Metrics │
│ • Executive-ready format │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ COMPLETE TEST STRATEGY │
│ Ready for review, export, and implementation │
└─────────────────────────────────────────────────────────────────────────┘
- Node.js 18+ - Download
- npm or yarn
- One of these LLM options:
- 🆓 Ollama (Recommended - Free, runs locally)
- 🆓 LM Studio (Free, runs locally with GUI)
- 💰 Google Gemini (Requires API key)
# 1. Clone the repository
git clone <repository-url>
cd "Autonomous Test Strategy (ATS) Generator"
# 2. Install dependencies
npm install
# 3. Set up environment
cp .env.example .env
# Edit .env with your LLM configuration (see next section)
# 4. Start the backend server
npm run server
# 5. In a new terminal, start the frontend
npm run dev
# 6. Open http://localhost:5173 in your browserYou can run this entire system 100% free on your local machine using Ollama. No cloud services, no API keys, no costs.
| Requirement | Cost | Notes |
|---|---|---|
| Node.js 18+ | Free | Download here |
| Ollama | Free | Local LLM runtime |
| ~4GB disk space | - | For model download |
| 8GB+ RAM | - | 16GB recommended for better performance |
macOS:
brew install ollamaLinux:
curl -fsSL https://ollama.com/install.sh | shWindows: Download installer from ollama.com/download
# Terminal 1: Start Ollama service (keep this running)
ollama serve# Terminal 2: Download a model based on your RAM
# For 8GB RAM:
ollama pull llama3.2:3b
# For 16GB RAM (recommended):
ollama pull llama3.2
# For 32GB+ RAM (best quality):
ollama pull mistral# Should return a list of installed models
curl http://localhost:11434/api/tags
# Or test with a simple prompt
ollama run llama3.2 "Say hello"# Navigate to project
cd "Autonomous Test Strategy (ATS) Generator"
# Install dependencies
npm install
# Create environment file (already configured for Ollama!)
cp .env.example .envThe default .env is pre-configured for Ollama:
LLM_PROVIDER=ollama
OLLAMA_MODEL=llama3.2
OLLAMA_BASE_URL=http://localhost:11434# Terminal 1: Start backend
npm run server
# ✓ Should show: "Server running on port 3001"
# Terminal 2: Start frontend
npm run dev
# ✓ Should show: "Local: http://localhost:5173/"- Open http://localhost:5173 in your browser
- Click "Generator" in the navigation
- Paste these sample requirements:
User Story 1: As a user, I want to register with my email address
so that I can create an account.
User Story 2: As a user, I want to login with my credentials
so that I can access my dashboard.
User Story 3: As a user, I want to reset my password via email
so that I can regain access if I forget it.
User Story 4: As an admin, I want to view all users
so that I can manage the user base.
- Click "Generate Test Strategy"
- Wait 30-90 seconds (local models are slower than cloud)
- View your generated Test Strategy! 🎉
| Your RAM | Model | Command | Speed | Quality |
|---|---|---|---|---|
| 8GB | Llama 3.2 3B | ollama pull llama3.2:3b |
Fast | Good |
| 8GB | Phi-3 Mini | ollama pull phi3 |
Fast | Good |
| 16GB | Llama 3.2 | ollama pull llama3.2 |
Medium | Better |
| 16GB | Mistral 7B | ollama pull mistral |
Medium | Better |
| 32GB+ | Llama 3.2 70B | ollama pull llama3.2:70b |
Slow | Best |
| 32GB+ | Mixtral 8x7B | ollama pull mixtral |
Slow | Best |
If you prefer a graphical interface:
- Download LM Studio from lmstudio.ai
- Open LM Studio → Search tab → Search "llama" or "mistral"
- Download a model (GGUF format)
- Go to Local Server tab → Click Start Server
- Update your
.env:
LLM_PROVIDER=lmstudio
LMSTUDIO_MODEL=local-model
LMSTUDIO_BASE_URL=http://localhost:1234/v1| Issue | Solution |
|---|---|
| "Ollama not running" | Run ollama serve in a terminal |
| "Model not found" | Run ollama pull llama3.2 |
| Very slow generation | Use smaller model: ollama pull llama3.2:3b |
| Out of memory errors | Close other apps, use smaller model |
| "Connection refused" | Check Ollama is running on port 11434 |
| Port 3001 already in use | Change PORT=3002 in .env |
- Close unnecessary applications - LLMs need RAM
- Use SSD storage - Faster model loading
- Start with smaller models - Test setup before downloading large models
- Keep Ollama running - First request loads model into memory (slow), subsequent requests are faster
Choose ONE of these options:
Ollama lets you run open-source LLMs locally on your machine.
# 1. Install Ollama
# macOS
brew install ollama
# Linux
curl -fsSL https://ollama.com/install.sh | sh
# Windows - Download from https://ollama.com/download
# 2. Start Ollama service
ollama serve
# 3. Pull a model (choose one)
ollama pull llama3.2 # Recommended - Good balance
ollama pull mistral # Fast and capable
ollama pull codellama # Better for code-related tasks
ollama pull llama3.2:70b # Best quality (needs 48GB+ RAM)
# 4. Configure .env
LLM_PROVIDER=ollama
OLLAMA_MODEL=llama3.2
OLLAMA_BASE_URL=http://localhost:11434Recommended Models by Hardware:
| RAM | Recommended Model |
|---|---|
| 8GB | llama3.2:3b, phi3 |
| 16GB | llama3.2, mistral |
| 32GB+ | llama3.2:70b, mixtral |
LM Studio provides a user-friendly interface for running local LLMs.
# 1. Download LM Studio from https://lmstudio.ai/
# 2. Open LM Studio and download a model:
# - Click "Search" tab
# - Search for "llama" or "mistral"
# - Download a model (GGUF format)
# 3. Start the local server:
# - Go to "Local Server" tab
# - Click "Start Server"
# - Note the port (usually 1234)
# 4. Configure .env
LLM_PROVIDER=lmstudio
LMSTUDIO_MODEL=local-model
LMSTUDIO_BASE_URL=http://localhost:1234/v1# 1. Get API key from https://makersuite.google.com/app/apikey
# 2. Configure .env
LLM_PROVIDER=gemini
GEMINI_API_KEY=your-api-key-here
GEMINI_MODEL=gemini-1.5-flashWorks with LocalAI, vLLM, text-generation-webui, etc.
LLM_PROVIDER=openai-compatible
OPENAI_BASE_URL=http://localhost:8080/v1
OPENAI_MODEL=your-model-name
OPENAI_API_KEY=optional-api-key# ===========================================
# LLM CONFIGURATION (Choose ONE provider)
# ===========================================
# Provider: ollama | lmstudio | gemini | openai-compatible
LLM_PROVIDER=ollama
# --- Ollama Settings ---
OLLAMA_MODEL=llama3.2
OLLAMA_BASE_URL=http://localhost:11434
# --- LM Studio Settings ---
# LMSTUDIO_MODEL=local-model
# LMSTUDIO_BASE_URL=http://localhost:1234/v1
# --- Gemini Settings ---
# GEMINI_API_KEY=your-api-key
# GEMINI_MODEL=gemini-1.5-flash
# --- OpenAI-Compatible Settings ---
# OPENAI_BASE_URL=http://localhost:8080/v1
# OPENAI_MODEL=gpt-3.5-turbo
# OPENAI_API_KEY=optional
# ===========================================
# LLM PARAMETERS (Optional)
# ===========================================
LLM_TEMPERATURE=0.7
LLM_MAX_TOKENS=8192
# ===========================================
# SERVER CONFIGURATION
# ===========================================
PORT=3001
NODE_ENV=developmentNavigate to the Generator page and paste your requirements:
User Story 1: As a user, I want to register with my email address
so that I can create an account.
User Story 2: As a user, I want to login with my credentials
so that I can access my dashboard.
User Story 3: As a user, I want to reset my password via email
so that I can regain access if I forget it.
User Story 4: As an admin, I want to view all users
so that I can manage the user base.
User Story 5: As a user, I want to update my profile
so that my information stays current.
Click "Generate Test Strategy" and watch the three agents work:
- Risk Analyst (10-15 seconds) - Analyzes risks
- Test Architect (10-15 seconds) - Creates test scenarios
- QA Manager (10-15 seconds) - Synthesizes strategy
The generated strategy includes:
- High-risk features identified
- Complexity assessment
- Potential failure modes
- Risk mitigation recommendations
- BDD/Gherkin scenarios for each feature
- Recommended test types (UI, API, Performance, Security)
- Test coverage matrix
- Executive Summary
- Scope and Objectives
- Testing Approach
- Test Types and Levels
- Entry/Exit Criteria
- Resource Requirements
- Timeline Estimates
- Risk Management
- Deliverables
Click Download to save as Markdown, or Copy to clipboard.
┌──────────────────────────────────────────────────────────────────┐
│ FRONTEND │
│ (React + TypeScript) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │
│ │ HomePage │ │ Generator │ │ StrategyViewPage │ │
│ │ │ │ Page │ │ (Output Display) │ │
│ └─────────────┘ └──────┬──────┘ └─────────────────────────┘ │
│ │ │
└──────────────────────────┼────────────────────────────────────────┘
│ REST API
▼
┌──────────────────────────────────────────────────────────────────┐
│ BACKEND │
│ (Express + TypeScript) │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ API Routes │ │
│ │ POST /api/strategy/generate │ │
│ │ GET /api/strategy/llm/status │ │
│ │ GET /api/strategy/agents │ │
│ └─────────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────▼───────────────────────────────────┐ │
│ │ StrategyGeneratorService │ │
│ │ • Orchestrates three-agent pipeline │ │
│ │ • Manages progress callbacks │ │
│ │ • Handles errors and retries │ │
│ └─────────────────────────┬───────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────▼───────────────────────────────────┐ │
│ │ LLM Provider │ │
│ │ • Abstracts different LLM backends │ │
│ │ • Supports: Ollama, LM Studio, Gemini, OpenAI-compatible │ │
│ └─────────────────────────┬───────────────────────────────────┘ │
│ │ │
└────────────────────────────┼──────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────┐
│ LLM ENGINE │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Ollama │ │LM Studio │ │ Gemini │ │ OpenAI-Compatible│ │
│ │ (Local) │ │ (Local) │ │ (Cloud) │ │ (Any) │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────────────┘ │
└──────────────────────────────────────────────────────────────────┘
Each agent has a carefully crafted system prompt that defines its persona and output format:
You are a Senior Risk Analyst specializing in software testing.
Your role is to analyze requirements and identify:
- High-risk features that need extensive testing
- Complexity factors that increase testing effort
- Potential failure modes and their impact
- Dependencies and integration points
...
You are a Senior Test Architect specializing in test design.
Your role is to create comprehensive test scenarios including:
- BDD/Gherkin scenarios (Given/When/Then)
- Test type recommendations (UI, API, Performance, Security)
- Test coverage matrix
- Edge cases and boundary conditions
...
You are a QA Manager responsible for test strategy.
Synthesize the analyses into a formal Test Strategy document:
- Executive Summary
- Scope and Objectives
- Testing Approach
- Resource Requirements
...
1. User enters requirements in textarea
│
▼
2. Frontend sends POST /api/strategy/generate
Body: { rawRequirements: "...", sessionId: "..." }
│
▼
3. StrategyGeneratorService.generateStrategy()
│
├─► runRiskAnalysis(requirements)
│ └─► callLLM(prompt, RISK_ANALYST_PROMPT)
│ └─► Returns: Risk Analysis text
│
├─► runTestDesign(requirements, riskAnalysis)
│ └─► callLLM(prompt, TEST_ARCHITECT_PROMPT)
│ └─► Returns: Test Design text
│
└─► runStrategySynthesis(riskAnalysis, testDesign)
└─► callLLM(prompt, QA_MANAGER_PROMPT)
└─► Returns: Test Strategy text
│
▼
4. API returns combined results
{
success: true,
riskAnalysis: "...",
testDesign: "...",
testStrategy: "...",
totalDuration: 45000
}
│
▼
5. Frontend stores in sessionStorage, navigates to /strategy
│
▼
6. StrategyViewPage displays results in tabbed interface
Generate a test strategy from raw requirements.
Request:
{
"rawRequirements": "User Story 1: As a user...",
"sessionId": "optional-session-id"
}Response:
{
"success": true,
"riskAnalysis": "## Risk Analysis\n...",
"testDesign": "## Test Scenarios\n...",
"testStrategy": "# Test Strategy Document\n...",
"agentOutputs": [
{
"agent": "Risk Analyst Agent",
"output": "...",
"timestamp": "2024-01-15T10:30:00Z",
"duration": 12500
}
],
"totalDuration": 45000
}Check LLM provider status and configuration.
Response:
{
"provider": "ollama",
"model": "llama3.2",
"baseUrl": "http://localhost:11434",
"available": true,
"error": null,
"supportedProviders": [...]
}Get descriptions of the three agents.
Response:
{
"agents": [
{
"id": "risk_analyst",
"name": "Risk Analyst Agent",
"description": "Analyzes requirements to identify high-risk features...",
"icon": "shield"
}
]
}Autonomous Test Strategy (ATS) Generator/
│
├── 📁 src/ # Frontend Source Code
│ ├── 📁 components/
│ │ ├── Layout.tsx # Main layout with navigation
│ │ └── 📁 ui/ # shadcn/ui components
│ │ ├── button.tsx
│ │ ├── card.tsx
│ │ ├── textarea.tsx
│ │ └── ...
│ │
│ ├── 📁 pages/
│ │ ├── HomePage.tsx # Landing page with features
│ │ ├── GeneratorPage.tsx # Requirements input + generation
│ │ └── StrategyViewPage.tsx # Strategy output display
│ │
│ ├── 📁 lib/
│ │ └── api.ts # Axios API utilities
│ │
│ ├── 📁 types/
│ │ └── index.ts # TypeScript type definitions
│ │
│ ├── App.tsx # Main React app with routing
│ ├── main.tsx # React entry point
│ └── index.css # Global styles + Tailwind
│
├── 📁 server/ # Backend Source Code
│ ├── 📁 lib/
│ │ ├── llm-provider.ts # Multi-LLM abstraction layer
│ │ └── gemini.ts # Agent prompts + Gemini client
│ │
│ ├── 📁 services/
│ │ └── StrategyGeneratorService.ts # Three-agent orchestration
│ │
│ ├── 📁 routes/
│ │ ├── strategy.ts # Strategy generation endpoints
│ │ └── requirements.ts # Requirements parsing endpoints
│ │
│ ├── 📁 agents/ # Legacy agent implementations
│ │ ├── BaseAgent.ts
│ │ ├── RiskAssessorAgent.ts
│ │ └── ...
│ │
│ └── index.ts # Express server entry point
│
├── 📁 public/ # Static assets
│
├── .env.example # Environment template
├── package.json # Dependencies
├── tsconfig.json # TypeScript config
├── vite.config.ts # Vite bundler config
├── tailwind.config.js # Tailwind CSS config
└── README.md # This file
Edit server/lib/gemini.ts to customize agent behavior:
export const AGENT_PROMPTS = {
RISK_ANALYST: `
You are a Senior Risk Analyst...
// Customize the persona and output format
`,
TEST_ARCHITECT: `...`,
QA_MANAGER: `...`
}Edit server/lib/llm-provider.ts:
// Add new provider type
export type LLMProvider = 'gemini' | 'ollama' | 'lmstudio' | 'your-provider'
// Add configuration
case 'your-provider':
return {
provider: 'your-provider',
model: process.env.YOUR_MODEL,
baseUrl: process.env.YOUR_BASE_URL
}
// Add API call function
async function callYourProvider(prompt, systemPrompt, config) {
// Implementation
}Edit tailwind.config.js and src/index.css for theming.
# Start Ollama service
ollama serve
# Check if running
curl http://localhost:11434/api/tags# Pull a model
ollama pull llama3.2
# List installed models
ollama list- Open LM Studio
- Go to "Local Server" tab
- Click "Start Server"
- Verify port matches
LMSTUDIO_BASE_URLin.env
- Verify API key is correct
- Check quota at https://console.cloud.google.com/
- Ensure billing is enabled (free tier available)
- Local LLMs are slower than cloud APIs
- Try a smaller model:
ollama pull llama3.2:3b - Reduce
LLM_MAX_TOKENSin.env
- Use a smaller model
- Close other applications
- Consider cloud provider (Gemini)
Contributions are welcome! Please:
- Fork the repository
- Create a feature branch
- Make your changes
- Submit a pull request
MIT License - feel free to use in personal and commercial projects.
- Ollama - Local LLM runtime
- LM Studio - Local LLM GUI
- Google Gemini - Cloud LLM API
- shadcn/ui - UI components
- Tailwind CSS - Styling