Merged
Conversation
…ck testing This commit introduces the core AI intelligence layer for TestAble, enabling true LLM-powered test automation with both frontend and backend testing capabilities. ## New Components (backend/ai/) ### engine.py - Core AI Engine - Proper Stagehand SDK integration using page.act(), page.observe(), page.extract() - Cache-first strategy with intelligent fallback to AI - Action execution with full metrics and reasoning - Async context manager for clean resource management ### action_parser.py - Natural Language Parser - Comprehensive NL instruction parsing (20+ action types) - Element type detection (button, link, input, etc.) - Value extraction from quoted strings - Position extraction (first, second, last) - Container context extraction (within forms, modals) - Confidence scoring for ambiguous instructions ### assertions.py - AI-Powered Assertions - Semantic assertion verification using AI understanding - Multiple assertion types: visibility, text, value, count - Element state verification with reasoning - Fluent expectation builder API - Fallback text verification for robustness ### api_tester.py - Backend API Testing - Full HTTP client with request/response handling - JSON schema validation - Content verification (partial matching) - CRUD operation testing helper - Test suite execution with aggregation ### test_runner.py - Test Orchestration - Unified runner for frontend and backend tests - Natural language test step parsing - TestCase fluent builder API - Suite execution with parallel support - Result aggregation and metrics ## Updated Components ### testable_client.py - Proper Stagehand AI integration using page.act() - Intelligent selector matching as fallback - Integration with new ActionParser module ## Tests - Comprehensive test suite (16 passing tests) - Tests for all parser action types - Tests for type inference and configuration - Integration tests for full flow This architecture enables TestAble to accurately test both frontend UI components and backend APIs, with AI-powered understanding of natural language test instructions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ck testing
This commit introduces the core AI intelligence layer for TestAble, enabling true LLM-powered test automation with both frontend and backend testing capabilities.
New Components (backend/ai/)
engine.py - Core AI Engine
action_parser.py - Natural Language Parser
assertions.py - AI-Powered Assertions
api_tester.py - Backend API Testing
test_runner.py - Test Orchestration
Updated Components
testable_client.py
Tests
This architecture enables TestAble to accurately test both frontend UI components and backend APIs, with AI-powered understanding of natural language test instructions.