Playwright-Pytest Automation Framework

A production-ready Python automation testing framework using Playwright and pytest with multi-app architecture, Page Object Model, and comprehensive test coverage.

📊 Test Statistics

Total: 92/92 tests passing (100%)

Category	Tests	Status
E2E Tests	51	✅ 100%
└─ Sauce Demo	19	✅ Complete
└─ The Internet	25	✅ Complete
└─ Medusa Store	7	✅ Complete
API Tests	41	✅ 100%
└─ Restful Booker	13	✅ Complete
└─ Petstore	9	✅ Complete
└─ OMDb	5	✅ Complete
└─ ReqRes	14	✅ Complete

Average Execution Time: ~3-5 minutes (all tests)
Success Rate: 100%

🚀 Features

Multi-App Architecture - Test multiple web applications and APIs from one framework
Page Object Model - Clean separation of test logic and page interactions
Enhanced Allure Reporting - Rich HTML reports with comprehensive debugging capabilities:
- Screenshots (on success & failure)
- Video recordings
- Playwright traces
- Network HAR files
- Test history tracking
- Automatic failure categorization
Composite Decorators - Clean, maintainable test code with 90% decorator reduction
100% Test Coverage - All implemented tests passing consistently
API Testing - RESTful API tests with full CRUD coverage
Environment Support - Dev, staging, and production configurations
Parallel Execution Ready - pytest-xdist support for faster test runs

📁 Project Structure

playwright-pytest/
├── apps/                    # All test applications
│   ├── e2e/                 # E2E/UI Tests
│   │   ├── sauce_demo/      # 19 tests - E-commerce flow
│   │   ├── the_internet/    # 25 tests - Component testing
│   │   └── medusa_store/    # 1 test - Modern Next.js checkout
│   └── api/                 # API Tests
│       ├── restful_booker/  # 13 tests - CRUD + Token Auth
│       ├── petstore/        # 9 tests - OpenAPI/Swagger
│       ├── omdb/            # 5 tests - Search & data retrieval
│       └── reqres/          # 14 tests - User/Resource CRUD
├── config/
│   ├── apps/                # Per-app YAML configurations
│   ├── environments.yml     # Environment settings
│   └── test_data.yml        # Shared test data
├── infrastructure/
│   ├── fixtures/            # Shared pytest fixtures
│   ├── hooks/               # Allure integration hooks
│   └── utils/               # Utility functions
├── pages/                   # Shared/base page objects
├── docs/
│   ├── SETUP.md            # Detailed setup instructions
│   ├── TESTING.md          # Test execution guide
│   ├── PAGE_OBJECTS.md     # Page Object patterns
│   └── DECORATOR_GUIDE.md  # Pytest vs Allure decorators
├── conftest.py             # Root pytest configuration
├── pytest.ini              # Pytest settings
└── pyproject.toml          # Python project config

🛠 Installation

Prerequisites

Python 3.11+ (pyenv recommended)
Git

⚠️ Virtual Environment Note

All python, pip, and pytest commands require your virtual environment to be activated.

Look for the (.venv) prefix in your terminal prompt. If you don't see it, activate first:

# Activate virtual environment (run this whenever you open a new terminal)
source venv/bin/activate  # On Windows: venv\Scripts\activate

# You should see (.venv) in your prompt now

pyenv Setup (if using pyenv)

If you have pyenv installed but python command is not found, you need to initialize it:

# Install Python 3.11 (if not already installed)
pyenv install 3.11

# Set it as global or local version
pyenv local 3.11  # Creates .python-version file

# Initialize pyenv in your current shell
eval "$(pyenv init -)"

# Verify Python is available
python --version

Make it permanent: Add this to your shell config (~/.zshrc or ~/.bashrc):

echo 'eval "$(pyenv init -)"' >> ~/.zshrc
source ~/.zshrc

Setup Steps

# 1. Clone the repository
git clone <repository-url>
cd playwright-pytest

# 2. Create virtual environment
$ python -m venv venv
$ source venv/bin/activate  # On Windows: venv\Scripts\activate
# You should now see (.venv) in your prompt

# 3. Install dependencies
(.venv) $ pip install -e ".[dev]"

# 4. Install Playwright browsers
(.venv) $ playwright install chromium

# 5. Configure API keys (optional, for OMDb tests)
(.venv) $ echo "OMDB_API_KEY=your_key_here" > .env

Note: ReqRes API key is pre-configured. OMDb requires your own API key from omdbapi.com.

Version Compatibility Note:

The framework requires pytest >= 8.0.0, < 9.0.0 and allure-pytest >= 2.15.0 for proper Allure integration
These versions are automatically installed when running make setup or pip install -e ".[dev]"
Pytest 9.0+ has breaking API changes that are incompatible with current allure-pytest versions

⚡ Quick Start

Ensure virtual environment is activated before running any commands:

source venv/bin/activate  # You should see (.venv) in prompt

🚀 Using Make Shortcuts (Recommended!)

The project includes a Makefile with convenient shortcuts so you don't need to remember long commands.

# See all available commands
$ make help

Common commands:

Command	What it does
`make setup`	Full setup (install deps + browsers)
`make test`	Run all tests with Allure results
`make test-smoke`	Run smoke tests only
`make test-api`	Run API tests (parallel)
`make test-e2e`	Run E2E tests only
`make test-sauce-demo`	Run Sauce Demo tests
`make test-the-internet`	Run The Internet tests
`make test-parallel`	Run all tests in parallel
`make test-headed`	Run tests with visible browser
`make test-with-video`	Run tests with video recording
`make test-with-trace`	Run tests with trace recording
`make test-with-har`	Run E2E tests with network HAR recording
`make test-with-all-artifacts`	Run E2E with video + trace + HAR
`make report`	Generate and open Allure report ⭐
`make report-serve`	Serve Allure report (auto-reloads)
`make clean`	Clean all test artifacts

Note: Make commands work regardless of venv activation. Python commands inside the Makefile will use your system's Python, so ensure your venv is active first OR the Makefile is configured to use the venv Python.

Examples:

# One-time setup
$ source venv/bin/activate
(.venv) $ make setup

# Run tests and view report
(.venv) $ make test
(.venv) $ make report

# Quick smoke test
(.venv) $ make test-smoke

# Clean everything and start fresh
(.venv) $ make clean

Passing custom arguments to pytest:

# Run specific test file
(.venv) $ make test ARGS='apps/e2e/sauce_demo/tests/e2e/test_login.py'

# Run tests matching a keyword
(.venv) $ make test ARGS='-k "test_login"'

# Run with visible browser
(.venv) $ make test ARGS='--headed'

# Run specific test with verbose output
(.venv) $ make test ARGS='apps/api/reqres/tests/test_users.py::test_get_users -v'

# Stop on first failure
(.venv) $ make test ARGS='-x'

# Combine multiple options
(.venv) $ make test ARGS='apps/e2e/sauce_demo/ -k "login" --headed -x'

Direct Pytest Commands

If you prefer to run pytest directly without make:

Run All Tests

# All tests (E2E + API)
(.venv) $ pytest

# With verbose output
(.venv) $ pytest -v

# With Allure reporting (IMPORTANT: run tests FIRST to generate results)
(.venv) $ pytest --alluredir=test-results/allure-results

Run by Category

# E2E tests only (45 tests)
(.venv) $ pytest apps/e2e/

# API tests only (41 tests)
(.venv) $ pytest apps/api/

# Specific application
(.venv) $ pytest apps/e2e/sauce_demo/      # 19 Sauce Demo tests
(.venv) $ pytest apps/api/restful_booker/  # 13 Restful Booker tests

Run by Test Type

# All E2E tests
(.venv) $ pytest -m e2e

# All API tests
(.venv) $ pytest -m api

# Smoke tests only
(.venv) $ pytest -m smoke

# Regression tests
(.venv) $ pytest -m regression

Advanced Options

# Headful mode (visible browser)
(.venv) $ pytest --headed apps/e2e/

# Record video
(.venv) $ pytest --video on apps/e2e/sauce_demo/

# Enable tracing (for debugging)
(.venv) $ pytest --tracing on

# Parallel execution (4 workers)
(.venv) $ pytest -n 4

# Stop on first failure
(.venv) $ pytest -x

# Run last failed tests
(.venv) $ pytest --lf

📊 Reports

Allure Reports

🎯 Recommended: Use make report

(.venv) $ make report

This single command handles everything - runs tests, generates the report with history, and opens it in your browser.

Enhanced Allure Features

The framework includes advanced Allure reporting capabilities:

📸 Screenshots

On Failure: Automatic screenshot capture for failed tests
On Success: Final state screenshots for E2E tests
Organized by app name with timestamps

🎥 Video Recordings

Captured on test failure by default
Use make test-with-video for comprehensive video capture
Useful for debugging complex UI issues
Embedded directly in Allure reports

🔍 Playwright Traces

Detailed trace files for debugging
Use make test-with-trace to enable
Open with: npx playwright show-trace <trace.zip>

🌐 Network HAR Files

Complete HTTP archive with requests/responses
Use make test-with-har to enable (E2E tests only)
Includes timing information and headers
Human-readable network summary included

📊 Test History Tracking

Track test results over time
Identify flaky tests
Historical trend analysis
Preserved across test runs

🏷 Automatic Failure Categorization

Failed tests are automatically categorized:

Infrastructure Failure: Network errors, API unavailability
Performance Issue: Timeout errors, slow responses
Test Code Defect: Flaky tests, wrong setup
Product Bug: Application logic errors

Each category includes:

Common causes
Recommended actions

📝 Rich Test Documentation

Test descriptions with business context
Links to JIRA tickets and API docs
Test case IDs and requirement traceability
Parameter display for data-driven tests

Manual Process (for understanding):

Allure is a TWO-step process:

Step 1: Run Tests to Generate Results

# Run tests with --alluredir flag to collect test results
(.venv) $ pytest --alluredir=allure-results

# This creates the allure-results directory in the project root with test data

Step 2: Generate and View Report

# Now you can generate the HTML report from the results
$ allure generate allure-results -o test-results/allure-report --clean
# Note: allure command does NOT need venv - it's a standalone tool

# Open the report in your browser
$ allure open test-results/allure-report

Complete Example

# 1. Run tests with Allure results collection
(.venv) $ pytest --alluredir=allure-results

# 2. Generate report (only after tests have run)
$ allure generate allure-results -o test-results/allure-report --clean

# 3. View report
$ allure open test-results/allure-report

Troubleshooting

Error: allure-results does not exist

This means you haven't run tests with --alluredir flag yet
Run pytest --alluredir=allure-results FIRST, then generate report

Error: AttributeError: 'str' object has no attribute 'iter_parents'

This is caused by pytest 9.0+ incompatibility with allure-pytest
Fix: Downgrade to pytest 8.x: pip install 'pytest<9.0.0'
The framework's pyproject.toml now pins pytest to < 9.0.0 to prevent this issue
Verify versions: pip list | grep -E "(pytest|allure)" should show pytest>=8.0.0,<9.0.0 and allure-pytest>=2.15.0

Install Allure CLI (if not installed):

# Using npm (recommended)
npm install -g allure-commandline

# Or using Homebrew (macOS)
brew install allure

# Verify installation
allure --version

🧪 Test Applications

E2E Applications

Application	URL	Tests	Description
Sauce Demo	saucedemo.com	19	E-commerce flow, login, cart, checkout
The Internet	the-internet.herokuapp.com	25	Component testing (alerts, frames, tables, etc.)
Medusa Store	next.medusajs.com	1	Modern Next.js guest checkout flow

API Applications

Application	URL	Tests	Auth	Description
Restful Booker	restful-booker.herokuapp.com	13	Token	CRUD + Auth flow
Petstore	petstore.swagger.io	9	None	OpenAPI/Swagger demo
OMDb	omdbapi.com	5	API Key	Movie database search
ReqRes	reqres.in	14	API Key	User/resource CRUD

🏷 Test Markers

Use markers to categorize and filter tests:

@pytest.mark.app("sauce_demo")    # App assignment
@pytest.mark.e2e                  # E2E UI test
@pytest.mark.api                  # API test
@pytest.mark.smoke                # Critical path
@pytest.mark.regression           # Full regression suite

Run examples:

(.venv) $ pytest -m smoke                   # Smoke tests only
(.venv) $ pytest -m "e2e and smoke"         # E2E smoke tests
(.venv) $ pytest -m "not slow"              # Exclude slow tests

⚙️ Configuration

Environment Variables

Create .env file in project root:

(.venv) $ echo "OMDB_API_KEY=your_omdb_api_key_here" > .env
(.venv) $ echo "ENV=dev" >> .env  # Optional: dev, staging, or production

Or create manually:

# .env file contents
OMDB_API_KEY=your_omdb_api_key_here
ENV=dev  # Optional: dev, staging, or production

App Configuration

Each app has a YAML config in config/apps/:

# config/apps/sauce_demo.yml
name: sauce_demo
display_name: "Sauce Demo"
type: e2e

base_urls:
  dev: "https://www.saucedemo.com"
  staging: "https://www.saucedemo.com"
  production: "https://www.saucedemo.com"

test_users:
  standard:
    username: "standard_user"
    password: "secret_sauce"
  
settings:
  default_timeout: 30000
  screenshot_on_failure: true

📚 Documentation

Setup Guide - Detailed installation and configuration
Testing Guide - Comprehensive test execution examples
Page Objects - Pattern guide and best practices
Decorator Guide - Pytest vs Allure decorators
Allure Enhancements - Complete guide to reporting features (100% complete)
Test Cases - Each app has TEST_CASES.md with scenarios

Allure Enhancement Summary

The framework includes 13 comprehensive enhancements to Allure reporting:

Phase 1: Quick Wins (Immediate value)

Test Descriptions & Links - Business context and external references
HTTP Request/Response Logging - Complete API interaction logs
Environment Information - Full environment tracking
Test Parameters Display - Clear data-driven test documentation
Performance Thresholds - API performance tracking

Phase 2: High Value (Better organization) 6. Custom Test Categories - Organized test suites 7. Timeline Visualization - Nested steps for test flow 8. Owner/Reviewer Labels - Test ownership traceability 9. Screenshot on Success - Visual documentation for E2E tests

Phase 3: Advanced (Enhanced debugging) 10. Test History Tracking - Historical trend analysis 11. Video Attachments - Complete test execution recordings

Phase 4: Expert (Deep debugging) 12. Network HAR Files - Complete network inspection 13. Custom Categories - Automated failure categorization

Bonus:

Composite Decorators (@api_test(), @e2e_test()) - 90% decorator reduction

Usage:

# Clean composite decorators (90% less code)
@api_test(
    epic="Petstore API",
    feature="Pets",
    story="Create Pet",
    testcase="TC-PS-001",
    requirement="US-PETS-001",
    smoke=True,
)
def test_create_pet(client):
    # Test code here

🤝 Contributing

Adding New Tests

Use Composite Decorators for Clean Test Code

The framework provides @api_test() and @e2e_test() composite decorators:

API Test Example

from infrastructure.utils.allure_helpers import api_test
import allure

@api_test(
    epic="Petstore API",
    feature="Pets",
    story="Create Pet",
    testcase="TC-PS-001",
    requirement="US-PETS-001",
    severity=allure.severity_level.CRITICAL,
    smoke=True,
)
def test_create_pet(petstore_client):
    pet_data = {"name": "Buddy", "status": "available"}
    response = petstore_client.create_pet(pet_data)
    assert response.status_code == 200

E2E Test Example

from infrastructure.utils.allure_helpers import e2e_test
import allure

@e2e_test(
    epic="Sauce Demo E2E",
    feature="Checkout",
    story="Complete Purchase",
    testcase="TC-SD-030",
    app="sauce_demo",
    smoke=True,
)
def test_complete_checkout(login_page, inventory_page, cart_page):
    login_page.login("standard_user", "secret_sauce")
    inventory_page.add_item_to_cart("sauce-labs-backpack")
    cart_page.checkout()

File Structure

E2E Test:

# Create page object
apps/e2e/your_app/pages/your_page.py

# Create test
apps/e2e/your_app/tests/e2e/test_your_feature.py

# Add configuration
config/apps/your_app_config.yml

# Run your new test
(.venv) $ pytest apps/e2e/your_app/tests/e2e/test_your_feature.py -v

API Test:

# Create client
apps/api/your_api/clients/your_client.py

# Create test
apps/api/your_api/tests/test_your_endpoint.py

# Run your new test
(.venv) $ pytest apps/api/your_api/tests/test_your_endpoint.py -v

Code Style

Follow PEP 8
Use type hints
Add docstrings to public methods
Use Allure steps for reporting
Use composite decorators (@api_test(), @e2e_test())

📄 License

MIT License

Made with ❤️ using Playwright + Pytest

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.github/workflows		.github/workflows
apps		apps
config		config
docs		docs
infrastructure		infrastructure
pages		pages
scripts		scripts
.env.example		.env.example
.gitignore		.gitignore
.python-version		.python-version
Dockerfile		Dockerfile
Makefile		Makefile
README.md		README.md
allure-categories.json		allure-categories.json
automation-testing-framework-analysis.md		automation-testing-framework-analysis.md
conftest.py		conftest.py
docker-compose.yml		docker-compose.yml
pyproject.toml		pyproject.toml
pytest.ini		pytest.ini
setup.py		setup.py
testing_website_list.md		testing_website_list.md

Folders and files

Latest commit

History

Repository files navigation

Playwright-Pytest Automation Framework

📊 Test Statistics

🚀 Features

📁 Project Structure

🛠 Installation

Prerequisites

⚠️ Virtual Environment Note

pyenv Setup (if using pyenv)

Setup Steps

⚡ Quick Start

🚀 Using Make Shortcuts (Recommended!)

Direct Pytest Commands

Run All Tests

Run by Category

Run by Test Type

Advanced Options

📊 Reports

Allure Reports

Enhanced Allure Features

📸 Screenshots

🎥 Video Recordings

🔍 Playwright Traces

🌐 Network HAR Files

📊 Test History Tracking

🏷 Automatic Failure Categorization

📝 Rich Test Documentation

Step 1: Run Tests to Generate Results

Step 2: Generate and View Report

Complete Example

Troubleshooting

🧪 Test Applications

E2E Applications

API Applications

🏷 Test Markers

⚙️ Configuration

Environment Variables

App Configuration

📚 Documentation

Allure Enhancement Summary

🤝 Contributing

Adding New Tests

API Test Example

E2E Test Example

File Structure

Code Style

📄 License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages