Skip to content

Testing Quick Start

Jump to bottom Edit New page
crocodilestick edited this page Oct 23, 2025 · 2 revisions

Testing Quick Start Guide

Get up and running with CWA tests in 5 minutes!

🚀 TL;DR

# Run the interactive test runner
./run_tests.sh

# Choose option 5 for a quick test

That's it! The script handles everything else.

📋 Prerequisites

You need:

  • ✅ Docker installed and running
  • ✅ Python 3.8+ (usually already available)
  • ✅ Bash shell

On macOS/Linux:

# Check Docker
docker --version

# Check Python
python3 --version

On Windows (WSL2):

# Install WSL2 first, then:
docker --version
python3 --version

🎯 Quick Test (30 Seconds)

Verify everything works:

# Run smoke tests
pytest tests/smoke/ -v

Expected output:

tests/smoke/test_smoke.py::test_python_version PASSED
tests/smoke/test_smoke.py::test_flask_app_can_be_imported PASSED
...
13 passed in 15.23s ✅

🎨 Interactive Test Runner

The easiest way to run tests:

1. Launch the Menu

./run_tests.sh

2. You'll See This Menu

╔════════════════════════════════════════════════════════════════╗
║         CALIBRE-WEB AUTOMATED - TEST RUNNER                    ║
╠════════════════════════════════════════════════════════════════╣
║ Environment: host                                              ║
║ Python: 3.13.0                                                 ║
║ Docker: Available ✓                                            ║
╠════════════════════════════════════════════════════════════════╣
║ TEST SUITE OPTIONS                                             ║
╠════════════════════════════════════════════════════════════════╣
║ 1) Integration Tests (Bind Mount Mode)                        ║
║    → Standard mode for local testing (20 tests, ~3-4 min)     ║
║                                                                ║
║ 2) Integration Tests (Docker Volume Mode)                     ║
║    → For dev containers/DinD (19 tests, ~3-4 min)             ║
║                                                                ║
║ 3) Docker Startup Tests                                       ║
║    → Container health checks (9 tests, ~1 min)                ║
║                                                                ║
║ 4) All Tests                                                  ║
║    → Complete test suite (~5-7 min)                            ║
║                                                                ║
║ 5) Quick Test                                                 ║
║    → Fast verification (1 test, ~30 sec)                       ║
║                                                                ║
║ 6) Custom Test Selection                                      ║
║    → Run specific tests/patterns                               ║
║                                                                ║
║ 7) Show Info                                                  ║
║    → Test environment & documentation                          ║
║                                                                ║
║ q) Quit                                                       ║
╚════════════════════════════════════════════════════════════════╝

Enter your choice [1-7, q]:

3. Choose an Option

First time? Choose option 5 for a quick test.

Want full test suite? Choose option 1 (standard mode).

In a dev container? Choose option 2 (it may auto-select this).

🔧 Manual Testing

If you prefer command-line:

Run Specific Test Categories

# Smoke tests (30 seconds)
pytest tests/smoke/ -v

# Unit tests (2 minutes)
pytest tests/unit/ -v

# Docker tests (1 minute)
pytest tests/docker/ -v

# Integration tests (3-4 minutes)
pytest tests/integration/ -v

# Everything (5-7 minutes)
pytest tests/ -v

Run Specific Tests

# Single test file
pytest tests/unit/test_cwa_db.py -v

# Single test function
pytest tests/smoke/test_smoke.py::test_python_version -v

# Tests matching a pattern
pytest -k "database" -v

# Tests in parallel (faster)
pytest tests/unit/ -n auto

🐋 Testing in Dev Containers

If you're running tests inside a Docker container (Docker-in-Docker):

# Set environment variable
export USE_DOCKER_VOLUMES=true

# Run tests
pytest tests/integration/ -v

Or use the interactive runner (it auto-detects):

./run_tests.sh
# It will show "Environment: docker" and auto-select volume mode

📊 Understanding Results

Success ✅

tests/integration/test_ingest_pipeline.py::test_ingest_epub_already_target_format PASSED

Failure ❌

tests/integration/test_ingest_pipeline.py::test_some_feature FAILED

Check the error message for details.

Skipped ⏭️

tests/integration/test_cwa_db_tracks_import SKIPPED (requires config volume)

This is expected in Docker volume mode.

Test Count

20 passed, 1 skipped in 187.45s
  • 20 passed - Tests ran successfully
  • 1 skipped - Test was intentionally skipped
  • 187.45s - Total duration (~3 minutes)

🎓 Next Steps

Learn More

Once you're comfortable with basic testing:

  1. Running Tests - All execution modes explained
  2. Writing Tests - Contribute your own tests
  3. Docker-in-Docker Mode - Advanced DinD usage

Install Test Dependencies

For development:

pip install -r requirements-dev.txt

This installs:

  • pytest (test runner)
  • pytest-flask (Flask testing)
  • pytest-timeout (prevent hanging)
  • pytest-mock (mocking utilities)
  • faker (test data generation)

View Coverage Reports

See which code is tested:

# Generate HTML coverage report
pytest --cov=cps --cov=scripts --cov-report=html

# Open in browser
open htmlcov/index.html  # macOS
xdg-open htmlcov/index.html  # Linux

🐛 Troubleshooting

"Docker is required but not found"

Fix: Install Docker Desktop

"pytest: command not found"

Fix: Install test dependencies

pip install pytest
# Or install all dev dependencies:
pip install -r requirements-dev.txt

"Permission denied" when running tests

Fix: Make test runner executable

chmod +x run_tests.sh

Tests are very slow

First run? Docker needs to download images (~1-2 GB). This happens once.

Subsequent runs should be much faster (3-4 minutes for integration tests).

Speed up tests:

# Run in parallel
pytest tests/unit/ -n auto

# Run only fast tests
pytest -m "not slow" tests/

"Container failed to start"

Fix: Check Docker is running

docker ps

If Docker isn't running, start Docker Desktop.

💡 Pro Tips

1. Quick Iteration

While developing, use quick tests:

# Test just what you're working on
pytest tests/unit/test_cwa_db.py::test_specific_function -v

2. Watch Mode

Auto-run tests when files change:

pip install pytest-watch
ptw tests/unit/

3. Stop on First Failure

Useful when debugging:

pytest tests/ -x  # Stop at first failure
pytest tests/ -vv  # Extra verbose output

4. Re-run Failed Tests

After fixing a bug:

pytest --lf  # Run last failed tests only

5. Use the Interactive Runner

Seriously, it's easier:

./run_tests.sh

✅ Checklist

Before submitting code:

  • Run smoke tests: pytest tests/smoke/ -v
  • Run relevant unit tests
  • Run integration tests if changing workflows
  • All tests pass ✅
  • No new errors or warnings

🤝 Getting Help

You're ready to go! 🎉

Start with the interactive test runner (./run_tests.sh) and explore from there.

Add a custom footer
Add a custom sidebar

Clone this wiki locally