Skip to content

Testing

Jump to bottom
Dragon edited this page Dec 22, 2025 · 1 revision

🧪 Testing NaviDuck: A Practical Guide

Last updated: 12/22/2025

🎯 The Testing Mindset

Testing isn't about bureaucracy—it's about confidence. Every test you write is a promise that something will keep working. For NaviDuck, testing means users can trust their searches, AI answers, and privacy features will work when they need them most.

🚀 Your Testing Toolkit

Essential Testing Stack

# Core testing tools
pip install pytest pytest-cov pytest-mock

# Quality checks
pip install flake8 black mypy

# Specialized testing
pip install pytest-docker pytest-playwright  # For advanced tests

The Test Hierarchy

Think of testing like building a pyramid:

        ┌─────────────────────┐
        │    User Journeys    │ ← 10% - Does the whole system work?
        ├─────────────────────┤
        │  Feature Integration│ ← 20% - Do components work together?
        ├─────────────────────┤
        │   Unit Tests        │ ← 70% - Do individual pieces work?
        └─────────────────────┘

🧪 Writing Tests That Matter

Unit Tests: The Foundation

Unit tests check individual functions in isolation. They're fast, reliable, and tell you exactly what's broken.

Example: Testing Search Logic

# tests/test_search.py
import pytest
from unittest.mock import Mock, patch
from naviduck.search_manager import SearchManager

def test_search_parses_results_correctly():
    """When we search, results should be properly formatted."""
    # Arrange: Set up the test
    mock_state = Mock()
    mock_network = Mock()
    manager = SearchManager(mock_state, mock_network)
    
    # Simulate HTML response from a search engine
    mock_html = """
    <html>
        <a href="https://example.com">Test Result</a>
        <a href="https://python.org">Python Website</a>
    </html>
    """
    mock_network.get.return_value.text = mock_html
    
    # Act: Perform the search
    results = manager.search("test query")
    
    # Assert: Verify expectations
    assert len(results) == 2
    assert results[0]["title"] == "Test Result"
    assert results[0]["url"] == "https://example.com"
    assert isinstance(results, list)  # Should always return a list

def test_empty_search_returns_empty_list():
    """Empty searches shouldn't crash."""
    manager = SearchManager(Mock(), Mock())
    results = manager.search("")
    assert results == []  # Graceful handling of edge cases

def test_search_handles_network_failures():
    """When network fails, we should get fallback results."""
    mock_network = Mock()
    mock_network.get.side_effect = ConnectionError("No internet")
    
    manager = SearchManager(Mock(), mock_network)
    results = manager.search("test")
    
    # Should return empty list, not crash
    assert results == []

Integration Tests: Making Components Talk

Integration tests verify that different parts of NaviDuck work together correctly.

Example: Search → Display Flow

# tests/integration/test_search_flow.py
def test_complete_search_flow():
    """From user typing to results showing."""
    # Set up real components (not mocks)
    state = BrowserState()
    network = NetworkManager(state)
    search = SearchManager(state, network)
    
    # Simulate user action
    user_query = "python tutorial"
    results = search.search(user_query)
    
    # Verify the chain worked
    assert results is not None
    assert len(state.history) == 1  # Should record the search
    assert state.history[0]["query"] == user_query
    assert state.current_results == results  # UI can display these

End-to-End Tests: The User's Perspective

E2E tests simulate real user interactions. They're slower but catch issues users would actually experience.

Example: Complete User Journey

# tests/e2e/test_user_journey.py
def test_search_and_bookmark_flow():
    """User searches, views result, bookmarks it."""
    # This might use simulated input/output
    # or tools like Selenium for browser automation
    
    # 1. User launches NaviDuck
    # 2. Types "s python documentation"
    # 3. Selects first result
    # 4. Views page
    # 5. Bookmarks it
    # 6. Verifies bookmark saved
    
    # These tests ensure the happy path works

🎨 Test Patterns That Work

The Arrange-Act-Assert Pattern

def test_something():
    # Arrange: Set up test conditions
    setup_data = create_test_data()
    system_under_test = initialize_component()
    
    # Act: Perform the action being tested
    result = system_under_test.do_something(setup_data)
    
    # Assert: Verify expected outcomes
    assert result.worked == True
    assert result.data == expected_data

Parameterized Tests

@pytest.mark.parametrize("query,expected_count", [
    ("python", 10),        # Normal search
    ("", 0),              # Empty search
    ("a" * 1000, 10),     # Very long query
    ("python 🐍", 10),    # Unicode characters
    ("test!@#$", 10),     # Special characters
])
def test_search_with_various_queries(query, expected_count):
    """Search should handle all kinds of queries gracefully."""
    results = search_manager.search(query)
    assert len(results) == expected_count

🔍 What to Test (and What Not To)

Must-Test Areas

  1. Search Engine Parsing

    • Each engine's unique HTML structure
    • CAPTCHA detection logic
    • Fallback mechanisms

  2. AI Response Generation

    • Answer accuracy for common questions
    • Error handling when APIs fail
    • Conversation flow

  3. Network Operations

    • Connection failures
    • Timeout handling
    • Proxy/Tor integration

  4. User Interface

    • Command parsing
    • Display formatting
    • Error messages

Lower Priority Tests

  • Third-party library internals (they should test themselves)
  • Python language features (Python already works)
  • Obvious getter/setter methods (unless complex logic)

🛠️ Building a Test Suite

Test Organization

tests/
├── unit/                    # Fast, isolated tests
│   ├── test_search.py      # Search logic
│   ├── test_ai.py          # AI responses
│   ├── test_network.py     # HTTP operations
│   └── test_ui.py          # User interface
├── integration/            # Component interaction
│   ├── test_search_flow.py # Search → Display
│   └── test_tor_flow.py    # Tor integration
├── e2e/                    # User journeys
│   └── test_user_flow.py   # Complete workflows
└── conftest.py            # Shared test setup

Shared Test Setup

# tests/conftest.py
import pytest
from naviduck.browser_state import BrowserState
from naviduck.search_manager import SearchManager

@pytest.fixture
def clean_state():
    """Fresh BrowserState for each test."""
    state = BrowserState()
    state.history = []  # Start with clean history
    state.bookmarks = []
    return state

@pytest.fixture
def search_manager(clean_state):
    """SearchManager with clean state."""
    from unittest.mock import Mock
    return SearchManager(clean_state, Mock())

🚨 Testing Edge Cases

Error Conditions

def test_search_when_internet_down():
    """Graceful handling of network failures."""
    mock_network = Mock()
    mock_network.get.side_effect = ConnectionError("Offline")
    
    manager = SearchManager(Mock(), mock_network)
    results = manager.search("test")
    
    # Should not crash
    assert results == []
    # Should log the error
    assert "ConnectionError" in caplog.text

def test_ai_when_api_rate_limited():
    """AI should handle API limits gracefully."""
    with patch('naviduck.navai.requests.get') as mock_get:
        mock_get.return_value.status_code = 429  # Rate limited
        
        ai = NavAI()
        response = ai.ask("test question")
        
        assert "rate limit" in response.lower()
        assert "try again" in response.lower()

Boundary Conditions

def test_very_long_urls():
    """URLs longer than typical should still work."""
    long_url = "https://example.com/" + "a" * 1000
    result = page_loader.load_page(long_url)
    assert result["success"] == True

def test_unicode_handling():
    """Emoji and special characters in searches."""
    results = search_manager.search("Python 🐍 programming ⚡")
    assert len(results) > 0

📊 Measuring Test Quality

Code Coverage

# Generate coverage report
pytest --cov=naviduck --cov-report=html tests/

# View in browser
open htmlcov/index.html  # macOS
start htmlcov/index.html # Windows
xdg-open htmlcov/index.html  # Linux

Good Coverage Targets:

  • 70%+: Decent coverage
  • 80%+: Good coverage
  • 90%+: Excellent coverage

Test Performance

# Run tests with timing
pytest --durations=10 tests/

# Fast tests (<0.1s): Unit tests
# Medium tests (0.1-1s): Integration tests  
# Slow tests (>1s): E2E tests

🔧 Test Utilities

Mock Responses

def create_mock_search_response():
    """Create realistic mock search response."""
    return Mock(
        text="""
        <html>
            <div class="result">
                <a href="https://example.com">Example</a>
                <p>Example description</p>
            </div>
        </html>
        """,
        status_code=200,
        headers={'Content-Type': 'text/html'}
    )

Test Data Factories

def create_test_bookmark():
    """Generate consistent test bookmark."""
    return {
        "title": "Test Bookmark",
        "url": "https://example.com",
        "added": "2024-01-01T12:00:00"
    }

def create_search_history_entry():
    """Create realistic history entry."""
    return {
        "type": "search",
        "query": "test query",
        "engine": "brave",
        "timestamp": "2024-01-01T12:00:00",
        "results": 10
    }

🎭 Testing Different Scenarios

Platform-Specific Tests

@pytest.mark.windows
def test_windows_path_handling():
    """Windows paths with backslashes."""
    if sys.platform != "win32":
        pytest.skip("Windows-only test")
    
    path = r"C:\Users\Test\NaviDuck\data.json"
    result = data_manager.validate_path(path)
    assert result == True

@pytest.mark.linux
def test_linux_permissions():
    """Linux file permission handling."""
    if sys.platform != "linux":
        pytest.skip("Linux-only test")

Configuration Tests

def test_with_different_configs():
    """Test behavior with various configurations."""
    configs = [
        {"tor_enabled": True, "engine": "ddg"},
        {"tor_enabled": False, "engine": "google"},
        {"use_emoji": True, "engine": "brave"},
    ]
    
    for config in configs:
        state = BrowserState()
        state.tor_enabled = config["tor_enabled"]
        state.current_engine = config["engine"]
        
        # Test search works with this config
        results = search_manager.search("test", state=state)
        assert len(results) > 0

🚀 Running Tests Effectively

Development Workflow

# Run tests on file change (auto-test)
ptw -- tests/  # Uses pytest-watch

# Run specific test categories
pytest -m "not slow"  # Skip slow tests
pytest -k "search"    # Only search tests
pytest -xvs           # Stop on first failure, verbose, no capture

# Run in parallel
pytest -n auto tests/  # Uses pytest-xdist

CI/CD Integration

# .github/workflows/test.yml
name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python: ['3.8', '3.9', '3.10', '3.11']
    
    steps:
    - uses: actions/checkout@v3
    - name: Python ${{ matrix.python }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python }}
    
    - name: Install dependencies
      run: pip install -r requirements-dev.txt
    
    - name: Run tests
      run: pytest --cov=naviduck --cov-report=xml
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3

🐛 Debugging Failing Tests

When Tests Fail

# Use these techniques:

# 1. Add debug prints
print(f"DEBUG: Variable value = {variable}")

# 2. Use pdb (Python debugger)
import pdb; pdb.set_trace()

# 3. Check test data
print(f"Test data: {test_data}")

# 4. Isolate the failure
# Comment out parts until it passes, then add back

Common Test Issues

# Flaky tests (sometimes pass, sometimes fail)
# Solution: Mock time, random, or network

# Slow tests
# Solution: Use mocks, skip if not necessary

# Tests that depend on each other
# Solution: Use fresh fixtures for each test

# Tests that break when code changes
# Solution: Test behavior, not implementation

📈 Test Metrics That Matter

Health Dashboard

# Example test metrics to track:
metrics = {
    "total_tests": 150,
    "passing": 148,
    "failing": 2,
    "coverage": 85.5,
    "avg_duration": "0.8s",
    "slowest_test": "test_e2e_user_flow: 12.3s",
    "flaky_tests": ["test_network_timeout"],  # Needs investigation
}

Test Quality Signals

  • ✅ Green: All tests pass
  • ⚠️ Yellow: Some failures, but known/acceptable
  • 🔴 Red: Critical failures
  • 🐌 Slow: Tests taking too long
  • 🎭 Flaky: Inconsistent test results

🎯 Writing Good Tests

Characteristics of Good Tests

  1. Fast: Run in milliseconds
  2. Isolated: Don't depend on other tests
  3. Repeatable: Same result every time
  4. Self-verifying: Pass/fail is obvious
  5. Timely: Written with the code

Test Naming Convention

# Good test names:
def test_search_returns_results(): ...
def test_empty_search_handled_gracefully(): ...
def test_ai_response_for_common_questions(): ...

# Bad test names:
def test1(): ...                     # What does it test?
def test_search(): ...               # Too vague
def test_that_thing_works(): ...     # Unclear

🔮 The Future of Testing in NaviDuck

Planned Improvements

  1. Property-based testing: Generate random inputs to find edge cases
  2. Fuzz testing: Malformed inputs to test robustness
  3. Performance regression tests: Ensure new features don't slow things down
  4. Security vulnerability tests: Automated security scanning

Testing Philosophy

We believe in pragmatic testing:

  • Test what users actually experience
  • Focus on critical paths
  • Make tests maintainable
  • Use testing to enable refactoring, not prevent it

🎉 Your Testing Journey Starts Here

First Test to Write

# tests/test_first.py
def test_naviduck_starts():
    """Most basic test: can we import and initialize?"""
    from naviduck import BrowserState
    state = BrowserState()
    assert state is not None
    print("✅ NaviDuck can start!")

Next Steps

  1. Run existing tests: pytest tests/
  2. Add a test for a bug you fixed
  3. Improve test coverage in an area you understand
  4. Write an integration test for a feature you use

Remember: Every test you write makes NaviDuck more reliable for users around the world. Your tests aren't just code—they're trust built into the system.

Last updated: 12/22/2025

Testing isn't about finding bugs—it's about preventing surprises. Write tests so users never see the bugs you caught.

Clone this wiki locally