Demo: aido test

[dvirdung]

A shipping-cost calculator with tiered logic, validation, and clear boundaries to showcase aido test — proposed cases, coverage gaps, and edge cases. Comment aido test. Part of the Aido demo set.

[dvirdung]

aido test

[github-actions]

🤖 Hi @dvirdung, I’ve queued your aido test request. Follow progress here: https://github.com/aido-dev/aido/actions/runs/29812282855

[github-actions]

✅ Aido Test Plan & Gaps

As a senior SDET and QA strategist, I've reviewed the shipping_cost function and propose the following test plan to ensure its robustness, accuracy, and maintainability. Our focus will be on thorough unit testing, acknowledging the current isolated nature of the function, and strategically planning for future integration within a larger system.

---

Shipping Cost Calculator Test Plan

1) Test Plan Overview & Risk Areas

This test plan addresses the shipping_cost function, a critical component for calculating shipping expenses based on weight, distance, and an optional express flag. Its logic involves several tiered conditions and a final rounding step.

Key Testing Areas:

• Input Validation: Ensuring the function correctly handles invalid (non-positive) weight_kg and distance_km inputs.
• Weight Tiers: Verifying calculations at and around the boundaries for weight_kg (1kg, 5kg, 20kg), including the formula for weights over 20kg.
• Distance Tiers: Confirming calculations at and around the boundary for distance_km (100km).
• Express Shipping: Validating the 1.5x cost multiplier when express is True.
• Rounding Precision: Ensuring the final cost is accurately rounded to two decimal places.
• Combinatorial Logic: Testing various combinations of weight, distance, and express status to catch unexpected interactions.

Identified Risk Areas:

• Boundary Conditions: Miscalculations are common at the exact tier boundaries for weight_kg and distance_km.
• Floating-Point Arithmetic: Potential for subtle precision errors before the final rounding step.
• Error Handling Robustness: Ensuring the ValueError is raised consistently and correctly for all invalid inputs.
• Maintainability: Lack of comprehensive tests could lead to regressions if logic changes in the future.

2) Test Matrix / Checklist

This matrix outlines the test coverage by focus area.

Unit Tests (Primary Focus)

• Input Validation
  • [ ] Test weight_kg = 0, distance_km > 0 (Expect ValueError)
  • [ ] Test weight_kg < 0, distance_km > 0 (Expect ValueError)
  • [ ] Test distance_km = 0, weight_kg > 0 (Expect ValueError)
  • [ ] Test distance_km < 0, weight_kg > 0 (Expect ValueError)
  • [ ] Test weight_kg = 0, distance_km = 0 (Expect ValueError)
• Weight Tier Logic (Base Cost)
  • [ ] weight_kg just below 1 (e.g., 0.9)
  • [ ] weight_kg exactly 1 (Boundary: Tier 1 -> Tier 2)
  • [ ] weight_kg just above 1 (e.g., 1.1)
  • [ ] weight_kg just below 5 (e.g., 4.9)
  • [ ] weight_kg exactly 5 (Boundary: Tier 2 -> Tier 3)
  • [ ] weight_kg just above 5 (e.g., 5.1)
  • [ ] weight_kg just below 20 (e.g., 19.9)
  • [ ] weight_kg exactly 20 (Boundary: Tier 3 -> Formula)
  • [ ] weight_kg just above 20 (e.g., 20.1)
  • [ ] weight_kg significantly above 20 (e.g., 50)
• Distance Tier Logic (Per-KM Cost)
  • [ ] distance_km just below 100 (e.g., 99.9)
  • [ ] distance_km exactly 100 (Boundary: 0.05 -> 0.03 per km)
  • [ ] distance_km just above 100 (e.g., 100.1)
  • [ ] distance_km significantly above 100 (e.g., 250)
• Express Shipping Logic
  • [ ] express = True with various weight_kg/distance_km combinations.
  • [ ] express = False (default) with various weight_kg/distance_km combinations.
• Rounding Behavior
  • [ ] Test cases where the unrounded result ends in .xx5 (should round up, e.g., 7.575 -> 7.58)
  • [ ] Test cases where the unrounded result ends in .xx4 (should round down, e.g., 7.574 -> 7.57)
• Combinatorial Tests
  • [ ] Smallest valid inputs (weight_kg=0.01, distance_km=0.01, express=False)
  • [ ] Max weight tier, max distance tier, express
  • [ ] Mid weight tier, mid distance tier, no express
  • [ ] Specific combination covering all boundaries (e.g., weight_kg=20, distance_km=100, express=True)

Integration Tests (Conceptual, Not Directly Applicable to Current Scope)

• Given this function is a pure, self-contained calculation, direct integration tests are not immediately applicable.
• Future Scenario: If shipping_cost were consumed by a larger "Order Processing" or "Checkout" service, integration tests would verify:
  • [ ] The shipping_cost function correctly receives inputs from the calling service.
  • [ ] The calculated cost is correctly propagated and used by the calling service (e.g., displayed on a checkout page, added to a total).
  • [ ] Error handling (ValueError) from shipping_cost is gracefully managed by the calling service.

End-to-End (E2E) Tests (Conceptual, Not Directly Applicable to Current Scope)

• Similar to integration, E2E tests are beyond the scope of this single function.
• Future Scenario: If part of a full e-commerce application, E2E tests would simulate a user flow:
  • [ ] User adds items to cart.
  • [ ] User proceeds to checkout, enters shipping address/details.
  • [ ] The final shipping cost (derived from shipping_cost internally) is displayed correctly on the UI.
  • [ ] User completes the order with the correct total.

Regression Tests

• [ ] All validated unit tests will form the initial regression suite. These should be run automatically on every code change.

3) Detailed Test Cases

Here are examples of detailed test cases covering validation, boundaries, and logic:

Validation & Error Handling

• TC_VAL_001: Zero Weight Input
  • Title: shipping_cost with zero weight_kg
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=0, distance_km=10, express=False)
  • Expected Result: A ValueError is raised with the message "weight and distance must be positive".
• TC_VAL_002: Negative Distance Input
  • Title: shipping_cost with negative distance_km
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=5, distance_km=-10, express=False)
  • Expected Result: A ValueError is raised with the message "weight and distance must be positive".

Weight Tier Boundaries

• TC_WGT_001: Weight at Tier 1 Boundary (1kg)
  • Title: Calculate cost for weight_kg = 1
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=1, distance_km=50, express=False)
  • Expected Result: base = 5, per_km = 0.05. Total cost = 5 + (50 * 0.05) = 7.50.
• TC_WGT_002: Weight Just Over Tier 3 Boundary (20.1kg)
  • Title: Calculate cost for weight_kg = 20.1 (activating formula)
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=20.1, distance_km=50, express=False)
  • Expected Result: base = 25 + (20.1 - 20) * 2 = 25 + 0.1 * 2 = 25 + 0.2 = 25.2. per_km = 0.05. Total cost = 25.2 + (50 * 0.05) = 25.2 + 2.5 = 27.70.

Distance Tier Boundaries

• TC_DST_001: Distance at Tier 1 Boundary (100km)
  • Title: Calculate cost for distance_km = 100
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=10, distance_km=100, express=False)
  • Expected Result: base = 25, per_km = 0.05. Total cost = 25 + (100 * 0.05) = 30.00.
• TC_DST_002: Distance Just Over Tier 1 Boundary (100.1km)
  • Title: Calculate cost for distance_km = 100.1 (activating lower per_km rate)
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=10, distance_km=100.1, express=False)
  • Expected Result: base = 25, per_km = 0.03. Total cost = 25 + (100.1 * 0.03) = 25 + 3.003 = 28.00. (Rounded to 2 decimal places: 28.00).

Express Shipping & Rounding

• TC_EXP_001: Express Shipping with Rounding Up
  • Title: Calculate cost with express=True resulting in .xx5 for rounding
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=1, distance_km=1, express=True)
  • Expected Result: base = 5, per_km = 0.05. cost_unexp = 5 + (1 * 0.05) = 5.05. cost_exp = 5.05 * 1.5 = 7.575. Rounded cost = 7.58.
• TC_EXP_002: Express Shipping with Rounding Down
  • Title: Calculate cost with express=True resulting in .xx4 for rounding
  • Preconditions: None
  • Steps: Call shipping_cost(weight_kg=20.1, distance_km=1, express=True)
  • Expected Result: base = 25.2 (from TC_WGT_002), per_km = 0.05. cost_unexp = 25.2 + (1 * 0.05) = 25.25. cost_exp = 25.25 * 1.5 = 37.875. Rounded cost = 37.88.

4) Gaps & TODOs

• Non-Numeric Input Types: The current validation only checks for positive numbers. If weight_kg or distance_km could be strings, None, or other non-numeric types, a TypeError might occur before the explicit ValueError.
  • TODO: Consider adding tests with non-numeric inputs (e.g., weight_kg='abc') to verify appropriate error handling (likely a TypeError from Python itself, which is acceptable).
• Very Large Numbers: While unlikely for typical shipping scenarios, very large weight_kg or distance_km values could theoretically lead to floating-point overflow or performance issues.
  • TODO: Briefly assess the realistic bounds of inputs and, if necessary, add tests for extremely large but valid numbers.
• aido test Integration: The PR mentions aido test. This plan is manually crafted.
  • TODO: Explore how aido test can augment this plan by generating additional boundary conditions, negative scenarios, or property-based tests (e.g., using hypothesis) to uncover edge cases we might have missed. Can it suggest optimal test data? Can it analyze code paths to confirm coverage?

5) Fixtures & Data Setup

For a pure function like shipping_cost, complex fixtures or external data setup are not required.

• Test Data: All necessary input parameters (weight_kg, distance_km, express) will be provided directly within each test case.
• Parametrization: Python's pytest.mark.parametrize decorator is highly recommended for efficiently testing numerous combinations and boundary conditions with minimal code duplication.

6) Mocks / Stubs

• Not Applicable: The shipping_cost function is entirely self-contained and has no external dependencies (database calls, API calls, file I/O). Therefore, no mocks or stubs are needed for testing this specific unit.
• Future Consideration: If this function were refactored to, for example, retrieve shipping rates from a configuration service or an external API, those external interactions would be prime candidates for mocking to ensure fast, reliable, and isolated unit tests.

7) Automation Notes

• Test Location: As indicated by the PR, the new tests should reside in demo/test_demo.py.
• Testing Framework: Utilize pytest for its powerful features, including parametrize for data-driven tests and pytest.raises for expected exceptions.
• Naming Conventions: Adhere to test_ prefix for test files and functions (e.g., test_shipping_cost_zero_weight()).
• CI/CD Integration: Configure the CI pipeline (e.g., GitHub Actions, GitLab CI, Jenkins) to automatically run pytest for all new and existing tests on every push or pull request to the main branch. This ensures immediate feedback on regressions.
• Coverage Reporting: Integrate a tool like pytest-cov to generate code coverage reports, helping identify areas of the code that lack test scrutiny. Aim for high statement coverage (e.g., >95%) for this critical business logic.

8) Regression Suite Updates

All newly created unit tests for shipping_cost will be directly incorporated into the project's regression test suite.

Specific additions to emphasize:

• Boundary Conditions: Tests covering weight_kg at 1, 5, 20, and just above 20. Also distance_km at 100 and just above 100. These are historically high-risk areas for regressions.
• Error Conditions: Tests ensuring that invalid inputs continue to raise ValueError as expected.
• Express Flag Combinations: Tests verifying the express multiplier works correctly across various weight/distance scenarios.
• Rounding Precision: Specific tests to assert the round() function's behavior for inputs that could result in more than two decimal places.

This robust set of tests will provide confidence in the shipping_cost function's correctness and help prevent regressions as the codebase evolves.

---

Response generated using gemini-2.5-flash