Test Suite

This page covers the structure, configuration, and execution of the Test Suite, detailing its folder organization, logging setup, environment configurations, test group classifications, and batch execution mechanisms.

Table of Contents

1. Overview
2. Folder Structure
3. Test Suite Configuration
   3.1 Cache and Database Isolation
   3.2 Environment Variables Configuration
   3.3 Logging Configuration
4. Custom Behavior Extensions
   4.1 CustomTestRunner
   4.2 Core Test Base Classes
     4.2.1 BaseTestMixin
     4.2.2 BaseTestCase and BaseTransactionTestCase
5. Test Groups
   5.1 Groups Overview
   5.2 Transaction-Dependent Tests
   5.3 Non-Transactional Tests
6. Test Coverage Structure
   6.1 Design Principles
   6.2 Test Categories Breakdown
7. Batch Test Runner

1. Overview

• The Test Suite ensures the stability and correctness of The Wall API through automated testing.
• It is structured into test groups and categories, covering database interactions, API behavior, background task execution, and system-wide functionality.
• Execution isolation, configurable runtime settings, and structured logging ensure consistent and efficient test runs.
• The Batch Test Runner automates consecutive executions across different concurrency simulation settings, reducing manual effort.

2. Folder Structure

Tests are organized within the the_wall_api/tests/ folder, following a structured layout. Most modules focus on validating specific components such as API views, models, and background task execution, while others provide utilities and execution logic for managing test runs.

Module	Description
tests/	Test suite folder.
├── test_views/	API endpoints tests.
├── base_test_views.py	Base logic for API view tests.
├── test_profiles_views.py	Profile-related API tests.
└── test_wallconfig_file_views.py	File management API tests.
├── batch_tests_runner.py	Manages batch test execution.
├── test_caching.py	Redis and database caching tests.
├── test_celery_concurrent_tasks.py	Concurrent Celery tasks tests.
├── test_celery_sequential_tasks.py	Sequential Celery tasks tests.
├── test_djoser_integration.py	Authentication and user management tests.
├── test_locks.py	Concurrency and locking tests.
├── test_models.py	ORM model validation.
├── test_requests_flow.py	API requests flow tests.
├── test_serializers.py	Data validation and transformation tests.
├── test_throttling.py	API rate-limiting tests.
├── test_urls.py	URL routing and resolution tests.
├── test_utils.py	Core test base logic and utilities.
└── test_wallconstruction.py	Wall construction simulation tests.

✔ The structured organization of the test suite ensures clear separation of validation, utility, and execution-related modules, making test management and maintenance efficient.

Return to Top

---

3. Test Suite Configuration

The test suite is configured to maintain execution isolation, ensure consistent behavior, and provide detailed logging and debugging tools.

3.1 Cache and Database Isolation

To ensure consistency across test runs, the suite enforces strict isolation for both database transactions and cached data.

• Database Isolation → All tests execute in a dedicated test database, which is created at the start of the test suite, reset between test runs, and destroyed afterward to maintain isolation (built-in behavior of the Django test framework).
• Cache Isolation → Unlike database isolation, Redis cache clearing is explicitly managed within the test suite. Redis DB1 is used exclusively for tests, and the cache is cleared before and after relevant tests.

✔ These isolation mechanisms prevent side effects between tests and guarantee reliable, repeatable results.

Return to Top

3.2 Environment Variables Configuration

Specific environment variables are used to control logging, execution behavior, and error reporting. These variables are primarily imported in env_utils.py and settings.py, ensuring they are accessible throughout the application.

Environment Variable	Description	Defined In
ACTIVE_TESTING	Indicates whether the test suite is running. Imported into settings.py to be available globally.	env_utils.py
SEND_EXPECTED_TEST_SUITE_ERRORS_TO_CELERY	If enabled, known test-related errors are sent to Celery Worker 2 for sequential logging.	settings.py
TEST_LOGGING_LEVEL	Controls test log verbosity.	settings.py
TEST_SUITE_FILE_LOGGING_ENABLED	Enables or disables file-based logging for test runs (disabled by default).	settings.py
TEST_SUITE_LOGS_DIR	Folder where test logs are stored.	settings.py
TEST_SUITE_LOGS_FILE	Name of the test suite rotating log file.	settings.py
TEST_RUNNER	Defines CustomTestRunner used for test execution.	settings.py

💡 For a detailed description of the environment variables affecting testing, refer to the Testing and Debugging Configuration section of the Environment Variables Index page.

Return to Top

3.3 Logging Configuration

The logging behavior is defined with test-specific components in the LOGGING dictionary inside settings.py.

Component	Type	Description
test_suite	Formatter	Defines structured log formatting.
test_suite_console_logger	StreamHandler	Outputs logs to the console.
test_suite	Logger	Uses the test_suite_console_logger and the test_suite (formatter) for structured logging.

Return to Top

---

4. Custom Behavior Extensions

The test suite extends Django’s built-in test framework with custom execution logic, structured logging, and enhanced test case management.
The test_utils.py module introduces a custom test runner and base classes that provide reusable test logic, ensuring consistency across different test cases.

4.1 CustomTestRunner

The CustomTestRunner, implemented in test_utils.py, extends Django’s default test runner by incorporating custom logging and test environment management.

• Extended Initialization

  • Configures a dedicated test logger ('test_suite'), ensuring structured logging of test results.
    • By default the logger uses a StreamHandler to output logs to the console.
    • If file logging is enabled (TEST_SUITE_FILE_LOGGING_ENABLED), an additional RotatingFileHandler is configured, reusing the StreamHandler formatter.
  • Clears Redis caches to remove stale data and ensure test isolation.
  • Logs the start of the test execution and the configured concurrent simulation mode for tracking purposes.

• Extended Teardown

  • Clears Redis caches again to prevent test artifacts from persisting across runs.
  • Logs the aggregated test summary, providing insights into execution results.

Return to Top

4.2 Core Test Base Classes

Reusable base classes that extend Django’s test framework are used to provide common setup, utility methods, and structured test execution.

4.2.1 BaseTestMixin

The utility class BaseTestMixin provides reusable helper methods and setup logic for test cases. It is designed to be combined with other test classes, adding common functionality without modifying Django’s built-in test classes. Its key features include:

• Test Execution Tracking

  • Implements global counters for tracking test pass, fail, and error states.
  • Maintains test case group counters to aggregate results from logically related test categories.

• Structured Logging

  • Uses the dedicated test logger ('test_suite'), initialized in CustomTestRunner, to capture and format test logs.
  • Supports different logging levels <ALL, PASSED, FAILED, ERROR, SUMMARY, NO-LOGGING>
    (see TEST_LOGGING_LEVEL).
  • Outputs test execution summaries at the start and end of each test group.

• Throttling Management

  • Includes a throttling bypass mechanism, which allows tests to temporarily disable API request throttling, preventing rate limit errors.
  • The throttling in the test suite is disabled by default via the bypass mechanism and is selectively enabled for specific test categories.

• Cache Handling

  • Provides a @cache_clear decorator to clear the Redis cache before and after test executions.
  • Ensures that stale data does not interfere with test results.

• User & Authentication Utilities

  • Provides helper methods for test user creation and authentication.

• Test Result Logging

  • Captures test status, input values, expected vs. actual output, and timestamps.
  • Supports custom log formatting for improved debugging.
  
  

✔ BaseTestMixin streamlines test execution and reduces boilerplate code across the test suite.

Return to Top

4.2.2 BaseTestCase and BaseTransactionTestCase

Both BaseTestCase and BaseTransactionTestCase extend Django’s built-in test classes, integrating logging, test tracking, and API utilities while ensuring structured execution. The key difference between them is how they handle database transactions.

• a. Shared Inherited Features

  Both classes inherit from BaseTestMixin, integrating structured logging, test tracking, API utilities, caching control, and optional throttling bypass to ensure consistency across test executions.
  
  

• b. Transaction Behavior Differences

  Class	Transaction Behavior
  BaseTestCase	Uses Django’s default test database setup in TestCase. Rolls back changes only after the entire test class completes.
  BaseTransactionTestCase	Uses Django’s TransactionTestCase, allowing tests to commit database transactions, ensuring changes persist across tests when needed.

  
  

• c. Controlled Inheritance Execution Order

  • The classes are declared as BaseTestCase(BaseTestMixin, TestCase) and BaseTransactionTestCase(BaseTestMixin, TransactionTestCase), ensuring that BaseTestMixin methods take precedence over Django’s built-in test classes when called via super(), following Python’s method resolution order (MRO).

  • In setUpClass(), the parent class (TestCase or TransactionTestCase) implementaton is called first, followed by super().setUpClass().

    Toggle details

    class BaseTestcase(BaseTestMixin, TestCase):
    @classmethod
    def setUpClass(cls, *args, **kwargs):
        TestCase.setUpClass()
        super().setUpClass(*args, **kwargs)
    
    # ================================================================
    
    class BaseTransactionTestcase(BaseTestMixin, TransactionTestCase):
    @classmethod
    def setUpClass(cls, *args, **kwargs):
        TransactionTestCase.setUpClass()
        super().setUpClass(*args, **kwargs)

    • This ensures that Django’s test class setup runs before BaseTestMixin, allowing test tracking and logging to be initialized after the default setup.
    
    

  • In tearDownClass(), super().tearDownClass() runs first, followed by the implementation in the parent class (TestCase or TransactionTestCase).

    Toggle details

    class BaseTestcase(BaseTestMixin, TestCase):
    @classmethod
    def tearDownClass(cls):
        super().tearDownClass()
        TestCase.tearDownClass()
    
    # ================================================================
    
    class BaseTransactionTestcase(BaseTestMixin, TransactionTestCase):
    @classmethod
    def tearDownClass(cls):
        super().tearDownClass()
        TransactionTestCase.tearDownClass()

    • This ensures that BaseTestMixin logs and finalizes test execution details before Django’s teardown runs, preventing structured logging from being interrupted.
    
    

  ✔ This structure preserves Django’s default behavior while ensuring that BaseTestMixin enhances test execution without interference for both non-transactional and transactional test cases.
  
  

✔ BaseTestCase is used for standard tests, while BaseTransactionTestCase is required for tests that depend on committed database transactions, including concurrency handling (locks), Celery task validation, request flow testing, and database constraint enforcement.

Return to Top

---

5. Test Groups

Tests are categorized by whether they require committed database transactions, defining two distinct execution strategies.

5.1 Groups Overview

• Tests in the suite are divided into two primary groups, each defining a distinct execution strategy:

  • Transaction-Dependent Tests → Require fully committed database transactions.
  • Non-Transactional Tests → Execute independently of database transactions.

• All tests in the suite are derived from either BaseTestCase or BaseTransactionTestCase, ensuring consistent setup, execution, and teardown.

• Each group is further divided into test categories, described in detail in Section 6.2 - Test Categories Breakdown.

✔ This structured approach ensures clear execution boundaries while maintaining efficiency across different testing scenarios.

Return to Top

5.2 Transaction-Dependent Tests

Tests in this group extend BaseTransactionTestCase, ensuring fully committed database transactions.

Test Category	Purpose
Caching Tests	Assess cache behavior in transactions.
Concurrent Celery Task Tests	Verify task execution with transactions.
Locking & Concurrency Tests	Test database locks and parallel access.
Database Model Tests	Validate models and constraints.
Request Flow Tests	Evaluate API sequences relying on transactions.

✔ These tests ensure database reliability by replicating real-world transactional behavior.

Return to Top

5.3 Non-Transactional Tests

Tests in this group extend BaseTestCase, operating in an isolated database state where transactions are rolled back after execution.

Test Category	Purpose
API Endpoint Tests	Validate request handling and responses.
Sequential Celery Task Tests	Test asynchronous task execution independent of transactions.
Serializer Tests	Ensure correct data validation and transformation.
Throttling Tests	Assess API rate-limiting behavior.
URL Routing Tests	Verify correct resolution of API endpoints.
Wall Construction Tests	Simulate and validate wall-building processes.

✔ These tests focus on application logic, avoiding database constraints while ensuring correctness in request handling, background tasks, and business logic.

Return to Top

---

6. Test Coverage Structure

Each test category focuses on a specific aspect of application functionality, ensuring targeted validation of different system components. These categories span across both Transaction-Dependent and Non-Transactional test groups, covering database integrity, API behavior, concurrency handling, and background task execution.

6.1 Design Principles

The test categories in The Wall API follow a structured design, ensuring modularity, maintainability, and reliability. While each category serves a distinct purpose, they share common techniques and implementation patterns to maintain consistency across the suite.

• Common Design Techniques

  • Test Isolation → Every test runs in an independent environment, preventing interference between test cases.
  • Base Test Classes → All tests extend either BaseTestCase or BaseTransactionTestCase, providing standardized setup, logging, and utility methods.
  • Object-Oriented Structure → Common logic is encapsulated in shared base classes, which are subclassed to fit specific test contexts.
  • Consistent Data Setup → Predefined test fixtures and dynamically created objects ensure reliable test execution.
  • Parameterized Testing → Some tests use parameterization to validate multiple scenarios within a single test function.

• Common Behavior Across Categories

  • Logging & Tracking → Tests log execution details using a dedicated test logger ('test_suite'), ensuring structured debugging information.
  • Assertion Strategies → A combination of exact value assertions, response structure validation, and exception handling ensures comprehensive verification.
  • API Request Standardization → Endpoint tests follow a common structure for request construction, response validation, and error handling.
  • Throttling Control → API-related tests incorporate mechanisms to bypass or validate rate-limiting rules.

• Implementation Patterns

  • Layered Execution Flow → Tests are structured to mirror the application's layered architecture
    (e.g., serializer validation ensuring data integrity before processing API requests).
  • Transactional & Non-Transactional Execution → Database-dependent tests enforce transaction consistency, while others validate logic independently.
  • Sequential vs. Concurrent Scenarios → Celery task tests distinguish between sequential execution and concurrent task handling.
  • State Management & Cleanup → All tests ensure a clean state before and after execution.
  
  

✔ By adhering to these design principles, the test suite ensures uniformity, minimizes redundant logic, and maintains a clear separation of concerns across all test categories.

Return to Top

6.2 Test Categories Breakdown

• API Endpoint Tests → Verify request handling, response formatting, and endpoint behavior under different scenarios.
  Tested in: test_profiles_views.py, test_djoser_integration.py, and test_wallconfig_file_views.py
  Group: Non-Transactional

• Caching Tests → Assess how caching mechanisms function within the application context, ensuring correct cache usage and invalidation alongside database operations.
  Tested in: test_caching.py
  Group: Transaction-Dependent

• Celery Task Tests (Concurrent) → Validate Celery tasks that process multiple jobs in parallel, ensuring correct synchronization, resource access, prioritization, and failure recovery.
  Tested in: test_celery_concurrent_tasks.py
  Group: Transaction-Dependent

• Celery Task Tests (Sequential) → Ensure Celery tasks that require a predefined execution order handle intermediate states, file retention, and task chaining correctly.
  Tested in: test_celery_sequential_tasks.py
  Group: Non-Transactional

• Locking & Concurrency Tests → Verify database-level locking mechanisms and concurrent access handling to prevent race conditions and data corruption.
  Tested in: test_locks.py
  Group: Transaction-Dependent

• Database Model Tests → Ensure that database models function correctly by validating constraints, relationships, and data integrity rules.
  Tested in: test_models.py
  Group: Transaction-Dependent

• Request Flow Tests → Evaluate sequences of API requests that depend on database state and transactional integrity.
  Tested in: test_requests_flow.py
  Group: Transaction-Dependent

• Serializer Tests → Ensure that data serialization and deserialization processes adhere to validation rules and business logic.
  Tested in: test_serializers.py
  Group: Non-Transactional

• Throttling Tests → Validate API rate-limiting mechanisms and their enforcement under various request patterns.
  Tested in: test_throttling.py
  Group: Non-Transactional

• URL Routing Tests → Confirm that API endpoints are correctly resolved and mapped to their intended views.
  Tested in: test_urls.py
  Group: Non-Transactional

• Wall Construction Tests → Simulate and validate wall-building processes, ensuring correctness in computational logic.
  Tested in: test_wallconstruction.py
  Group: Non-Transactional

✔ By maintaining a structured test categorization, the suite guarantees systematic validation of all critical system components, ensuring stable and scalable test coverage. This organization allows for targeted testing while preserving execution efficiency and modular test expansion.

Return to Top

---

7. Batch Test Runner

The Batch Test Runner, implemented in batch_tests_runner.py, automates the structured execution of test runs while allowing for selective execution across different CONCURRENT_SIMULATION_MODE settings. It provides batch execution, configurable test selection, and aggregated logging to streamline debugging and performance analysis.

Key Features:

• Automated Batch Execution

  • Runs multiple test iterations dynamically, reducing manual intervention.
  • Allows specifying individual test modules or executing the entire test suite.

• CONCURRENT_SIMULATION_MODE selection

  • Executes the test suite sequentially for each selected mode.

• Configurable Execution Parameters

  • Accepts CLI arguments to specify:

    • Number of iterations (-i) → Defines how many times tests should run in a batch (Default: 1).
    • Test selection (--test-path-list) → Allows running specific tests instead of the full suite.
    • Mode selection (--mode-list) → Specifies a custom list of CONCURRENT_SIMULATION_MODE values for batch execution (Default: all).
    • Logging verbosity (-v) → Enables detailed test output when needed (Default: False).
    • Logging file disabling (--disable-logfile) → Disables file logging for lightweight runs (Default: False).

• Aggregated Logging & Error Tracking

  • Filters out unnecessary console output to keep logs readable.
  • Logs structured test summaries for each batch execution.
  • Captures failed test cases and errors, highlighting patterns for debugging.
  
  

✔ The Batch Test Runner automates test suite execution across different CONCURRENT_SIMULATION_MODE settings, allowing multiple consecutive runs without manual intervention.

---

Next:   Containerization
Back to: Sequential and Concurrent Simulation Modes

Return to Top
DTS Index
Home Page

---