Environment Variables Index

This page provides an exhaustive overview of all environment variables used in The Wall API. These variables configure database connections, caching, task queuing, security settings, and other core application functions.

Overview
Environment Configuration Files

2.1 Folder Organization and Purpose

2.2 Folder structure
Complete Index of Environment Variables

3.1 General Application Settings
ALLOWED_HOSTS
API_VERSION
DEBUG
MAX_USER_WALL_CONFIGS
PROJECT_MODE
SECRET_KEY

3.2 PostgreSQL Database Configuration
DB_CONNECT_TIMEOUT
DB_ENGINE
DB_HOST
DB_PORT
DB_STATEMENT_TIMEOUT
POSTGRES_DB
POSTGRES_PASSWORD
POSTGRES_USER

3.3 Redis Caching and Message Queue Configuration
REDIS_CACHE_TRANSIENT_DATA_TIMEOUT
REDIS_DB_NUMBER
REDIS_DB_NUMBER_CELERY
REDIS_PASSWORD
REDIS_SOCKET_CONNECT_TIMEOUT
REDIS_SOCKET_TIMEOUT
REDIS_URL

3.4 Celery Task Processing Configuration
CELERY_BROKER_URL
CELERY_RESULT_BACKEND
CELERY_RESULT_EXPIRES
CELERY_TASK_PRIORITY
CELERY_WORKER_PREFETCH_MULTIPLIER
LIGHT_CELERY_CONFIG
STARTED_FROM_CELERY_SERVICE

3.5 Wall Construction and Simulation
ICE_COST_PER_CUBIC_YARD
ICE_PER_FOOT
MAX_SECTION_HEIGHT
MAX_WALL_LENGTH
MAX_WALL_PROFILE_SECTIONS

3.6 Concurrency and Performance
CONCURRENT_SIMULATION_MODE
CPU_THREADS
MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING
MAX_CONCURRENT_NUM_CREWS_THREADING
MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING
MAX_SECTIONS_COUNT_CONCURRENT_THREADING
MAX_SECTIONS_COUNT_FULL_RANGE_CACHING
MAX_SECTIONS_COUNT_SYNCHRONOUS_RESPONSE
SECTION_COMPLETION_GRACE_PERIOD_MULTIPROCESSING
SECTION_COMPLETION_GRACE_PERIOD_THREADING
VERBOSE_MULTIPROCESSING_LOGGING

3.7 Testing and Debugging Configuration
ACTIVE_TESTING
SEND_EXPECTED_TEST_SUITE_ERRORS_TO_CELERY
TEST_LOGGING_LEVEL
TEST_SUITE_FILE_LOGGING_ENABLED

3.8 Logging Configuration
BUILD_SIM_LOGS_ARCHIVE_RETENTION_DAYS
BUILD_SIM_LOGS_ARCHIVE_DIR
BUILD_SIM_LOGS_DIR
BUILD_SIM_LOGS_RETENTION_DAYS
ERROR_LOGS_DIR
LOGS_DIR_NAME
TEST_SUITE_LOGS_DIR
TEST_SUITE_LOGS_FILE

3.9 Linux Development Configuration Variables
APP_USER_ID, APP_GROUP_ID, APP_GROUP_NAME

1. Overview

Environment variables are stored in .env and .conf files inside the config/envs/ directory.
In some cases, certain variables are defined only in the application code (e.g., settings.py, env_utils.py).
Sensitive credentials (e.g., database passwords) are never hardcoded in the source code but are securely stored in these configuration files.
Example configuration files (*.example) are included in the repository for reference.

2. Environment Configuration Files

2.1. Folder Organization and Purpose

Common settings are defined in the config/envs/common/ folder and are shared across all environments.
Development-specific settings are stored in config/envs/dev/.
Production-specific settings are located in config/envs/prod/.

2.2 Folder structure:

config/envs/ ├── common/ │ └── redis.conf # Redis main configuration file ├── dev/ │ ├── postgres_dev.env # PostgreSQL credentials for development │ └── the_wall_api_dev.env # General environment variables for development ├── prod/ │ ├── postgres_prod.env # PostgreSQL credentials for production │ ├── redis_prod.env # Redis authentication settings for production │ └── the_wall_api_prod.env # General environment variables for production

🗒️ These files are loaded dynamically based on the deployment environment.

Return to Top

3. Complete Index of Environment Variables

3.1 General Application Settings

ALLOWED_HOSTS

Description:

Defines a list of host/domain names that the Django application will serve.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to localhost,127.0.0.1)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Dependencies:
  - If DEBUG=False, ALLOWED_HOSTS must be set, or Django raises a DisallowedHost error.
  - If DEBUG=True and ALLOWED_HOSTS is empty ([]), Django allows requests from any host.

Current Setup in Different Environments:

Environment	ALLOWED_HOSTS Value
`demo`	`localhost,127.0.0.1`
`dev`	`localhost,127.0.0.1` (+ `WSL2_IP` if running on Linux)
`prod_v1`	`localhost,127.0.0.1`
`prod_v2`	`localhost,127.0.0.1` (+ `WSL2_IP` if running on Linux)

🗒️ When running on WSL2, the current IP is automatically added to ALLOWED_HOSTS using the refresh_allowed_hosts_wsl2.sh script.

Additional Details:
- Requests with a Host header not listed in ALLOWED_HOSTS will be blocked with a 400 Bad Request response.
- 127.0.0.1 is included alongside localhost in ALLOWED_HOSTS to ensure compatibility across operating systems, as some resolve localhost differently.
- The wildcard (*) can allow all hosts but is not recommended in production due to security risks.

API_VERSION

Description:

Defines the active version of the API. It ensures consistent API routing, maintains backward compatibility, and facilitates versioning across different environments.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: Dockerfile.demo (app and Celery workers images)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used to dynamically construct API paths within api_utils.exposed_endpoints.
  - Enables API documentation generation via drf-spectacular, ensuring that OpenAPI schemas reflect the current API version.
Current Setup in Different Environments:

Environment API_VERSION Value

demo v2

dev v2

prod_v1 v2

prod_v2 v2

Additional Details:
- API versioning is URL-based, meaning clients must explicitly include the version in their requests, e.g., /api/v2/.
- The variable allows seamless upgrades between API versions while maintaining backward compatibility.
- If a new API version is introduced, it must be added to the ALLOWED_VERSIONS list in config/settings.py:
```
REST_FRAMEWORK = {
    'ALLOWED_VERSIONS': ['v2'],
    'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.URLPathVersioning',
    'DEFAULT_VERSION': 'v2',
}
```
  - DEFAULT_VERSIONING_CLASS: Specifies how DRF extracts the API version, e.g, URLPathVersioning requires the version in the request URL - /api/v2/.
  - DEFAULT_VERSION: The fallback API version when none is provided.

DEBUG

Description:

Controls whether Django runs in debug mode.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to True)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - When set to True, Django provides detailed error messages and stack traces, useful for development.
  - When set to False, Django hides error details and applies stricter security settings.
- Dependencies:
  - If DEBUG=False, ALLOWED_HOSTS must be set, or Django raises a DisallowedHost error.
Current Setup in Different Environments:

Environment DEBUG Value

demo False

dev True

prod_v1 False

prod_v2 False

Additional Details:
- In production environments, DEBUG=False ensures:
  - Sensitive error details are hidden from users.
  - Certain security-related settings (e.g., HTTPS-only cookies, XSS protection) are automatically enforced.

MAX_USER_WALL_CONFIGS

Description:

Defines the maximum number of wall configurations a single user can store in the system.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (Defaults to 5)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in validation checks before storing new wall configurations.
  - If a user exceeds this limit, further uploads are blocked with an error response.
  - Directly impacts file validation logic in the_wall_api/serializers.py within WallConfigFileUploadSerializer.
Current Setup in Different Environments:

Environment MAX_USER_WALL_CONFIGS Value

demo 5

dev 5

prod_v1 5

prod_v2 5

Additional Details:
- This setting ensures fairness in resource allocation among users.
- The value may be adjusted depending on server capacity and performance constraints.
- If increased significantly, database query performance should be monitored for potential slowdowns.

PROJECT_MODE

Description:

Determines the active deployment mode of the application. It defines environment-specific configurations, affecting database access, logging, and containerized service behavior.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml (app, Celery Beat, and workers)
  - Development: env_utils.py (defaults to dev)
  - Production v1: docker-compose-prod-v1.yml
  - Production v2: docker-compose-prod-v2.yml
- Behavior:
  - Used in settings.py to:
    - Dynamically determine which .env file is loaded using load_dotenv, affecting all environment variables.
    - Configure database connections:
      - dev, prod_v1, and demo modes load credentials from .env files.
      - prod_v2 reads credentials from Docker secrets.
  - Defines the deployment mode for different environments (dev, demo, prod_v1, prod_v2) in Docker Compose files.
  - Testing:
    - In the test_celery_concurrent_tasks.py embedded Celery worker configuration:
      - Uses thread-based execution for dev mode.
      - Uses multiprocessing (prefork) in containerized environments (demo, prod_v1, prod_v2).
    - In test_celery_sequential_tasks.py on Linux:
      - Controls file permission handling when Celery workers create log folders.
- Dependencies:
  - In the_wall_api/utils/env_utils.py controls:
    - DB_HOST resolution in dev mode when running with Celery.
    - REDIS_PASSWORD injection in prod_v1 and Docker secret loading in prod_v2.
    - The correct CELERY_BROKER_URL configuration.
  - In batch_tests_runner.py:
    - Determines the log folder (LOGS_DIR_NAME).
Current Setup in Different Environments:

Environment PROJECT_MODE Value

demo demo

dev dev

prod_v1 prod_v1

prod_v2 prod_v2

SECRET_KEY

Description:

Defines the cryptographic key used by Django.
It must be kept secret and should never be hardcoded in the repository.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: Dockerfile.demo
  - Development: the_wall_api_dev.env
  - Production v1: django_secret_key_prod.env
  - Production v2: Docker secrets file
- Behavior:
  - Used in settings.py:
```
if PROJECT_MODE in ['dev', 'prod_v1', 'demo']:
    SECRET_KEY = os.getenv('SECRET_KEY')
else:
    SECRET_KEY = ''

if not SECRET_KEY:
  raise ImproperlyConfigured('The SECRET_KEY setting must not be empty. Set it in your environment variables.')
```
  - Required for:
    - Django cryptographic signing (e.g., password hashing, CSRF tokens, session management).
    - Ensuring application security by preventing certain types of attacks.
    - Generating cryptographic hashes used in various security mechanisms.
  - In prod_v2, the value is loaded from a Docker secrets file instead of an environment variable → settings.py:
```
elif PROJECT_MODE == 'prod_v2':
    SECRET_KEY = open(os.environ['DJANGO_SECRET_KEY_FILE']).read().strip()
```
- Dependencies:
  - The value is managed, based on PROJECT_MODE.

Additional Details:
- Must never be exposed in logs, error messages, or public repositories.
- In production, should be generated dynamically and stored securely.
- Changing the SECRET_KEY invalidates all Django sessions and can impact password hashes.

Return to Top

3.2 PostgreSQL Database Configuration

DB_CONNECT_TIMEOUT

Description:

Specifies the maximum time (in seconds) that the database connection attempt can take before timing out. This setting helps prevent long waits in case of unreachable database servers.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 10)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the database configuration inside settings.py to set the connect_timeout option for PostgreSQL connections:
```
DATABASES = {
    'default': {
        'OPTIONS': {
            'connect_timeout': int(os.getenv('DB_CONNECT_TIMEOUT', 10)),
        }
    }
}
```
  - Ensures fast failure if the database is unreachable, avoiding unnecessary delays in request processing.
- Dependencies:
  - Works in combination with DB_STATEMENT_TIMEOUT, which limits the execution time of SQL queries.
Current Setup in Different Environments:

Environment DB_CONNECT_TIMEOUT Value

demo 10

dev 10

prod_v1 10

prod_v2 10

Additional Details:
- This setting applies only to the initial connection attempt and does not impact long-running queries.
- Can be adjusted to fine-tune connection performance in different environments.
  A lower value is beneficial in environments where a database outage should be detected quickly.

DB_ENGINE

Description:

Defines the driver responsible for database communication.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to django.db.backends.postgresql)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the database configuration inside settings.py to specify the Django database engine:
```
DATABASES = {
    'default': {
        'ENGINE': os.getenv('DB_ENGINE', 'django.db.backends.postgresql'),
    }
}
```
Current Setup in Different Environments:

Environment DB_ENGINE Value

demo django.db.backends.postgresql

dev django.db.backends.postgresql

prod_v1 django.db.backends.postgresql

prod_v2 django.db.backends.postgresql

Additional Details:
- PostgreSQL is the recommended database due to its support for advanced indexing, transactions, and JSON fields.
- The SQLite backend is sometimes used for lightweight development/testing but lacks scalability.
- Switching engines requires modifying DATABASES settings and ensuring compatibility with the chosen backend.
- Other supported engines include:
  - SQLite (django.db.backends.sqlite3)
  - MySQL (django.db.backends.mysql)
  - Oracle (django.db.backends.oracle)

DB_HOST

Description:

Defines how the application resolves the database location, whether on a local machine or a networked environment.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to postgres)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the database configuration inside settings.py to configure the database connection:
```
DATABASES = {
    'default': {
        'HOST': os.getenv('DB_HOST', 'postgres'),
    }
}
```
  - dev → Uses DB_HOST=localhost because the application connects to the database from the host machine.
  - prod_v1, prod_v2, demo → Use DB_HOST=postgres, as all services communicate within a Docker network.
  - Used in the wait_for_postgres.py script to check database availability before application startup.
- Dependencies:
  - Set in the_wall_api/utils/env_utils.py based on PROJECT_MODE to ensure the correct DB_HOST value in containerized environments.
Current Setup in Different Environments:

Environment DB_HOST Value

demo postgres

dev localhost

prod_v1 postgres

prod_v2 postgres

Additional Details:
- In containerized deployments, typically references the database service name instead of localhost.
- Changes may require updates to networking settings, such as Docker Compose service names or DNS resolution.

DB_PORT

Description:

Defines the port number used for connecting to the PostgreSQL database server.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 5432)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the database configuration inside settings.py to set the database connection parameters:
```
DATABASES = {
    'default': {
        'PORT': os.getenv('DB_PORT', 5432),
    }
}
```
  - Determines the PostgreSQL instance the application connects to.
  - Used in Docker-based deployments, ensuring the Django app communicates with the correct database container.
  - Used in the wait_for_postgres.py script to check database availability before application startup.
Current Setup in Different Environments:

Environment DB_PORT Value

demo 5432

dev 5432

prod_v1 5432

prod_v2 5432

Additional Details:
- Default PostgreSQL port is 5432, but it can be changed if running multiple instances.
- Used in database connection checks within containerized environments.
- Changing this value requires updating docker-compose configurations and possibly firewall settings.

DB_STATEMENT_TIMEOUT

Description:

Defines the maximum execution time (in milliseconds) for SQL statements. If a query exceeds this duration, it is automatically terminated to prevent long-running transactions from affecting database performance.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 5000)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Applied in database configuration within settings.py using the statement_timeout parameter:
```
DATABASES = {
    'default': {
        'OPTIONS': {
            'options': f'-c statement_timeout={int(os.getenv("DB_STATEMENT_TIMEOUT", 5000))}',
        }
    }
}
```
  - Enforces a hard limit on SQL query execution time to prevent excessive resource usage.
- Dependencies:
  - Works alongside DB_CONNECT_TIMEOUT, which defines how long the database connection attempt can last before failing
Current Setup in Different Environments:

Environment DB_STATEMENT_TIMEOUT Value

demo 5000 (ms)

dev 5000 (ms)

prod_v1 5000 (ms)

prod_v2 5000 (ms)

Additional Details:
- Helps mitigate the risk of locking issues and long-running queries that could degrade database performance.
- Recommended for applications where response time is critical, as it forces slow queries to fail instead of blocking other operations.
- Can be adjusted per deployment to balance query performance and timeout constraints.

POSTGRES_DB

Description:

Specifies the name of the PostgreSQL database used by the application.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml (postgres service), Dockerfile.demo (app and Celery Beat + workers images)
  - Development: postgres_dev.env
  - Production v1: postgres_prod.env
  - Production v2: Docker secrets file
- Behavior:
  - Used in the database configuration inside settings.py:
```
DATABASES['default']['NAME'] = os.getenv('POSTGRES_DB')
```
  - Defines the database name that PostgreSQL will create at initialization and use for all connections.
  - In prod_v2, the value is loaded from a Docker secrets file instead of an environment variable → settings.py:
```
DATABASES['default']['NAME'] = open(os.environ['POSTGRES_DB_FILE']).read().strip()
```
💡 See the Development and Production Docker network diagrams for a visual representation of POSTGRES_DB usage.

POSTGRES_PASSWORD

Description:

Defines the password required for authentication to the PostgreSQL database.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml (postgres service), Dockerfile.demo (app and Celery Beat + workers images)
  - Development: postgres_dev.env
  - Production v1: postgres_prod.env
  - Production v2: Docker secrets file
- Behavior:
  - Used in the database configuration inside settings.py:
```
DATABASES['default']['PASSWORD'] = os.getenv('POSTGRES_PASSWORD')
```
  - Required for PostgreSQL authentication when the application connects to the database.
  - In prod_v2, the value is loaded from a Docker secrets file instead of an environment variable → settings.py:
```
DATABASES['default']['PASSWORD'] = open(os.environ['POSTGRES_PASSWORD_FILE']).read().strip()
```
💡 Refer to the Development and Production Docker network diagrams for a visual representation of POSTGRES_PASSWORD usage.

POSTGRES_USER

Description:

Specifies the username of the PostgreSQL database user that the application will use for authentication.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml (postgres service), Dockerfile.demo (app and Celery Beat + workers images)
  - Development: postgres_dev.env
  - Production v1: postgres_prod.env
  - Production v2: Docker secrets file
- Behavior:
  - Used in the database configuration inside settings.py:
```
DATABASES['default']['USER'] = os.getenv('POSTGRES_USER')
```
  - Defines the username for database authentication.
  - In prod_v2, the value is loaded from a Docker secrets file instead of an environment variable → settings.py:
```
DATABASES['default']['USER'] = open(os.environ['POSTGRES_USER_FILE']).read().strip()
```
💡 Check the Development and Production Docker network diagrams for a visual representation of POSTGRES_USER usage.

PostgreSQL - Database Configuration

Return to Top

3.3 Redis Caching and Message Queue Configuration

REDIS_CACHE_TRANSIENT_DATA_TIMEOUT

Description:

Defines the expiration time (in seconds) for temporary cache entries in Redis.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - Used in the Redis caching operations inside the_wall_api/utils/storage_utils.py:
```
def set_redis_cache(redis_cache_key: str, redis_cache_value: Any) -> None:
    
    cache.set(redis_cache_key, redis_cache_value, timeout=REDIS_CACHE_TRANSIENT_DATA_TIMEOUT)
```
  - Applied to cost and usage requests, ensuring that frequently accessed data remains available for a limited time.
  - Controls the lifespan of transient cache entries, preventing outdated data from persisting indefinitely.
  - Unlike Celery-related Redis keys, which are persistent, transient cache data is subject to automatic expiration.
Current Setup in Different Environments:

Environment REDIS_CACHE_TRANSIENT_DATA_TIMEOUT Value

demo 604800 (7 days)

dev 604800 (7 days)

prod_v1 604800 (7 days)

prod_v2 604800 (7 days)

Additional Details:
- Transient cache expiration helps balance performance and data accuracy, ensuring timely updates while reducing unnecessary computations.

REDIS_DB_NUMBER

Description:

Specifies the Redis database index used for caching application data.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo: env_utils.py (Defaults to 0)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Ensures that different Redis databases are used for caching application data and Celery tasks.
  - Ensures testing is isolated from production data.
- Dependencies:
  - The Redis database index in the REDIS_URL is dynamically set in env_utils.py based on ACTIVE_TESTING evaluation:
```
if not ACTIVE_TESTING:
    REDIS_URL = REDIS_URL.replace('REDIS_DB_NUMBER', REDIS_DB_NUMBER)
else:
    REDIS_URL = REDIS_URL.replace('REDIS_DB_NUMBER', '1')
```
💡 Refer to the Development and Production Docker network diagrams for a visual representation of REDIS_DB_NUMBER usage.
Current Setup in Different Environments:

Environment REDIS_DB_NUMBER Value

demo 0

dev 0

prod_v1 0

prod_v2 0

REDIS_DB_NUMBER_CELERY

Description:

Specifies the Redis database index used for Celery task queuing and result storage.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo: env_utils.py (Defaults to 2)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Celery broker configuration inside env_utils.py:
```
CELERY_BROKER_URL = REDIS_URL.replace('REDIS_DB_NUMBER', REDIS_DB_NUMBER_CELERY)
```
  - Ensures separate Redis databases for caching and Celery.
💡 Check the Development and Production Docker network diagrams for a visual representation of REDIS_DB_NUMBER_CELERY usage.
Current Setup in Different Environments:

Environment REDIS_DB_NUMBER_CELERY Value

demo 2

dev 2

prod_v1 2

prod_v2 2

REDIS_PASSWORD

Description:

Defines the authentication password for the Redis instance used by the application.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo: (Not used)
  - Development: (Not used)
  - Production v1: redis_prod.env
  - Production v2: Docker secrets file
- Behavior:
  - Used in the Redis connection configuration inside env_utils.py:
```
if PROJECT_MODE == 'prod_v1':
    REDIS_PASSWORD = os.getenv('REDIS_PASSWORD', '')
    REDIS_URL = REDIS_URL.replace('REDIS_PASSWORD', REDIS_PASSWORD)
elif PROJECT_MODE == 'prod_v2':
    REDIS_PASSWORD = open(os.environ['REDIS_PASSWORD_FILE']).read().strip()
    REDIS_URL = REDIS_URL.replace('REDIS_PASSWORD', REDIS_PASSWORD)
```
  - Ensures secure Redis access in production while allowing password-less local development.
  - In prod_v2, the value is loaded from a Docker secrets file instead of an environment variable.
- Dependencies:
  - There is a password placeholder in the REDIS_URL env. variable, which is dynamically replaced with the actual password, based on the PROJECT_MODE value.
  - The utilization of REDIS_PASSWORD depends on whether the Redis service in docker-compose is started with the --requirepass flag. If this flag is set, authentication is enforced, requiring the password in REDIS_URL.
💡 See the Production Docker network diagram for a visual representation of REDIS_PASSWORD usage.

Additional Details:
- The REDIS_PASSWORD is optional - the default Redis configuration allows connections without credentials.

REDIS_SOCKET_CONNECT_TIMEOUT

Description:

Defines the timeout (in seconds) for establishing a socket connection to the Redis server.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (Defaults to 2)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Redis connection configuration inside settings.py:
```
REDIS_SOCKET_CONNECT_TIMEOUT = int(os.getenv('REDIS_SOCKET_CONNECT_TIMEOUT', 2))

CACHES = {
    'default': {
        'OPTIONS': {
            'SOCKET_CONNECT_TIMEOUT': REDIS_SOCKET_CONNECT_TIMEOUT
        }
    }
}
```
  - Sets the maximum duration (in seconds) the system waits when attempting to connect to Redis.
- Dependencies:
  - This timeout applies only to the initial connection attempt, while read/write operation timeouts are managed separately via REDIS_SOCKET_TIMEOUT.
Current Setup in Different Environments:

Environment REDIS_SOCKET_CONNECT_TIMEOUT Value

demo 2

dev 1

prod_v1 1

prod_v2 1

Additional Details:
- A low value prevents long delays if Redis is unavailable but might cause issues with slow connections.

REDIS_SOCKET_TIMEOUT

Description:

Specifies the timeout (in seconds) for Redis socket read/write operations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (Defaults to 2)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Redis caching configuration inside settings.py:
```
REDIS_SOCKET_TIMEOUT = int(os.getenv('REDIS_SOCKET_TIMEOUT', 2))

CACHES = {
    'default': {
        'OPTIONS': {
            'SOCKET_TIMEOUT': REDIS_SOCKET_TIMEOUT
        }
    }
}
```
  - Defines the maximum time Redis waits when performing read/write operations before timing out.
- Dependencies:
  - This timeout applies only to read/write operations, while connection timeouts are managed separately via REDIS_SOCKET_CONNECT_TIMEOUT.
Current Setup in Different Environments:

Environment REDIS_SOCKET_TIMEOUT Value

demo 2

dev 1

prod_v1 1

prod_v2 1

Additional Details:
- Helps prevent hanging connections when Redis is under high load or facing network issues.
- A higher value may be needed for long-running Redis commands, but it can also increase delays in handling failures.
- Lowering this value may help detect unresponsive Redis instances more quickly but could lead to premature timeouts in high-latency networks.

REDIS_URL

Description:

Defines the connection URL for the Redis instance used for caching and Celery task queuing.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo: Dockerfile.demo (app and Celery workers images)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Redis connection and caching configuration inside env_utils.py:
```
REDIS_URL = os.getenv('REDIS_URL', 'redis://redis:6379/REDIS_DB_NUMBER')

if PROJECT_MODE == 'prod_v1':
    REDIS_PASSWORD = os.getenv('REDIS_PASSWORD', '')
    REDIS_URL = REDIS_URL.replace('REDIS_PASSWORD', REDIS_PASSWORD)
elif PROJECT_MODE == 'prod_v2':
    REDIS_PASSWORD = open(os.environ['REDIS_PASSWORD_FILE']).read().strip()
    REDIS_URL = REDIS_URL.replace('REDIS_PASSWORD', REDIS_PASSWORD)
```
  - Ensures secure Redis access in production while allowing local development without authentication.
- Dependencies:
  - Dynamically configures database numbers for caching and Celery tasks using REDIS_DB_NUMBER and REDIS_DB_NUMBER_CELERY.
  - Supports optional authentication via REDIS_PASSWORD.
💡 Refer to the Development and Production Docker network diagrams for a visual representation of REDIS_URL usage.

Current Setup in Different Environments:

Environment	REDIS_URL Value
`demo`	`redis://redis:6379/REDIS_DB_NUMBER`
`dev`	`redis://localhost:6379/REDIS_DB_NUMBER`
`prod_v1`	`redis://:REDIS_PASSWORD@redis:6379/REDIS_DB_NUMBER`
`prod_v2`	`redis://:REDIS_PASSWORD@redis:6379/REDIS_DB_NUMBER`

Additional Details:
- The Redis URL format follows redis://[:password]@host:port/db_number, where password is optional depending on the authentication setup.
- In production, authentication is required if Redis is started with --requirepass, and the password must be included in the URL.
- For local development, Redis typically runs without authentication, allowing a simpler redis://localhost:6379/0 format.
- The database number (db_number) in the URL determines logical isolation within a single Redis instance.

Redis - In-Memory Cache & Celery Message Broker

Return to Top

3.4 Celery Task Processing Configuration

CELERY_BROKER_URL

Description:

Defines the connection URL for the Celery message broker, which is used to queue and distribute background tasks.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo, Development, and Production v1, v2: env_utils.py (Transforms the REDIS_URL)
- Behavior:
  - Used in the Celery configuration inside env_utils.py:
```
CELERY_BROKER_URL = REDIS_URL.replace('REDIS_DB_NUMBER', REDIS_DB_NUMBER_CELERY)

if PROJECT_MODE == 'dev' and STARTED_FROM_CELERY_SERVICE:
    CELERY_BROKER_URL = CELERY_BROKER_URL.replace('localhost', 'redis')
```
  - Ensures separate Redis database instances are used for general caching and Celery task queuing.
- Dependencies:
  - Celery’s message broker connection is derived from REDIS_URL, ensuring consistent Redis-based task queuing.
  - PROJECT_MODE & service resolution:
    - In local development (dev), the broker URL uses localhost, assuming Redis is running inside a Docker container on the local machine.
    - In dockerized environments (demo, prod_v1, prod_v2), localhost is dynamically replaced with redis to correctly reference the Redis service inside the Docker network.
  - REDIS_DB_NUMBER_CELERY:
    - Ensures Celery tasks use a dedicated Redis database (db=2), avoiding conflicts with caching.
  - LIGHT_CELERY_CONFIG:
    - The CELERY_BROKER_URL is dynamically set in config/celery.py when lightweight Celery configuration is enabled:
      if LIGHT_CELERY_CONFIG: connections_settings = configure_connections_settings() os.environ['CELERY_BROKER_URL'] = connections_settings['CELERY_BROKER_URL']
  - Supports optional authentication via REDIS_PASSWORD.
💡 For a visual representation of how CELERY_BROKER_URL is utilized, refer to the Development and Production Docker network diagrams.
Current Setup in Different Environments:

Environment CELERY_BROKER_URL Value

demo redis://redis:6379/2

dev redis://localhost:6379/2

prod_v1 redis://:REDIS_PASSWORD@redis:6379/2

prod_v2 redis://:REDIS_PASSWORD@redis:6379/2

CELERY_RESULT_BACKEND

Description:

Defines the storage location for Celery task results, enabling retrieval of task execution status and return values.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo, Development, and Production v1, v2: env_utils.py (Takes the value of REDIS_URL)
- Behavior:
  - Used in the Celery configuration inside env_utils.py:
```
CELERY_RESULT_BACKEND = CELERY_BROKER_URL
```
  - Enables temporary storage of Celery task results in Redis, using the same database as the broker (Redis DB2). It allows the application to retrieve task outcomes and supports graceful abort signals for wall construction tasks.
- Dependencies:
  - Task results in Redis are temporary and expire based on CELERY_RESULT_EXPIRES, preventing unnecessary memory usage.
  - CELERY_RESULT_BACKEND is dynamically set in config/celery.py when lightweight Celery configuration (LIGHT_CELERY_CONFIG) is enabled:
```
if LIGHT_CELERY_CONFIG:
    connections_settings = configure_connections_settings()
    os.environ['CELERY_RESULT_BACKEND'] = connections_settings['CELERY_RESULT_BACKEND']
```
Current Setup in Different Environments:

Environment CELERY_RESULT_BACKEND Value

demo Takes the value of REDIS_URL

dev Takes the value of REDIS_URL

prod_v1 Takes the value of REDIS_URL

prod_v2 Takes the value of REDIS_URL

Additional Details:
- Celery does not require a result backend unless task result tracking is explicitly needed.
- Tasks with ignore_result=True do not generate result records, optimizing performance for fire-and-forget workloads.

CELERY_RESULT_EXPIRES

Description:

Defines the expiration time (in seconds) for Celery task results stored in the result backend.
Usage:
- Defined in: env_utils.py
- Behavior:
  - Used in the Celery configuration inside env_utils.py:
```
result['CELERY_RESULT_EXPIRES'] = '600'
```
  - Specifies how long task results remain available before automatic expiration.
  - Ensures completed task metadata does not persist indefinitely, preventing unnecessary storage usage.
- Dependencies:
  - CELERY_RESULT_EXPIRES is assigned dynamically in config/celery.py when lightweight Celery configuration is enabled via LIGHT_CELERY_CONFIG.
```
if LIGHT_CELERY_CONFIG:
    connections_settings = configure_connections_settings()
    os.environ['CELERY_RESULT_EXPIRES'] = connections_settings['CELERY_RESULT_EXPIRES']
```
  - Works alongside CELERY_RESULT_BACKEND, defining how long stored results are retained.

Additional Details:
- The expiration time should be balanced based on task retrieval needs and storage constraints.

CELERY_TASK_PRIORITY

Description:

Defines the priority levels assigned to Celery tasks, influencing execution order in the task queue.
Usage:
- Defined in: settings.py
- Behavior:
  - Used in Celery task execution inside settings.py:
```
CELERY_TASK_PRIORITY = {
    'HIGH': 0,
    'MEDIUM': 5,
    'LOW': 9
}
```
  - The lower the numeric value, the higher the priority (i.e., 0 is the highest).
  - Applied in Celery tasks using the apply_async(priority=CELERY_TASK_PRIORITY['MEDIUM']) pattern.
  - Ensures that critical tasks have a higher chance of being processed earlier, while background or non-urgent tasks wait in the queue.

Additional Details:
- In Redis, priority is not strictly enforced, but higher-priority tasks are more likely to be dequeued earlier as they are pushed higher in the queue priority list.
- However, actual execution order is also influenced by CELERY_WORKER_PREFETCH_MULTIPLIER, which determines how many tasks a worker preloads before processing.
- This means that even if a higher-priority task enters the queue, a worker may still process previously prefetched lower-priority tasks first.

CELERY_WORKER_PREFETCH_MULTIPLIER

Description:

Defines how many tasks each Celery worker prefetches before processing, affecting task distribution and execution order.
Usage:
- Defined in: settings.py
- Behavior:
  - Used in the Celery worker configuration inside settings.py:
```
CELERY_WORKER_PREFETCH_MULTIPLIER = 1
```
    - Determines how many tasks a worker preloads before executing any of them.
- Dependencies:
  - Works alongside CELERY_TASK_PRIORITY, influencing actual task execution order.

Additional Details:
- If set to 1, each worker fetches only one task at a time, reducing the risk of priority inversion.
- If set to 0, prefetching is disabled, and workers fetch tasks dynamically when available.
- Higher values can be beneficial for high-throughput workloads but may lead to task starvation in priority-based queues.
- In Redis-based brokers, this setting influences how task priority is handled in practice, as prefetched tasks are executed before newly available higher-priority tasks.

LIGHT_CELERY_CONFIG

Description:

Enables a lightweight configuration mode for Celery Beat, allowing the Beat service to start without loading the full Django application.
Usage:
- Defined in: celery.py
- Loaded from:
  - Demo: docker-compose-demo.yml
  - Development: docker-compose-dev.yml
  - Production v1: docker-compose-prod-v1.yml
  - Production v2: docker-compose-prod-v2.yml
- Behavior:
  - Used in the Celery Beat startup configuration inside celery.py:
```
LIGHT_CELERY_CONFIG = os.getenv('LIGHT_CELERY_CONFIG', 'False') == 'True'
```
  - When enabled, Celery Beat:
    - Does not load the full Django application.
    - Only configures essential connection settings for scheduling tasks.
    - Reduces memory usage and startup time for the Celery Beat container.
  - This mode is not used for Celery workers, as they require the full Django context to execute tasks.

Additional Details:
- Used to optimize containerized deployments where the Celery Beat service is responsible for scheduling tasks but does not execute them.

STARTED_FROM_CELERY_SERVICE

Description:

Identifies whether the application is started from the Celery service.
Usage:
- Defined in: env_utils.py
- Loaded from:
  - Demo: (Not used)
  - Development: docker-compose-dev.yml
  - Production v1, v2: (Not used)
- Behavior:
  - Used in env_utils.py to ensure proper service resolution for the PostgreSQL and Celery services in dev mode:
```
STARTED_FROM_CELERY_SERVICE = os.getenv('STARTED_FROM_CELERY_SERVICE', 'False') == 'True'
```
    - PosgreSQL
      if PROJECT_MODE == 'dev' and STARTED_FROM_CELERY_SERVICE and DATABASES: DATABASES['default']['HOST'] = 'postgres'
    - Celery
      if PROJECT_MODE == 'dev' and STARTED_FROM_CELERY_SERVICE: CELERY_BROKER_URL = CELERY_BROKER_URL.replace('localhost', 'redis')
  - These settings ensure the PostgreSQL and Celery services are accessible both from the local machine and within the Docker network.

Celery - Asynchronous Task Processing

Return to Top

3.5 Wall Construction and Simulation

ICE_COST_PER_CUBIC_YARD

Description:

Defines the cost per cubic yard of ice (in Gold Dragon coins) used in wall construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 1900)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Profiles API endpoints responsible for returning wall construction progress data.
  - Cost-related values are dynamically calculated using the stored ice amounts and the current ICE_COST_PER_CUBIC_YARD value.
Current Setup in Different Environments:

Environment ICE_COST_PER_CUBIC_YARD Value

demo 1900

dev 1900

prod_v1 1900

prod_v2 1900

Additional Details:
- Adjusting ICE_COST_PER_CUBIC_YARD does not modify stored ice volume data in the database.
- Simulated wall constructions retain their original ice consumption values, but cost estimates update dynamically based on this setting.
- This approach ensures pricing consistency for new calculations while preserving historical simulation data.

ICE_PER_FOOT

Description:

Defines the amount of ice (in cubic yards) required per 1-foot increase in wall height.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 195)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in wall construction simulations to calculate the total ice volume based on the wall’s height.
  - The total ice required for a given wall is computed as:
```
total_ice_volume = (Sum of all section height increases) * ICE_PER_FOOT
```
  - Ice volume calculations are stored in the database, meaning changes to this value affect only new simulations and do not modify previously stored results.
Current Setup in Different Environments:

Environment ICE_PER_FOOT Value

demo 195

dev 195

prod_v1 195

prod_v2 195

Additional Details:
- Adjusting ICE_PER_FOOT does not impact historical simulation data — stored ice volume values remain unchanged.
- Any modifications influence newly created simulations only, preserving consistency in previously calculated results.

MAX_SECTION_HEIGHT

Description:

Defines the maximum allowed height (in feet) for a single wall profile section in simulations. Once a section reaches this height, no further height increments are applied.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 30)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Enforced during wall configuration validation. It takes place before any simulation begins, ensuring that invalid configurations are rejected early.
  - During each simulation step, if a section's height is less than MAX_SECTION_HEIGHT, it is incremented.
  - If a section reaches this limit, the assigned crew is realocated to another section or relieved if no unfinished sections remain.
Current Setup in Different Environments:

Environment MAX_SECTION_HEIGHT Value

demo 30

dev 30

prod_v1 30

prod_v2 30

Additional Details:
- This constraint ensures realistic construction progress, preventing sections from exceeding the configured structural limits.
- Any increase in MAX_SECTION_HEIGHT would also increase total ice amounts per section.

MAX_WALL_LENGTH

Description:

Defines the maximum allowed number of wall profiles in construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 300)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Enforced during wall configuration validation, ensuring that no wall exceeds this predefined length.
  - The check is done before any simulation begins, ensuring that invalid configurations are rejected early.
Current Setup in Different Environments:

Environment MAX_WALL_LENGTH Value

demo 300

dev 300

prod_v1 300

prod_v2 300

Additional Details:
- Changing MAX_WALL_LENGTH does not affect existing stored wall profiles — only new configurations are validated against the updated value.
- Increasing this value may impact performance, as longer walls require more processing power and storage.
- Used to ensure system stability, preventing excessive resource consumption from extremely large simulations.

MAX_WALL_PROFILE_SECTIONS

Description:

Defines the maximum number of sections allowed per wall profile in simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 2000)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Applied during wall configuration validation. The check takes place before any simulation starts, ensuring that configurations exceeding the limit are discarded before computational resources are allocated.
Current Setup in Different Environments:

Environment MAX_WALL_PROFILE_SECTIONS Value

demo 2000

dev 2000

prod_v1 2000

prod_v2 2000

Additional Details:
- This constraint ensures efficient memory usage by preventing overly large wall profiles.
- Increasing this value does not impact existing simulations but may increase processing time for new configurations.

Wall Construction Overview

Return to Top

3.6 Concurrency and Performance

CONCURRENT_SIMULATION_MODE

Description:

Defines the concurrency mode used for wall construction simulations. Determines whether simulations execute in a threaded or multiprocessing strategy.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Evaluated in wall_construction.py to:
    - Determine the simulation_type (SEQUENTIAL or CONCURRENT), based on the provided num_crews value.
    - Determine the appropriate build simulation calculation method.
  - Controls whether wall construction simulations execute using threading or multiprocessing in CONCURRENT simulation_type.
  - Impacts the synchronization mechanisms, as different execution modes alter how progress updates are managed:
    - Uses Events or Conditions depending on the concurrency mode to coordinate build progress between the separate crews.
    - Determines how and when crews are marked as finished, affecting when they are reassigned or released from the simulation.
Current Setup in Different Environments:

Environment CONCURRENT_SIMULATION_MODE Value

demo threading_v1

dev threading_v1

prod_v1 threading_v1

prod_v2 threading_v1

CPU_THREADS

Description:

Represents the total number of CPU threads available on the host system.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 8)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used exclusively for multiprocessing-based simulations.
- Dependencies:
  - Determines MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING using the formula:
```
MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING = CPU_THREADS - 2
```
    where 2 CPU threads are reserved for the main process.
Current Setup in Different Environments:

Environment CPU_THREADS Value

demo 8

dev 8

prod_v1 8

prod_v2 8

MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING

Description:

Defines the maximum number of crews that can work concurrently in multiprocessing-based wall construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to CPU_THREADS - 2)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Controls the maximum processes allocation for multiprocessing wall construction.
- Dependencies:
  - The value is calculated based on CPU_THREADS using the formula:
```
MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING = CPU_THREADS - 2
```
    - This ensures that two CPU threads remain reserved for system tasks.
  - Evaluated in wall_construction.py alongside MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING to determine whether CONCURRENT mode can be used in the current simulation:
```
if (
    num_crews > settings.MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING or
    sections_count > settings.MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING
):
    simulation_type = SEQUENTIAL
```
    - If the number of crews limit is exceeded, the simulation falls back to SEQUENTIAL mode
Current Setup in Different Environments:

Environment MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING Value

demo CPU_THREADS - 2

dev CPU_THREADS - 2

prod_v1 CPU_THREADS - 2

prod_v2 CPU_THREADS - 2

Additional Details:
- Ensures multiprocessing simulations do not overcommit CPU resources.
- Increasing CPU_THREADS increases the maximum allowed crews, but excessive values may impact system stability.

MAX_CONCURRENT_NUM_CREWS_THREADING

Description:

Defines the maximum number of crews that can work concurrently in threading-based wall construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 250)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Controls the maximum threads allocation for threading wall construction.
- Dependencies:
  - Evaluated in wall_construction.py alongside MAX_SECTIONS_COUNT_CONCURRENT_THREADING to determine whether CONCURRENT mode can be used in the current simulation:
```
if (
    num_crews > settings.MAX_CONCURRENT_NUM_CREWS_THREADING or
    sections_count > settings.MAX_SECTIONS_COUNT_CONCURRENT_THREADING
):
    simulation_type = SEQUENTIAL
```
    - If the number of crews limit is exceeded, the simulation falls back to SEQUENTIAL mode
Current Setup in Different Environments:

Environment MAX_CONCURRENT_NUM_CREWS_THREADING Value

demo 250

dev 250

prod_v1 250

prod_v2 250

Additional Details:
- Increasing this value allows more crews to work concurrently, but excessive values may cause higher context-switching overhead.

MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING

Description:

Defines the maximum number of sections that can be processed concurrently in multiprocessing-based wall construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 2000)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used to check whether multiprocessing is allowed for a given simulation.
  - Ensures that simulations do not overload system resources by limiting parallel section processing.
- Dependencies:
  - Applied in wall_construction.py alongside MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING to determine whether CONCURRENT mode can be used in the current simulation:
```
if (
    num_crews > settings.MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING or
    sections_count > settings.MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING
):
    simulation_type = SEQUENTIAL
```
    - If the sections count limit is exceeded, the simulation falls back to SEQUENTIAL mode
Current Setup in Different Environments:

Environment MAX_SECTIONS_COUNT_CONCURRENT_MULTIPROCESSING Value

demo 2000

dev 2000

prod_v1 2000

prod_v2 2000

Additional Details:
- Ensures multiprocessing-based simulations remain within system limits, preventing excessive memory and CPU consumption.
- Increasing this value allows more sections to be processed in parallel, but may introduce higher synchronization overhead.

MAX_SECTIONS_COUNT_CONCURRENT_THREADING

Description:

Defines the maximum number of sections that can be processed concurrently in threading-based wall construction simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 4000)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Controls the maximum threads allocation for threading wall construction.
- Dependencies:
  - Evaluated in wall_construction.py alongside MAX_CONCURRENT_NUM_CREWS_THREADING to determine whether CONCURRENT mode can be used in the current simulation:
```
if (
    num_crews > settings.MAX_CONCURRENT_NUM_CREWS_THREADING or
    sections_count > settings.MAX_SECTIONS_COUNT_CONCURRENT_THREADING
):
    simulation_type = SEQUENTIAL
```
    - If the sections count limit is exceeded, the simulation falls back to SEQUENTIAL mode
Current Setup in Different Environments:

Environment MAX_SECTIONS_COUNT_CONCURRENT_THREADING Value

demo 4000

dev 4000

prod_v1 4000

prod_v2 4000

Additional Details:
- Ensures threading-based simulations remain within system limits, preventing excessive memory and CPU consumption.
- Increasing this value allows more sections to be processed in parallel, but may introduce higher synchronization overhead.

MAX_SECTIONS_COUNT_FULL_RANGE_CACHING

Description:

Defines the maximum number of sections a wall configuration can have for full-range precomputation.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 100)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - On wall configuration upload:
    - If the sections count is within this limit, the system precomputes simulations for all num_crews values within the defined range and stores the results in the DB for fast retrieval.
    - If the sections count is above this limit, simulations are triggered on-demand per API request for the specified num_crews. Once computed, the result is cached in both Redis and the DB,
      but only for that specific num_crews value.
Current Setup in Different Environments:

Environment MAX_SECTIONS_COUNT_FULL_RANGE_CACHING Value

demo 100

dev 100

prod_v1 100

prod_v2 100

Additional Details:
- This setting ensures that small to medium wall configurations are precomputed for fast retrieval while larger configurations remain efficient by avoiding excessive precomputations.
- Increasing this value allows more configurations to be cached on upload, but may increase database storage usage.
- Decreasing this value results in more on-demand calculations, which may increase API response times for non-cached configurations.

MAX_SECTIONS_COUNT_SYNCHRONOUS_RESPONSE

Description:

Defines the maximum number of sections that can be processed synchronously before the system switches to asynchronous execution using Celery tasks.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 10000)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - If the number of sections in a wall configuration is below this threshold, the simulation is processed synchronously, returning the result immediately.
    - Results are stored in the DB and cached in Redis for fast retrieval in subsequent requests.
  - If the number of sections exceeds this limit:
    - The simulation request is deferred to a Celery background task.
    - The API returns HTTP 202 (Accepted), and the client must poll for results once processing is complete.
    - The calculation result is stored in the DB.
Current Setup in Different Environments:

Environment MAX_SECTIONS_COUNT_SYNCHRONOUS_RESPONSE Value

demo 10000

dev 10000

prod_v1 10000

prod_v2 10000

Additional Details:
- This threshold helps balance system load, preventing large computations from blocking API responses.
- If set too high, API response times may increase significantly.
- If set too low, more asynchronous tasks are created, increasing the Celery task queue load.

SECTION_COMPLETION_GRACE_PERIOD_MULTIPROCESSING

Description:

Defines a grace period (in seconds) applied before marking a section completion event in multiprocessing-based wall simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 0.05)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in concurrency_utils/multiprocessing_utils.py to:
    - Delay the section completion log entry, ensuring correct order in the simulation logs.
    - Introduce a controlled delay before marking a section as complete. This prevents potential deadlocks caused by waiting processes missing the event signal before resuming work.
  - Ensures that all workers have registered completion events before the next set of tasks is assigned.
Current Setup in Different Environments:

Environment SECTION_COMPLETION_GRACE_PERIOD_MULTIPROCESSING Value

demo 0.05

dev 0.05

prod_v1 0.05

prod_v2 0.05

Additional Details:
- Helps synchronize process event handling, particularly during end-of-day crew synchronization.
- Ensures correct log order and event synchronization when multiple processes handle concurrent section completions.
- Without a grace period, multiprocessing log entries may appear out of order, affecting debugging and data tracking.

SECTION_COMPLETION_GRACE_PERIOD_THREADING

Description:

Defines a grace period (in seconds) applied before marking a section completion event in threading-based wall simulations.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 0.04)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in concurrency_utils/threading_utils.py to:
    - Delay the section completion log entry, ensuring correct order in the simulation logs.
    - Introduce a controlled delay before marking a section as complete. This prevents potential deadlocks caused by waiting threads missing the event signal before resuming work.
  - Ensures that all worker threads register completion events before the next set of tasks is assigned.
Current Setup in Different Environments:

Environment SECTION_COMPLETION_GRACE_PERIOD_THREADING Value

demo 0.04

dev 0.04

prod_v1 0.04

prod_v2 0.04

Additional Details:
- Helps synchronize thread event handling, particularly during end-of-day crew synchronization.
- Ensures correct log order and event synchronization when multiple threads handle concurrent section completions.
- Without a grace period, threading log entries may appear out of order, affecting debugging and data tracking.

VERBOSE_MULTIPROCESSING_LOGGING

Description:

Controls whether detailed log entries are generated during wall simulations for both multiprocessing and threading modes.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to False)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Applied in concurrency_utils/multiprocessing_utils.py and concurrency_utils/threading_utils.py.
  - If enabled (True):
    - Logs detailed section progress updates, including daily height increments, assigned crews, and day tracking, as part of the build simulation logs.
    - Introduces a small log delay (0.01 - 0.02 sec) to ensure proper log sequencing for the verbose records in concurrent execution.
Current Setup in Different Environments:

Environment VERBOSE_MULTIPROCESSING_LOGGING Value

demo False

dev False

prod_v1 False

prod_v2 False
Additional Details:
- Verbose logging increases disk I/O and may slow down large simulations.
- Can generate big log files, making debugging harder.
- Generally recommended only for debugging purposes:
  - Debugging concurrency issues (e.g., log order inconsistencies).
  - Investigating race conditions in multiprocessing.
  - Analyzing crew assignments and end-of-day synchronizations.

Sequential and Concurrent Simulation Modes

Return to Top

3.7 Testing and Debugging Configuration

ACTIVE_TESTING

Description:

Determines whether the application is running in testing mode, ensuring that tests do not interfere with production data.
Usage:
- Defined in: env_utils.py (Dynamically set based on application execution).
- Behavior:
  - Set to True in env_utils.py if 'test' is detected in the command-line arguments (sys.argv):
```
ACTIVE_TESTING = True if 'test' in sys.argv else False
```
  - In utils/celery_task_utils.py when ACTIVE_TESTING=True, Celery tasks use test-specific aliases:
```
def import_wall_task(active_testing: bool):
    if not active_testing:
        from the_wall_api.tasks import create_wall_task
    else:
        from the_wall_api.tasks import create_wall_task_test as create_wall_task
```
    - This remaps imports to test-specific wrappers.
      The core task logic remains unchanged.
      Allows controlled task execution in tests without modifying production behavior.
    - Used additionally in:
      - celery_task_utils.import_wall_config_deletion_task()
      - storage_utils.import_orchestrate_wall_config_processing_task()
  - Introduces delays in specific test cases to simulate race conditions in concurrent Celery tasks:
```
if active_testing:
    sleep(1)
```
- Dependencies:
  - Impacts the REDIS_URL database selection for caching:
```
if not ACTIVE_TESTING:
    REDIS_URL = REDIS_URL.replace('REDIS_DB_NUMBER', REDIS_DB_NUMBER)
else:
    REDIS_URL = REDIS_URL.replace('REDIS_DB_NUMBER', '1')
```
    - A separate Redis database (db=1) is used for caching, isolating test data.

Additional Details:
- When Django's test suite runs (e.g., via manage.py test), it adds 'test' to the command-line arguments, activating ACTIVE_TESTING. This ensures the application runs in an isolated test environment with separate configurations.
- Ensures that test executions do not modify real application data.
- When running automated tests, the application automatically switches to testing mode without requiring manual configuration.
- Disables or modifies certain application features to create a controlled test environment.

SEND_EXPECTED_TEST_SUITE_ERRORS_TO_CELERY

Description:

Controls whether expected errors from the test suite are sent to Celery Worker 2 for background logging.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to False)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - When disabled (False), expected test suite errors:
    - Are not sent to Celery Worker 2.
    - Do not appear in logs or console output.
    - Are completely suppressed, preventing unnecessary logging of expected failures during test execution.
Current Setup in Different Environments:

Environment SEND_EXPECTED_TEST_SUITE_ERRORS_TO_CELERY Value

demo False

dev False

prod_v1 False

prod_v2 False

Additional Details:
- Helps maintain cleaner logs by preventing test-related errors from polluting primary worker logs.
- Reducing log noise makes it easier to spot real failures in Celery tasks.

TEST_LOGGING_LEVEL

Description:

Defines the logging verbosity for test execution, controlling which test results are logged.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: docker-compose-demo.yml
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in test_utils.py to determine which test results get logged:
```
if (
    TEST_LOGGING_LEVEL == 'ALL' or
    (TEST_LOGGING_LEVEL == 'PASSED' and passed) or
    (TEST_LOGGING_LEVEL in ['FAILED', 'SUMMARY'] and not passed) or
    (TEST_LOGGING_LEVEL in ['ERROR', 'SUMMARY'] and error_occurred)
):
    self.logger.info(f'Test method: {test_case_source}')
```
  - Configures logging verbosity during test execution:
    - ALL: Logs every test case.
    - FAILED: Logs only failed test cases.
    - PASSED: Logs only successful test cases.
    - ERROR: Logs only errors.
    - SUMMARY: Logs test summary information.
    - NO-LOGGING: Disables test logging entirely.
  - Works independently of standard application logging, applying only to the test suite.
  - By default if enabled, logs are always printed to the console.
- Dependencies:
  - Works alongside TEST_SUITE_FILE_LOGGING_ENABLED, which controls whether test logs should also be written to a file.
Current Setup in Different Environments:

Environment TEST_LOGGING_LEVEL Value

demo FAILED

dev FAILED

prod_v1 FAILED

prod_v2 FAILED
Additional Details:
- Helps reduce noise in logs by allowing fine-grained control over which test outcomes are recorded.
- Useful for debugging specific test failures while avoiding unnecessary log output.

TEST_SUITE_FILE_LOGGING_ENABLED

Description:

Controls whether test suite logs are written to a dedicated log file.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to False)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - If True, logs generated during test runs are persisted to a separate log file.
- Dependencies:
  - TEST_LOGGING_LEVEL controls the verbosity of test logging.
  - Test suite logs are stored in the TEST_SUITE_LOGS_DIR folder.
Current Setup in Different Environments:

Environment TEST_SUITE_FILE_LOGGING_ENABLED Value

demo False

dev False

prod_v1 False

prod_v2 False

Test Suite

Return to Top

3.8 Logging Configuration

BUILD_SIM_LOGS_ARCHIVE_RETENTION_DAYS

Description:

Specifies the number of days archived build simulation logs are retained before being automatically deleted.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py (defaults to 7)
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Used in the Celery Archive Cleanup Task to trigger automated deletion of archived logs.
- Dependencies:
  - Logs exceeding this retention period are removed from the simulations log archive folder (BUILD_SIM_LOGS_ARCHIVE_DIR) to free up storage.
Current Setup in Different Environments:

Environment BUILD_SIM_LOGS_ARCHIVE_RETENTION_DAYS Value

demo 7

dev 7

prod_v1 14

prod_v2 14

Additional Details:
- Ensures compliance with storage policies while preserving recent logs for debugging and audits.

BUILD_SIM_LOGS_ARCHIVE_DIR

Description:

Defines the folder path where archived build simulation logs are stored.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - The scheduled Celery Archive Task moves old archived build simulation logs to this folder.
- Dependencies:
  - The archive task archives logs from the LOGS_DIR_NAME/BUILD_SIM_LOGS_DIR and moves them to this directory, where they are retained for BUILD_SIM_LOGS_ARCHIVE_RETENTION_DAYS.

Current Setup in Different Environments:

Environment	BUILD_SIM_LOGS_ARCHIVE_DIR Value	Full path
`demo`	`archive`	`LOGS_DIR_NAME`/`BUILD_SIM_LOGS_DIR`/`archive`/
`dev`	`archive`	`LOGS_DIR_NAME`/`BUILD_SIM_LOGS_DIR`/`archive`/
`prod_v1`	`archive`	`LOGS_DIR_NAME`/`BUILD_SIM_LOGS_DIR`/`archive`/
`prod_v2`	`archive`	`LOGS_DIR_NAME`/`BUILD_SIM_LOGS_DIR`/`archive`/

Additional Details:
- Log files in this folder are compressed using gzip to reduce storage usage while keeping them accessible for debugging.

BUILD_SIM_LOGS_DIR

Description:

Defines the folder path where build simulation logs are stored.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - Build simulation logs are generated and stored in this folder.
- Dependencies:
  - The scheduled Celery Archive Task scans this folder and moves logs older than BUILD_SIM_LOGS_RETENTION_DAYS to LOGS_DIR_NAME/BUILD_SIM_LOGS_ARCHIVE_DIR.

Current Setup in Different Environments:

Environment	BUILD_SIM_LOGS_DIR Value	Full path
`demo`	`build_simulations`	`LOGS_DIR_NAME`/`build_simulations`
`dev`	`build_simulations`	`LOGS_DIR_NAME`/`build_simulations`
`prod_v1`	`build_simulations`	`LOGS_DIR_NAME`/`build_simulations`
`prod_v2`	`build_simulations`	`LOGS_DIR_NAME`/`build_simulations`

Additional Details:
- Each log file records timestamps, completed sections, and crew release events.

BUILD_SIM_LOGS_RETENTION_DAYS

Description:

Specifies the number of days build simulation logs are retained before being archived.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: settings.py
  - Development: the_wall_api_dev.env
  - Production v1, v2: the_wall_api_prod.env
- Behavior:
  - Build simulation logs are stored for the duration of this retention period.
- Dependencies:
  - Any logs that have exceeded the retention period are archived by the scheduled Celery Archive Task and moved from BUILD_SIM_LOGS_DIR to BUILD_SIM_LOGS_ARCHIVE_DIR.
Current Setup in Different Environments:

Environment BUILD_SIM_LOGS_RETENTION_DAYS Value

demo 1

dev 1

prod_v1 7

prod_v2 7

Additional Details:
- Logs in dev are retained for a shorter period (1 day) to minimize storage usage while ensuring fresh logs are available for debugging.
- In prod_v1,2, the logs are kept longer (7 days) to maintain a longer audit trail for deeper analysis and operational monitoring, balancing storage constraints across environments.

ERROR_LOGS_DIR

Description:

Defines the folder path where application error logs are stored.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - The logging system dynamically writes error logs to categorized subfolders of this folder:
    - /caching/
    - /celery_tasks/
    - /wall_configuration/
    - /wall_creation/

Current Setup in Different Environments:

Environment	ERROR_LOGS_DIR Value	Full path
`demo`	`errors`	`LOGS_DIR_NAME`/`errors`
`dev`	`errors`	`LOGS_DIR_NAME`/`errors`
`prod_v1`	`errors`	`LOGS_DIR_NAME`/`errors`
`prod_v2`	`errors`	`LOGS_DIR_NAME`/`errors`

LOGS_DIR_NAME

Description:

Defines the base folder for all log storage in the application.
It serves as the root folder for multiple log folders across all environments.
Usage:
- Defined in: settings.py
- Loaded from:
  - Demo: Dockerfile.demo
  - Development: the_wall_api_dev.env
  - Production v1, v2: Dockerfile.prod
- Behavior:
  - Used in settings.py:
```
LOGS_DIR_NAME = os.getenv('LOGS_DIR_NAME', 'logs')
```
    - Serves as the root folder for all log folders.
  - In the demo and prod_v1,2 environments the environment variable:
    - Is used in Dockerfile.demo and Dockerfile.prod for logs folder creation.
    - Ensures that the logs folder is created iside the container and has correct ownership:
      ENV LOGS_DIR_NAME=logs RUN mkdir -p /app/${LOGS_DIR_NAME} && chown -R appuser:appuser /app
    - Also ensures correct ownership for Celery-related folders.
      RUN mkdir -p /var/lib/celery/data && \ mkdir -p /app/${LOGS_DIR_NAME} && \ chown -R appuser:appuser /app && \ chown -R appuser:appuser /var/lib/celery/data
  - In the dev environment this variable:
    - Is loaded from the_wall_api_dev.env in the docker-compose-dev+migrations.sh script to be:
      - Used in the creation of the logs folder if not existing.
      - Exported as an environment variable, to be used in the [docker-compose-dev.yml](Docker-Local-Development#3-docker-compose-services.
    - In docker-compose-dev.yml the variable is used in the build section:
      build: args: LOGS_DIR_NAME: ${LOGS_DIR_NAME:-logs}
      - It is passed as a build argument to Dockerfile.dev to be used for the same purpose, as described in the demo and prod_v1,2 environments section.
- Dependencies:
  - BUILD_SIM_LOGS_DIR, BUILD_SIM_LOGS_ARCHIVE_DIR, ERROR_LOGS_DIR, and TEST_SUITE_LOGS_DIR:
    - Subfolders created under LOGS_DIR_NAME for different types of logs.
  - APP_USER_ID, APP_GROUP_ID, andAPP_GROUP_NAME (Linux deployment):
    - Ensure the logs folder remains accessible to both the host application and Celery Worker 1 /(inside the container)*.
Current Setup in Different Environments:

Environment LOGS_DIR_NAME Value

demo logs

dev logs

prod_v1 logs

prod_v2 logs

Additional Details:
- On Linux development environment (dev) the logs folder is part of the bind-mounted application directory (./logs), making logs persist across container restarts without an explicit volume mount.

TEST_SUITE_LOGS_DIR

Description:

Defines the folder path where logs related to test suite execution are stored.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - Stores logs generated during Django test suite executions.
- Dependencies:
  - The TEST_SUITE_LOGS_FILE is stored in this folder.
  - Works alongside TEST_SUITE_FILE_LOGGING_ENABLED which controls whether logs are written to disk or only displayed in the console.

Current Setup in Different Environments:

Environment	TEST_SUITE_LOGS_DIR Value	Full path
`demo`	`test_suite`	`LOGS_DIR_NAME`/`test_suite`
`dev`	`test_suite`	`LOGS_DIR_NAME`/`test_suite`
`prod_v1`	`test_suite`	`LOGS_DIR_NAME`/`test_suite`
`prod_v2`	`test_suite`	`LOGS_DIR_NAME`/`test_suite`

TEST_SUITE_LOGS_FILE

Description:

Defines the log file path used to store logs related to test suite executions.
Usage:
- Defined in: settings.py
- Loaded from:
  - Currently loaded from settings.py, but structured to allow inclusion in .env files if needed.
- Behavior:
  - Stores logs generated during Django test suite executions.
- Dependencies:
  - The file is stored in the TEST_SUITE_LOGS_DIR folder.
  - Works alongside TEST_SUITE_FILE_LOGGING_ENABLED which controls whether logs are written to disk or only displayed in the console.

Current Setup in Different Environments:

Environment	TEST_SUITE_LOGS_FILE Value	Full path
`demo`	`test_suite.log`	`LOGS_DIR_NAME`/`TEST_SUITE_LOGS_DIR`/`test_suite.log`
`dev`	`test_suite.log`	`LOGS_DIR_NAME`/`TEST_SUITE_LOGS_DIR`/`test_suite.log`
`prod_v1`	`test_suite.log`	`LOGS_DIR_NAME`/`TEST_SUITE_LOGS_DIR`/`test_suite.log`
`prod_v2`	`test_suite.log`	`LOGS_DIR_NAME`/`TEST_SUITE_LOGS_DIR`/`test_suite.log`

Error Logging

Return to Top

3.9 Linux Development Configuration Variables

APP_USER_ID, APP_GROUP_ID, APP_GROUP_NAME

Description:

The environment variables APP_USER_ID, APP_GROUP_ID, and APP_GROUP_NAME are used to define the user and group settings for the Celery Worker 1 and Celery Beat containers in development mode. They ensure that processes inside these containers run with the correct non-root user permissions, allowing proper access to logs and shared resources.

⚠️ The permission policies described in this section are Linux-specific and do not apply to Windows deployments.
Usage:
- Defined in: Dockerfile.dev
- Loaded from:
  - Demo: (Not used)
  - Development: the_wall_api_dev.env
  - Production v1, v2: (Not used)
- Behavior:
  1. The docker-compose-dev+migrations.sh script loads the values of APP_USER_ID, APP_GROUP_ID, and APP_GROUP_NAME from the_wall_api_dev.env and exports them as environment variables.
  1. The exported values are passed as build arguments in docker-compose-dev.yml:
    - Ensuring that they are available during the container build process at runtime.
      build: args: APP_USER_ID: ${APP_USER_ID:-5678} APP_GROUP_ID: ${APP_GROUP_ID:-5678} APP_GROUP_NAME: ${APP_GROUP_NAME:-wall_app_group}
      - The default values (5678, wall_app_group) are used if the variables are not properly exported in the script referenced above.
  2. The passed build arguments are used inside Dockerfile.dev to define a system user and group inside the container:
    - Passed as ARG values during the build phase:
      ARG APP_USER_ID ARG APP_GROUP_ID ARG APP_GROUP_NAME
    - Used to create a dedicated user and group inside the container:
      RUN groupadd --gid ${APP_GROUP_ID} ${APP_GROUP_NAME} && \ adduser --uid ${APP_USER_ID} --gid ${APP_GROUP_ID} --disabled-login --gecos "" appuser
    - Ensuring that:
      - Celery Worker 1 runs under the correct user and retains the necessary read/write access to the logs folder, which is located inside the bind-mounted application directory.
      - New log files created by Celery Worker 1 inherit the correct group ownership, allowing both the containerized service and the host system to access them.
  3. The Celery Beat schedule is stored in a dedicated Docker volume, where group ownership ensures it remains accessible to the containerized process.
- Dependencies:
  - LOGS_DIR_NAME → Defines the base directory for log storage, located within the bind-mounted application folder.
- Current Setup in Different Environments:
  
  Environment APP_USER_ID APP_GROUP_ID APP_GROUP_NAME
  
  demo Not used Not used Not used
  
  dev 5678 5678 wall_app_group
  
  prod_v1 Not used Not used Not used
  
  prod_v2 Not used Not used Not used

Additional Details:
- The group defined by APP_GROUP_ID and APP_GROUP_NAME ensures that both Celery Worker 1 (inside the container) and the application running on the host can write logs to the logs folder, which is located within the bind-mounted application directory.
- This setup ensures that files created by either process remain accessible to both, preventing ownership mismatches when sharing data between the host and container.
- This approach is specific to Linux, as Windows handles mounted folder permissions differently.

Next: PostgreSQL - Database Configuration
Back to: API Throttling

Return to Top
DTS Index
Home Page

📌 Notable Technical Sections

System Overview

👤 Author

LinkedIn: Aleksandar Dimitrov
Yahoo: sasho_1983@yahoo.com

🗺️ Wiki Navigation

🔷 Home

🔷 Project Overview

🔷 Features

🔷 Dependencies

🔷 Quick Start

🔷 Installation Instructions Index

Toggle Index

Cross-Platform
- Quick App Setup
- Production Setup v1
Windows
Linux (WSL2)
Application Scripts Index

🔷 API Access Overview

🔷 Detailed Functional Specification

Toggle Index

🔶 Detailed Technical Specification

Toggle Index

Environment	DEBUG Value
`demo`	`False`
`dev`	`True`
`prod_v1`	`False`
`prod_v2`	`False`

Environment	PROJECT_MODE Value
`demo`	`demo`
`dev`	`dev`
`prod_v1`	`prod_v1`
`prod_v2`	`prod_v2`

Environment	DB_ENGINE Value
`demo`	`django.db.backends.postgresql`
`dev`	`django.db.backends.postgresql`
`prod_v1`	`django.db.backends.postgresql`
`prod_v2`	`django.db.backends.postgresql`

Environment	DB_HOST Value
`demo`	`postgres`
`dev`	`localhost`
`prod_v1`	`postgres`
`prod_v2`	`postgres`

Environment	DB_STATEMENT_TIMEOUT Value
`demo`	5000 (ms)
`dev`	5000 (ms)
`prod_v1`	5000 (ms)
`prod_v2`	5000 (ms)

Environment	REDIS_CACHE_TRANSIENT_DATA_TIMEOUT Value
`demo`	`604800` (7 days)
`dev`	`604800` (7 days)
`prod_v1`	`604800` (7 days)
`prod_v2`	`604800` (7 days)

Environment	CELERY_BROKER_URL Value
`demo`	`redis://redis:6379/2`
`dev`	`redis://localhost:6379/2`
`prod_v1`	`redis://:REDIS_PASSWORD@redis:6379/2`
`prod_v2`	`redis://:REDIS_PASSWORD@redis:6379/2`

Environment	CELERY_RESULT_BACKEND Value
`demo`	Takes the value of `REDIS_URL`
`dev`	Takes the value of `REDIS_URL`
`prod_v1`	Takes the value of `REDIS_URL`
`prod_v2`	Takes the value of `REDIS_URL`

Environment	ICE_COST_PER_CUBIC_YARD Value
`demo`	`1900`
`dev`	`1900`
`prod_v1`	`1900`
`prod_v2`	`1900`

Environment	MAX_WALL_PROFILE_SECTIONS Value
`demo`	`2000`
`dev`	`2000`
`prod_v1`	`2000`
`prod_v2`	`2000`

Environment	MAX_CONCURRENT_NUM_CREWS_MULTIPROCESSING Value
`demo`	`CPU_THREADS` - `2`
`dev`	`CPU_THREADS` - `2`
`prod_v1`	`CPU_THREADS` - `2`
`prod_v2`	`CPU_THREADS` - `2`

Environment	CONCURRENT_SIMULATION_MODE Value
`demo`	`threading_v1`
`dev`	`threading_v1`
`prod_v1`	`threading_v1`
`prod_v2`	`threading_v1`

Environment	MAX_SECTIONS_COUNT_CONCURRENT_THREADING Value
`demo`	`4000`
`dev`	`4000`
`prod_v1`	`4000`
`prod_v2`	`4000`

Environment	MAX_SECTIONS_COUNT_SYNCHRONOUS_RESPONSE Value
`demo`	`10000`
`dev`	`10000`
`prod_v1`	`10000`
`prod_v2`	`10000`

Environment	SECTION_COMPLETION_GRACE_PERIOD_MULTIPROCESSING Value
`demo`	`0.05`
`dev`	`0.05`
`prod_v1`	`0.05`
`prod_v2`	`0.05`

Environment	SECTION_COMPLETION_GRACE_PERIOD_THREADING Value
`demo`	`0.04`
`dev`	`0.04`
`prod_v1`	`0.04`
`prod_v2`	`0.04`

Environment	TEST_LOGGING_LEVEL Value
`demo`	`FAILED`
`dev`	`FAILED`
`prod_v1`	`FAILED`
`prod_v2`	`FAILED`

Environment	LOGS_DIR_NAME Value
`demo`	`logs`
`dev`	`logs`
`prod_v1`	`logs`
`prod_v2`	`logs`

Environment	APP_USER_ID	APP_GROUP_ID	APP_GROUP_NAME
`demo`	Not used	Not used	Not used
`dev`	`5678`	`5678`	`wall_app_group`
`prod_v1`	Not used	Not used	Not used
`prod_v2`	Not used	Not used	Not used

Environment Variables Index

Table of Contents

1. Overview

2. Environment Configuration Files

2.1. Folder Organization and Purpose

2.2 Folder structure:

3. Complete Index of Environment Variables

3.1 General Application Settings

3.2 PostgreSQL Database Configuration

3.3 Redis Caching and Message Queue Configuration

3.4 Celery Task Processing Configuration

3.5 Wall Construction and Simulation

3.6 Concurrency and Performance

3.7 Testing and Debugging Configuration

3.8 Logging Configuration

3.9 Linux Development Configuration Variables

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!