Error Handling

This page outlines the structured error-handling mechanisms used within the application. It covers standardized response formats, logging strategies, and key utility functions that ensure robust error reporting and debugging.

Table of Contents

1. Overview
2. Error Handling Basics
   2.1 Example Error Response Structure
   2.2 Known Errors
   2.3 Unknown Errors
   2.4 Error Logging
    2.4.1 Folder Structure
    2.4.2 Django's Built-in Logging System
   2.5 Sequential Error Logging with Celery

1. Overview

The Wall API implements a centralized error-handling approach using DRF response objects, structured logging, and Celery-based asynchronous error logging. The goal is to:

• Standardize API error responses for consistency.
• Ensure all errors are logged and traceable.
• Differentiate between known and unknown errors.
• Offload error logging asynchronously, ensuring traceability without delaying API responses.

---

2. Error Handling Basics

• All errors are processed and returned in the error_utils.py module, ensuring API exceptions are logged and categorized correctly.

• Error messages are formatted using predefined templates from the message_themes folder.
  💡 For more details, refer to the Message Themes page.

2.1 Example Error Response Structure

{
    "error": "This wall configuration is already uploaded with config_id 'test_config_id'.",
    "error_details": {
        "request_params": {
            "config_id": "test_config_1"
        },
        "error_id": "17821"
    }
}

• Explanation of response Keys:
  • error: A human-readable description of the issue.
  • error_details (optional): Additional structured data providing more context.
    • request_params: Parameters associated with the request that triggered the error.
    • error_id: A unique identifier for the error instance, useful for debugging.

Return to Top

2.2 Known Errors

• Known errors are explicitly handled by the API using error_utils.handle_known_error() and return structured responses.

• Common known errors include:

  • Validation Errors (400): Invalid/missing input parameters, incorrect file formats, invalid wall configurations.
  • Authentication Errors (401): Invalid or missing authentication credentials.
  • Not Found Errors (404): Non-existing wall configuration files or construction progress data.
  • Conflict Errors (409): Ongoing calculations, duplicate uploads, invalid statuses.
  • Throttling Errors (429): Exceeding API rate limits.
  • Service Unavailable (503): File upload requests blocked by an ongoing wall configuration deletion.

• Each of these errors follows a standardized response pattern, varying in structure depending on the context.

Return to Top

2.3 Unknown Errors

• Unexpected failures are captured and logged using error_utils.handle_unknown_error().

• A generic (500) response is returned to avoid exposing internal details:

  {
      "error": "Wall construction simulation failed. Please contact support.",
      "error_details": {
          "request_params": {
              "day": 14,
              "profile_id": 1,
              "num_crews": 5,
              "config_id": "test_config_1"
          },
          "error_id": "14189",
          "tech_info": "Exception: Exception message."
      }
  }

  • The tech_info (optional) field is included to provide additional context about the exception.

• The erros are logged in dedicated error files with additional error traceback details.

💡 For a full reference to all possible error response structures, refer to the OpenAPI Schema.

Return to Top

2.4 Error Logging

Error logging is configured in settings.py

2.4.1 Folder Structure

• The ERROR_LOG_FILES_CONFIG dictionary in settings.py defines the complete folder structure for error logging, mapping each error category to its designated directory and log file.

  Folder Structure	Description
  ├── logs/	Root directory for all logs.
  ├── caching/	Stores caching-related error logs.
  └── caching_errors.log	Logs issues related to Redis caching failures.
  ├── celery_tasks/	Stores Celery task-related error logs.
  └── celery_tasks_errors.log	Captures errors from background Celery tasks.
  ├── wall_configuration/	Stores wall configuration error logs.
  └── wall_config_errors.log	Logs issues related to wall configuration processing.
  ├── wall_creation/	Stores wall construction error logs.
  └── wall_creation_errors.log	Captures failures in wall construction processes.

  
  

• Environment variables controlling the error logs folder structure:

  • LOGS_DIR_NAME - Name of the main logs folder inside the application.
  • ERROR_LOGS_DIR - Root directory where all error logs are stored.

Return to Top

2.4.2 Django's Built-in Logging System

• The LOGGING dictionary in settings.py configures Django’s built-in logging framework to facilitate error logging.

  Logger Name	Handlers	Log Level
  caching	caching_errors_file, console_logger	ERROR
  celery_tasks	celery_tasks_errors_file, console_logger	ERROR
  wall_configuration	wall_config_errors_file, console_logger	ERROR
  wall_creation	wall_creation_errors_file, console_logger	ERROR

  
  

• Log Handlers

  • Multiple handlers are defined in the LOGGING configuration to manage log output.

  • All log files are managed using logging.handlers.RotatingFileHandler, ensuring that logs are automatically rotated when they reach a predefined size limit.

    Handler Name	Type	Target Destination	Rotation	Formatter
    caching_errors_file	RotatingFileHandler	logs/errors/ caching_errors.log	5MB, 4 backups	json
    celery_tasks_errors_file	RotatingFileHandler	logs/errors/ celery_tasks_errors.log	5MB, 4 backups	json
    wall_config_errors_file	RotatingFileHandler	logs/errors/ wall_config_errors.log	5MB, 4 backups	json
    wall_creation_errors_file	RotatingFileHandler	logs/errors/ wall_creation_errors.log	5MB, 4 backups	json
    console_logger	StreamHandler	Console output	N/A	json_stdout

    ✔ The log rotation prevents excessive log file growth while maintaining historical logs for debugging.
    
    

• Log Formatters

  The LOGGING configuration defines custom json formatters to structure log messages consistently.

  Formatter Name	Description
  json	Formats logs as JSON with fields for timestamp, message, level, traceback, request info, and error ID.
  json_stdout	Similar to json, but formatted with 4-space indentation for improved readability in console output.

  
  
  • utils.custom_json_formatter.CustomJsonFormatter:
    • Extends JsonFormatter to customize JSON log formatting.
    • Renames fields:
      • asctime → timestamp
      • levelname → level
    • Enforces a structured field order:
      • error_id, level, timestamp, request_info, message, traceback
      
      

✔ This logging setup centralizes error tracking, ensuring consistency across all error categories while preventing log file overgrowth.

Return to Top

2.5 Sequential Error Logging with Celery

Error logging is processed asynchronously using Celery Worker 2, ensuring sequential handling of log entries without blocking API responses.

• Error Logging Task

  • All errors are logged via the log_error_task, which is sent to Celery Worker 2.
  • Worker 2 processes these log entries sequentially, preventing race conditions and ensuring ordered error tracking.

• Error ID Generation

  • Each error entry is assigned a unique error_id for traceability.
  • The unknown_errors_counter sets the value of error_id, ensuring each logged instance receives a unique, sequential identifier.

• Global unknown_errors_counter in Redis

  • A persistent unknown_errors_counter is stored in Redis (DB2) to track system-wide unexpected errors.
  • The counter is incremented atomically, preventing concurrency issues when multiple processes log errors simultaneously.
  • Unlike transient cache entries, this counter is not subject to TTL eviction, ensuring long-term error tracking.

✔ The Celery-based approach ensures that logging remains efficient, structured, and non-blocking while maintaining data integrity across concurrent API operations.

---

Next:   Message Themes
Back to: Profiles Endpoints

Return to Top
DTS Index
Home Page

---