Implement Application-Wide Logging System

[JoeKarow]

Overview

The Mango Tango CLI currently lacks a standardized logging mechanism, making debugging and monitoring difficult. This issue proposes implementing a robust, application-wide logging system with structured JSON output, dual streams (console for critical alerts, file for diagnostics), and user-configurable verbosity.

Goals

• Establish a centralized, reusable logging configuration
• Provide structured JSON logging for easy parsing by log management systems
• Separate critical alerts (console) from detailed diagnostics (rotating log files)
• Add user-controlled log verbosity via CLI flag
• Create an extensible pattern for adoption across all modules

Proposed Implementation

The implementation will introduce:

• A new --log-level CLI flag for controlling verbosity
• Centralized logging configuration in app/logging_config.py
• Automatic log file management with rotation (10MB max, 5 backups)
• Initial adoption in the n-gram analyzer as a demonstration

This will establish a foundation for better observability that can be incrementally adopted by other modules.

Acceptance Criteria

• Logging system routes messages correctly based on level
• --log-level CLI flag controls verbosity as expected
• Log files rotate automatically (10MB max, 5 backups)
• All log output is structured JSON
• Unit and integration tests included

---

DRAFT Implementation Plan

PRD: Application-Wide Logging System Implementation

Date: 2025-07-30

1. Overview

This document outlines the requirements for implementing a robust, application-wide logging system for the Mango Tango CLI. The current application lacks a standardized logging mechanism, making debugging and monitoring difficult. This implementation will introduce a structured, configurable, and production-ready logging framework that can be incrementally adopted across all modules.

2. Key Objectives

• Implement Structured Logging: All log output will be in JSON format for easy parsing and analysis by log management systems.
• Separate Log Outputs: Critical alerts (ERROR and CRITICAL) will be directed to the console (stderr), while detailed diagnostics (INFO and above) will be written to a log file.
• Implement Log Rotation: Log files will be automatically rotated to prevent them from consuming excessive disk space.
• User-Controlled Verbosity: Provide a CLI flag (--log-level) to allow users to easily adjust the logging verbosity for debugging purposes.
• Establish an Extensible Pattern: Create a centralized and reusable logging configuration that can be easily adopted by any module in the application.

3. Implementation Details

3.1. CLI Flag for Log Level

A new command-line argument will be added to mangotango.py to control the application's logging verbosity.

• File to Modify: mangotango.py
• Implementation: Use the argparse module.
• Flag: --log-level
• Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
• Default: INFO
• Behavior: The value of this flag will determine the minimum level of log messages that the application processes.

3.2. Centralized Logging Configuration

A new module will be created to house the centralized logging configuration.

• New File: app/logging_config.py
• Core Function: setup_logging(level=logging.INFO, log_file_path: Path)
• Implementation: The function will use logging.config.dictConfig to configure the root logger with two handlers.

3.2.1. Console Handler (for Critical Alerts)

• Type: logging.StreamHandler
• Level: ERROR
• Output: sys.stderr
• Formatter: python_json_logger.jsonlogger.JsonFormatter
• Purpose: To provide immediate visibility of critical issues on the console.

3.2.2. Rotating File Handler (for Diagnostics)

• Type: logging.handlers.RotatingFileHandler
• Level: INFO
• Path: A dynamically determined path in the user's data directory.
• Rotation Policy:
  • maxBytes: 10485760 (10MB)
  • backupCount: 5
• Formatter: python_json_logger.jsonlogger.JsonFormatter
• Purpose: To capture detailed information for debugging and analysis without cluttering the console.

3.3. Log File Path Management

The application will manage the log file's location automatically.

• File to Modify: mangotango.py
• Logic:
  1. Use the existing Storage class (which uses platformdirs) to get the application's user data directory.
  2. Create a logs subdirectory within the data directory if it does not exist.
  3. Construct the full log file path (e.g., ~/.local/share/MangoTango/logs/mangotango.log).
  4. Pass this path to the setup_logging() function at startup.

4. Affected Files

• New:
  • app/logging_config.py
• Modified:
  • mangotango.py: Add CLI flag and initialize logging.
  • requirements.txt: Add python-json-logger.

5. Success Criteria

• Correct Log Routing: ERROR and CRITICAL logs appear on the console; INFO, WARNING, ERROR, and CRITICAL logs appear in the log file.
• Log Rotation: Log files are automatically rotated when they reach 10MB, and no more than 5 backup files are kept.
• CLI Control: The --log-level flag correctly sets the logging verbosity. For example, setting --log-level DEBUG results in DEBUG messages appearing in the log file.
• Structured Logs: All logs written to the console and the file are in a valid JSON format.
• Graceful Integration: The existing progress reporting system continues to function as expected.

6. Testing Plan

• Unit Tests:
  • Create app/test_logging_config.py to test the setup_logging function.
  • Mock the handlers to verify that they are configured with the correct levels and formatters.
• Integration Tests:
  • Add tests to mangotango.py's test suite to verify that the --log-level flag works as expected.
• Manual Verification:
  • Manually inspect the log file to confirm its location, content (JSON format), and rotation behavior when a large volume of logs is generated.

[KristijanArmeni]

@JoeKarow This was merged already I see. Was that part of v0.8.0 release?

[JoeKarow]

No, it wasn't - I wanted to leave it in develop for a little bit to see if any bugs cropped up during the development cycle. This was merged right around the same time I finished this. (the release PR was #178)
I'm going to close the issue - for some reason it didn't get closed when the PR was merged.