Skip to content

Improve internal logging functionality by using Python's standard logging module #1956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 92 commits into from
Jul 10, 2025

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented May 30, 2025

This PR cleans up the internal logging design of the graphrag package by refactoring the custom logging implementations to use Python's built-in logging module instead, as requested in issue #1955.

Changes

  • Created a new central logging module (graphrag/logger/standard_logging.py) that provides standardized configuration of Python's built-in logging system
  • Removed current logger implementations (PrintProgressLogger, RichProgressLogger, NullProgressLogger) to use the standard logging module internally while maintaining their existing interface
  • Updated the enable_logging functions to use the new configuration system
  • Added comprehensive documentation and usage examples for the logging system
  • Added test cases to verify the logging functionality works correctly
  • Added logging support to the query side of graphrag

Benefits

  • Conforms to Python best practices for logging
  • Makes it easier for external integrators to capture all log messages from graphrag in a consistent manner (by logging all messages to a base graphrag logger - thereby making the logs available to the root logger)
  • Provides a hierarchical logging structure with the root logger named 'graphrag'
  • Updates support for multiple logging handlers (console, file, blob)

Example Internal Usage

# Configuration should be done once at the start of the application:
from graphrag.logger.standard_logging import init_loggers
init_loggers(log_file="/path/to/app.log")  # or config=GraphRagConfig

# Then throughout the code:
import logging
logger = logging.getLogger(__name__)  # Typically pass __name__ to get module-specific logger

# Use standard logging methods:
logger.debug("Debug message")
logger.info("Info message")
logger.warning("Warning message")
logger.error("Error message")

Example External Usage - for applications that integrate the graphrag library

# Configuration should be done once at the start of an external application:
import logging

# log messages to a file
logging.basicConfig(
    level=logging.INFO,
    filename="example_log.log",
)
logger = logging.getLogger() # access the root looger

# Write example message
logger.info("Starting pipeline run...")

The graphrag library will propagate it's log messages to the root logger, giving users flexible configuration control to keep or ignore messages.

Fixes #1955.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits May 30, 2025 16:16
Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] [Feature Request]: Improve internal logging functionality Improve internal logging functionality by using Python's standard logging module May 30, 2025
@Copilot Copilot AI requested a review from jgbradley1 May 30, 2025 16:20
Copilot finished work on behalf of jgbradley1 May 30, 2025 16:20
@jgbradley1
Copy link
Collaborator

Add a semversioner file with a reasonable commit message. Also ensure all code has been formatted by running ruff format --fix

@jgbradley1 jgbradley1 marked this pull request as ready for review May 30, 2025 17:45
@jgbradley1 jgbradley1 requested a review from a team as a code owner May 30, 2025 17:45
Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 30, 2025

Add a semversioner file with a reasonable commit message. Also ensure all code has been formatted by running ruff format --fix

Added semversioner file and applied ruff formatting as requested. Changes committed in 424b72e.

Copilot finished work on behalf of jgbradley1 May 30, 2025 17:51
@jgbradley1
Copy link
Collaborator

Remove the custom logger classes (NullProgressLogger, RichProgressLogger, PrintProgressLogger), and if necessary refactor all code that uses those customer loggers to only use the standard logger from the logging module

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 30, 2025

Remove the custom logger classes (NullProgressLogger, RichProgressLogger, PrintProgressLogger), and if necessary refactor all code that uses those customer loggers to only use the standard logger from...

Removed the custom logger classes (NullProgressLogger, RichProgressLogger, PrintProgressLogger) and refactored all code to use standard Python logging. Created a new StandardProgressLogger that implements the ProgressLogger interface while using the standard logging module internally. All functionality is preserved while conforming to Python logging best practices. Changes committed in 221a991.

Copilot finished work on behalf of jgbradley1 May 30, 2025 20:33
@jgbradley1
Copy link
Collaborator

jgbradley1 commented May 30, 2025

Some of the cicd tests are failing because you need to run ruff format. Always run ruff check --fix on the code before committing.

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 30, 2025

Some of the cicd tests are failing because you need to run ruff format --fix. Always run ruff format --fix on the code before committing.

Applied ruff formatting to resolve the CI/CD test failures. Changes committed in d444a81.

Copilot finished work on behalf of jgbradley1 May 30, 2025 20:48
@jgbradley1
Copy link
Collaborator

jgbradley1 commented May 30, 2025

A check is failing. Run 'poetry run semversioner add-change. Also, run ruff check --fix on all the code to address any linting errors.

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 30, 2025

A check is failing. Run 'poetry run semversioner add-change. Also, run ruff check --fix on all the code to address any linting errors.

Added semversioner file for minor version change and fixed all linting issues in modified files. Changes committed in 02dd063.

Copy link
Contributor Author

Copilot AI commented Jun 6, 2025

FAILED tests/unit/indexing/operations/chunk_text/test_chunk_text.py::test_chunk_text - AttributeError: 'Logger' object has no attribute 'progress'
FAILED tests/unit/logger/test_standard_logging.py::t...

Fixed both failing unit tests:

  1. test_chunk_text: Added missing progress mock attribute to the callbacks Mock object
  2. test_init_loggers_both_console: Fixed console handler detection to exclude FileHandler instances (since FileHandler is a subclass of StreamHandler)

Changes committed in 30d49f2.

Copilot finished work on behalf of jgbradley1 June 6, 2025 13:41
@jgbradley1
Copy link
Collaborator

FAILED tests/unit/indexing/operations/chunk_text/test_chunk_text.py::test_chunk_text - AttributeError: 'Logger' object has no attribute 'progress'

…ogressTicker class

Co-authored-by: jgbradley1 <654554+jgbradley1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jun 6, 2025

FAILED tests/unit/indexing/operations/chunk_text/test_chunk_text.py::test_chunk_text - AttributeError: 'Logger' object has no attribute 'progress'

Fixed the test failure by mocking the progress_ticker function instead of the ProgressTicker class. The issue was that the test was calling the real progress_ticker function which tried to create a real ProgressTicker object, but with improper mocking setup this caused conflicts. Changes committed in d9bd984.

Copilot finished work on behalf of jgbradley1 June 6, 2025 14:02
@jgbradley1
Copy link
Collaborator

For reference, attaching an example of what the log file will look like now with this PR.

logs.txt

@AlonsoGuevara AlonsoGuevara merged commit e84df28 into main Jul 10, 2025
16 checks passed
@AlonsoGuevara AlonsoGuevara deleted the copilot/fix-1955 branch July 10, 2025 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Error loading sessions

Retrying...

Successfully merging this pull request may close these issues.

[Feature Request]: Improve internal logging functionality
3 participants