Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ONNX] Migrate to PT2 logging #106592

Closed
wants to merge 8 commits into from
Closed

Commits on Aug 4, 2023

  1. [ONNX] Migrate to PT2 logging

    Summary
    - 'dynamo_export' diagnostics now utilize PT2 artifact logger to manage
    the verbosity level of logs recorded into each diagnostic in SARIF log.
    The terminal logging is by default turned off. Setting environment variable
    'TORCH_LOGS="+onnx_diagnostics"' turns on terminal logging, as well as
    adjusts the verbosity level which overrides that in diagnostic options.
    Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    d66e267 View commit details
    Browse the repository at this point in the history
  2. fix ci on "[ONNX] Migrate to PT2 logging"

    Summary
    - 'dynamo_export' diagnostics now utilize PT2 artifact logger to manage
    the verbosity level of logs recorded into each diagnostic in SARIF log.
    The terminal logging is by default turned off. Setting environment variable
    'TORCH_LOGS="+onnx_diagnostics"' turns on terminal logging, as well as
    adjusts the verbosity level which overrides that in diagnostic options.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    3e36196 View commit details
    Browse the repository at this point in the history
  3. fix ci on "[ONNX] Migrate to PT2 logging"

    Summary
    - 'dynamo_export' diagnostics now utilize PT2 artifact logger to manage
    the verbosity level of logs recorded into each diagnostic in SARIF log.
    The terminal logging is by default turned off. Setting environment variable
    'TORCH_LOGS="+onnx_diagnostics"' turns on terminal logging, as well as
    adjusts the verbosity level which overrides that in diagnostic options.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    794757e View commit details
    Browse the repository at this point in the history
  4. fix ci, add some docs on "[ONNX] Migrate to PT2 logging"

    Summary
    - The 'dynamo_export' diagnostics leverages the PT2 artifact logger to handle the verbosity
    level of logs that are recorded in each SARIF log diagnostic. In addition to SARIF log,
    terminal logging is by default disabled. Terminal logging can be activated by setting
    the environment variable `TORCH_LOGS="onnx_diagnostics"`. When the environment variable
    is set, it also fixes logging level to `logging.DEBUG`, overriding the verbosity level
    specified in the diagnostic options.
    See `torch/_logging/__init__.py` for more on PT2 logging.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    5045b7d View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2023

  1. add unittests on "[ONNX] Migrate to PT2 logging"

    Summary
    - The 'dynamo_export' diagnostics leverages the PT2 artifact logger to handle the verbosity
    level of logs that are recorded in each SARIF log diagnostic. In addition to SARIF log,
    terminal logging is by default disabled. Terminal logging can be activated by setting
    the environment variable `TORCH_LOGS="onnx_diagnostics"`. When the environment variable
    is set, it also fixes logging level to `logging.DEBUG`, overriding the verbosity level
    specified in the diagnostic options.
    See `torch/_logging/__init__.py` for more on PT2 logging.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    990737a View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Update on "[ONNX] Migrate to PT2 logging"

    Summary
    - The 'dynamo_export' diagnostics leverages the PT2 artifact logger to handle the verbosity
    level of logs that are recorded in each SARIF log diagnostic. In addition to SARIF log,
    terminal logging is by default disabled. Terminal logging can be activated by setting
    the environment variable `TORCH_LOGS="onnx_diagnostics"`. When the environment variable
    is set, it also fixes logging level to `logging.DEBUG`, overriding the verbosity level
    specified in the diagnostic options.
    See `torch/_logging/__init__.py` for more on PT2 logging.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    18c7610 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. rebase w/ dispatcher change on "[ONNX] Migrate to PT2 logging"

    Summary
    - The 'dynamo_export' diagnostics leverages the PT2 artifact logger to handle the verbosity
    level of logs that are recorded in each SARIF log diagnostic. In addition to SARIF log,
    terminal logging is by default disabled. Terminal logging can be activated by setting
    the environment variable `TORCH_LOGS="onnx_diagnostics"`. When the environment variable
    is set, it also fixes logging level to `logging.DEBUG`, overriding the verbosity level
    specified in the diagnostic options.
    See `torch/_logging/__init__.py` for more on PT2 logging.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    32b6ad2 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Fix CI, rebase, address comments on "[ONNX] Migrate to PT2 logging"

    Summary
    - The 'dynamo_export' diagnostics leverages the PT2 artifact logger to handle the verbosity
    level of logs that are recorded in each SARIF log diagnostic. In addition to SARIF log,
    terminal logging is by default disabled. Terminal logging can be activated by setting
    the environment variable `TORCH_LOGS="onnx_diagnostics"`. When the environment variable
    is set, it also fixes logging level to `logging.DEBUG`, overriding the verbosity level
    specified in the diagnostic options.
    See `torch/_logging/__init__.py` for more on PT2 logging.
    - Replaces 'with_additional_message' with 'Logger.log' like apis.
    - Introduce 'LazyString', adopted from 'torch._dynamo.utils', to skip
    evaluation if the message will not be logged into diagnostic.
    - Introduce 'log_source_exception' for easier exception logging.
    - Introduce 'log_section' for easier markdown title logging.
    - Updated all existing code to use new api.
    - Removed 'arg_format_too_verbose' diagnostic.
    - Rename legacy diagnostic classes for TorchScript Onnx Exporter to avoid
    confusion.
    
    Follow ups
    - The 'dynamo_export' diagnostic now will not capture python stack
    information at point of diagnostic creation. This will be added back in
    follow up PRs for debug level logging.
    - There is type mismatch due to subclassing 'Diagnostic' and 'DiagnosticContext'
    for 'dynamo_export' to incorporate with PT2 logging. Follow up PR will
    attempt to fix it.
    - More docstrings with examples.
    
    [ghstack-poisoned]
    BowenBao committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    0e223a3 View commit details
    Browse the repository at this point in the history