How to capture INFO from logging?

[hubentu]

Greeting,

I tried to use py_capture_output to capture INFO from a python package, but the return is empty. I couldn't change the python functions. Could you please help to capture the info? Thanks!

Here is a simplified example,
log.py

import logging
import sys

def testFun():
    logging.basicConfig(level=20, stream=sys.stderr)
    logging.info("test")

 > testFun <- py_run_file("log.py")
 > py_capture_output(testFun$testFun())
 INFO  @ Fri, 21 Aug 2020 16:18:21: test
 [1] ""

[kevinushey]

I believe the issue here is that logging.basicConfig() is a no-op if the logging module has already been initialized. To re-initialize the logger, you need to clear any existing handlers and then re-initialize. For example, the following works for me:

library(reticulate)

py_run_string('
import logging
import sys
    
def testFun():

    for h in logging.root.handlers[:]:
        logging.root.removeHandler(h)
        h.close()
    
    logging.basicConfig(level=20, stream=sys.stderr)
    logging.info("test")
')

main <- import_main()
main$testFun()
py_capture_output(main$testFun())

Python 3.8 also adds a 'force' argument that basically does exactly this:

https://github.com/python/cpython/blob/d2b0ce6c35dc18d3f81b1fa73579d2f65fbc85af/Lib/logging/__init__.py#L1989-L1992

This might not interact well if other loggers are already initialized / used, but something of this form may be worth exploring.

[hubentu]

Hi @kevinushey , your solution works perfectly. By adding the chunk of your codes before invoking python function, py_capture_outpus did capture the INFOs. Thank so much for the quick response.

BTW, with your new commit, will this function be improved to capture the logs directly? Thanks!

[kevinushey]

Yes, I think things should "just work" in the development version of reticulate. :-)