Custom logger doesn't work with created resources (like producers) #194
Comments
Samreay
changed the title
|
@Samreay If you just want to set the log level, a possible workaround is to pass the |
|
Thanks @merlimat, can confirm that import logging
import pulsar
pulsar_logger = pulsar.ConsoleLogger(log_level=pulsar.LoggerLevel.Error)
logging.basicConfig(level=logging.INFO, format="{message}", style="{")
client = pulsar.Client("pulsar://localhost:6650", logger=pulsar_logger)
producer = client.create_producer("A")
producer.close()
client.close()does seem to silence the loggers better than using the standard library logging. I'll leave the issue open if possible though, because the inconsistency between the behaviour is something which should probably be looked into at some point. |
Did you mean the last two lines? The logs above are printed by the default logger because they are logged during the exit process of the Python interpreter. At this moment, the Python logger cannot be used. So we fall back to the default logger from the C extension instead. See pulsar-client-python/src/config.cc Lines 57 to 59 in 4b61547 Reference: https://docs.python.org/3/c-api/init.html#c.Py_IsInitialized |
|
Hmm, that does make it tricky. Effectively my concern here is just that we have our logs from all services routed into a common place, and they all undergo a json transformation. So I'm trying to ensure everything which is collected looks correct. When using python logging, this is easy, because I can attach a json handler or intercept third party library logging, just not when its a direct stdout call. I noticed in another thread that the C++ client now has an ability to silence all logging, maybe exposing that to the python client would be the easiest way to ensure that developers can have a clean stdout under their control? If not, perhaps just documenting the difference between passing in a python logger and a C++ logger has might alleviate any confusion for other developers trying to integrate the pulsar client |
|
Here is a workaround to trigger the cleanup (destruction of the objects from the C++ library) explicitly. import logging
import pulsar
pulsar_logger = logging.getLogger("pulsarDummyLogger")
pulsar_logger.setLevel(logging.INFO)
logging.basicConfig(level=logging.INFO, format="{message}", style="{")
logging.info("This should be logged")
pulsar_logger.info("This isn't logged (correctly)")
client = pulsar.Client("pulsar://localhost:6650", logger=pulsar_logger)
producer = client.create_producer("A")
producer.close()
client.close()
del producer
del clientThen |
Fixes apache#194 ### Motivation When the Python logger is configured as the logger, during the exit process of a Python script, the default logger will be used for logging instead because the Python logger is unavailable after the interpreter is finalized. It might lead to some unexpected outputs to standard output.
Fixes #194 ### Motivation When the Python logger is configured as the logger, during the exit process of a Python script, the default logger will be used for logging instead because the Python logger is unavailable after the interpreter is finalized. It might lead to some unexpected outputs to standard output.
There is an option to pass in a Logger when creating the pulsar client to log to. It seems this does not work when creating resources.
Reproduction:
First, run a pulsar standalone instance:
And then run this code:
It will write this to your terminal:
Potentially related to apache/pulsar#7713
I note that, if you set the log level to INFO for the pulsar logger, you do get more output:
But I would expect that the logger is still used when creating resources like producers, readers, and consumers.
Expected behaviour
We can control the formatting and level of the logs produced by the pulsar client. Hopefully this is an easy fix, and the custom logger in the parent client can simply be passed on, if set, to the producer/consumer/readers when they are created
The text was updated successfully, but these errors were encountered: