Scrapy provides a logging facility which can be used through the scrapy.log
module. The current underlying implementation uses Twisted logging but this may change in the future.
The logging service must be explicitly started through the scrapy.log.start
function.
Scrapy provides 5 logging levels:
~scrapy.log.CRITICAL
- for critical errors~scrapy.log.ERROR
- for regular errors~scrapy.log.WARNING
- for warning messages~scrapy.log.INFO
- for informational messages~scrapy.log.DEBUG
- for debugging messages
You can set the log level using the --loglevel/-L command line option, or using the LOG_LEVEL
setting.
Here's a quick example of how to log a message using the WARNING
level:
from scrapy import log
log.msg("This is a warning", level=log.WARNING)
The recommended way to log from spiders is by using the Spider ~scrapy.spider.BaseSpider.log
method, which already populates the spider
argument of the scrapy.log.msg
function. The other arguments are passed directly to the ~scrapy.log.msg
function.
scrapy.log
started
A boolean which is True
if logging has been started or False
otherwise.
start(logfile=None, loglevel=None, logstdout=None)
Start the logging facility. This must be called before actually logging any messages. Otherwise, messages logged before this call will get lost.
- param logfile
the file path to use for logging output. If omitted, the
LOG_FILE
setting will be used. If both areNone
, the log will be sent to standard error.- type logfile
str
- param loglevel
the minimum logging level to log. Availables values are:
CRITICAL
,ERROR
,WARNING
,INFO
andDEBUG
.- param logstdout
if
True
, all standard output (and error) of your application will be logged instead. For example if you "print 'hello'" it will appear in the Scrapy log. If ommited, theLOG_STDOUT
setting will be used.- type logstdout
boolean
msg(message, level=INFO, spider=None)
Log a message
- param message
the message to log
- type message
str
- param level
the log level for this message. See
topics-logging-levels
.- param spider
the spider to use for logging this message. This parameter should always be used when logging things related to a particular spider.
- type spider
~scrapy.spider.BaseSpider
object
CRITICAL
Log level for critical errors
ERROR
Log level for errors
WARNING
Log level for warnings
INFO
Log level for informational messages (recommended level for production deployments)
DEBUG
Log level for debugging messages (recommended level for development)
These settings can be used to configure the logging:
LOG_ENABLED
LOG_ENCODING
LOG_FILE
LOG_LEVEL
LOG_STDOUT