Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filtering examples in Scrappy logging documentation #4257

Closed
wants to merge 1 commit into from

Conversation

akamanzi
Copy link

@akamanzi akamanzi commented Dec 27, 2019

Added tutorial example to filter substring in log message using logging filters.
Example given doesn't cover the use of regular expressions, there is an external link to redirect users to Python's standard library re module.

Fixes #4216

@codecov
Copy link

codecov bot commented Dec 27, 2019

Codecov Report

Merging #4257 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4257      +/-   ##
==========================================
+ Coverage      84%   84.01%   +0.01%     
==========================================
  Files         165      165              
  Lines        9676     9676              
  Branches     1447     1447              
==========================================
+ Hits         8128     8129       +1     
  Misses       1294     1294              
+ Partials      254      253       -1
Impacted Files Coverage Δ
scrapy/utils/trackref.py 85.71% <0%> (+2.85%) ⬆️

@elacuesta elacuesta changed the title Filtering examples in Scrappy logging documentation (#4216) Filtering examples in Scrappy logging documentation Dec 27, 2019
def filter(self, record):
return 'filter' in record.msg

logger = logging.getLogger('mycustomlogger')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make it clear how to grab the root logger that Scrapy installs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gallaecio, thank you for the feedback, I will look into this next week!

Thank you

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gallaecio, sorry for this late reply, i had exams at the moment. so by root logger do you mean calling logging.getLogger() instead of logging.getLogger('mycustomlogger') as in the above example? could you please elaborate on this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not sure if logging.getLogger() would work, or if you would need to access something else that is not a logger. The point is that we should document an approach that can be used for:

  • The whole project. If a matching message is logger by a middleware, a spider, or something else, it should be filtered out. I’m not sure if the same steps using getLogger() would work, or something else is needed. Try.
  • A specific logger (i.e. from a specific middleware or from a specific spider). If this is as simple as using getLogger with the right parameter, I think the documentation just needs to show an example of logging and indicate what is the logger name there (for Scrapy users not familiar yet with the basics of Python logging).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gallaecio , i now get your point and it is valid. am not sure how either to get the root logger for the whole project. trying to look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cover arbitrary filtering in the Scrapy logging documentation
2 participants