Permalink
Browse files

Use correct parameter to is_allowed. Thanks to Thomas Bikeev.

  • Loading branch information...
Andrew Wilkinson
Andrew Wilkinson committed Nov 25, 2011
1 parent 15ae16b commit e74c04d8006340a2e4d2d8398c083c52c523785c
Showing with 1 addition and 1 deletion.
  1. +1 −1 crawler/models.py
View
@@ -34,7 +34,7 @@ def update(self):
parse = urlparse(self.url)
robotstxt = RobotsTxt.get_by_domain(parse.scheme, parse.netloc)
- if not robotstxt.is_allowed(parse.netloc):
+ if not robotstxt.is_allowed(self.url):
return False
while cache.get(parse.netloc) is not None:

0 comments on commit e74c04d

Please sign in to comment.