Releases: scrapy/protego
0.3.1
0.3.0
-
Changed requirements:
-
Dropped support for Python 2.7, 3.5, 3.6, and 3.7, and added support
for 3.11 and for the upcoming 3.12. -
six
is no longer a dependency.
-
-
Added support for the
Visit-Time
directive. -
Fixed leading asterisks in allow and disallow values not being properly
interpreted. -
Protego.parse()
now raises value error when content is not a string.
Protego 0.2.1
- Fixes incorrect readme
content-type
specified insetup.py
(#21)
Protego 0.2.0
Protego 0.1.16
Protego 0.1.16 fixes (#8) an issue with robots.txt
files containing absolute URLs as values for allow
and disallow
directives, where their path would be interpreted ignoring their protocol and netloc, leading to unexpected results (#4). Now, values of allow
and disallow
directives are always interpreted as if they started with a /
, as Google’s specification dictates that they must always start with /
.
We’ve also improved (#3) our test coverage for malformed disallow
directives.
Initial release of Protego
0.1.7 changes travis config
Initial release of Protego
0.1.3 Changes Travis config
Initial release of Protego
v0.1.2 Changes Travis config
Initial release of Protego
v0.1.1 Changes travis config.
Initial release of Protego
v0.1 remove pytest.ini from MANIFEST.in