Skip to content

Releases: scrapy/protego

0.3.1

05 Apr 10:11
Compare
Choose a tag to compare
  • Added official support for Python 3.12.

  • = is no longer percent-encoded in patterns, fixing many scenarios where
    patterns included query strings.

0.3.0

09 Aug 17:42
Compare
Choose a tag to compare
  • Changed requirements:

    • Dropped support for Python 2.7, 3.5, 3.6, and 3.7, and added support
      for 3.11 and for the upcoming 3.12.

    • six is no longer a dependency.

  • Added support for the Visit-Time directive.

  • Fixed leading asterisks in allow and disallow values not being properly
    interpreted.

  • Protego.parse() now raises value error when content is not a string.

Protego 0.2.1

19 Feb 14:19
aed7040
Compare
Choose a tag to compare
  1. Fixes incorrect readme content-type specified in setup.py (#21)

Protego 0.2.0

12 Feb 09:42
Compare
Choose a tag to compare
  1. Fixes disallow not working when no path is provided (#19)
  2. Adds support to handle sitemaps that come before any User-Agent directives (#15)
  3. Makes smaller wheel distributions (#10)
  4. Adds code coverage support (#7)
  5. Adds test coverage for Python versions 3.8, 3.9 and 3.10 (#20).
  6. Code cleanups (#9, #11)

Protego 0.1.16

09 Dec 12:26
Compare
Choose a tag to compare

Protego 0.1.16 fixes (#8) an issue with robots.txt files containing absolute URLs as values for allow and disallow directives, where their path would be interpreted ignoring their protocol and netloc, leading to unexpected results (#4). Now, values of allow and disallow directives are always interpreted as if they started with a /, as Google’s specification dictates that they must always start with /.

We’ve also improved (#3) our test coverage for malformed disallow directives.

Initial release of Protego

28 Aug 15:37
Compare
Choose a tag to compare
0.1.7

changes travis config

Initial release of Protego

20 Aug 14:36
452364a
Compare
Choose a tag to compare
0.1.3

Changes Travis config

Initial release of Protego

20 Aug 14:31
452364a
Compare
Choose a tag to compare
v0.1.2

Changes Travis config

Initial release of Protego

20 Aug 14:21
363e930
Compare
Choose a tag to compare
v0.1.1

Changes travis config.

Initial release of Protego

20 Aug 12:44
Compare
Choose a tag to compare
v0.1

remove pytest.ini from MANIFEST.in