Skip to content

/ scrapy

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

utils.is_gzipped returns false on application/x-gzip;charset=utf-8 #2049

Closed
Tethik opened this issue Jun 12, 2016 · 1 comment
Closed

utils.is_gzipped returns false on application/x-gzip;charset=utf-8 #2049

Tethik opened this issue Jun 12, 2016 · 1 comment
Labels
bug
Milestone
v1.1.1

Comments

@Tethik
Copy link
Contributor

@Tethik Tethik commented Jun 12, 2016 •
edited

I had a site return the following header, which caused SitemapSpider to not parse a sitemap xml that was gzipped.

Content-Type: application/x-gzip;charset=utf-8

Looking into the code it seems that the function utils.is_gzipped does not take into account cases where the Content-Type header would include charset.
@Tethik Tethik mentioned this issue Jun 12, 2016
Merged
@redapple redapple added the bug label Jun 14, 2016
@eliasdorneles
Copy link
Member

@eliasdorneles eliasdorneles commented Jul 8, 2016

Fixed in #2050!
@Tethik @robsonpeixoto @redapple thanks fellows!
@redapple redapple added this to the v1.1.1 milestone Jul 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
v1.1.1
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@eliasdorneles @Tethik @redapple
You can’t perform that action at this time.