Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding new Google+ scraper bot #1422

Merged
merged 2 commits into from
May 15, 2017
Merged

Conversation

jaydiablo
Copy link
Contributor

Fixes #1415.

I looked back through our logs from 2013 - today looking for various versions of this google bot, and only found 4 different useragents:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/)
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/)
Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)
Mozilla/5.0 (Windows NT 6.1 rv:6.0) Gecko/20110814 Firefox/6.0 Google (+https://developers.google.com/+/web/snippet/)

The two Gecko ones are already covered by tests, and the first two Webkit ones are covered by the new test I added.

The other platforms of this bot could probably be removed, but I left them there just in case.

@jaydiablo
Copy link
Contributor Author

Oops, forgot to test Lite/Standard, will fix.

@codecov
Copy link

codecov bot commented May 15, 2017

Codecov Report

Merging #1422 into master will increase coverage by <.01%.
The diff coverage is 75%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1422      +/-   ##
============================================
+ Coverage     33.48%   33.49%   +<.01%     
  Complexity      816      816              
============================================
  Files          1433     1433              
  Lines         53615    53619       +4     
  Branches      21035    21036       +1     
============================================
+ Hits          17955    17959       +4     
  Misses        35660    35660
Impacted Files Coverage Δ Complexity Δ
...ser-agents/crawlers/google/google-web-snippet.json 34.78% <75%> (+13.72%) 0 <0> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7ef509...abba1f8. Read the comment docs.

@mimmi20 mimmi20 self-assigned this May 15, 2017
@mimmi20 mimmi20 merged commit 798fc60 into browscap:master May 15, 2017
@jaydiablo jaydiablo deleted the issue-1415 branch May 16, 2017 22:50
@mimmi20 mimmi20 added this to the 6024 milestone May 19, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants