Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding some bing bot patterns #1386

Merged
merged 1 commit into from Apr 21, 2017
Merged

Conversation

jaydiablo
Copy link
Contributor

This fixes #1380.

I added the agents for that issue that weren’t already tested for, and
also dug through our logs a little bit for other Bing agents that
weren’t detecting properly (I didn’t find too much, but did find a
WebHistory bot, so added that as well).

I didn’t add device detection to these bing agents that have devices
(other than for iOS devices). I didn’t think it was particularly
useful for crawlers, as I’m pretty sure they’re not trying to emulate
the device they claim they are. I did add some rendering engines to
some of these bing bots though.

This fixes browscap#1380.

I added the agents for that issue that weren’t already tested for, and
also dug through our logs a little bit for other Bing agents that
weren’t detecting properly (I didn’t find too much, but did find a
WebHistory bot, so added that as well).

I didn’t add device detection to these bing agents that have devices
(other than for iOS devices).  I didn’t think it was particularly
useful for crawlers, as I’m pretty sure they’re not trying to emulate
the device they claim they are. I did add some rendering engines to
some of these bing bots though.
"platforms": [
"iOS_A_3_0", "iOS_A_3_1", "iOS_A_3_2", "iOS_A_4_0", "iOS_A_4_1", "iOS_A_4_2", "iOS_A_4_3",
Copy link
Contributor Author

@jaydiablo jaydiablo Apr 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot to mention in the description that I removed all of these iOS versions because there's no evidence that these bots use these versions, so they just add unnecessary pattern bloat, IMO.

I looked through our logs and other parser test suites and didn't find any other cases of iOS other than 6.0 and 7.0 being used by the bing bots.

The iOS_A pattern should pick up any new versions that Microsoft decides to use (they just won't have the right iOS version). Browscap's tests don't hit any of these versions either (obviously :) ).

@codecov
Copy link

codecov bot commented Apr 21, 2017

Codecov Report

Merging #1386 into master will increase coverage by 0.06%.
The diff coverage is 37.37%.

Impacted file tree graph

@@             Coverage Diff             @@
##             master   #1386      +/-   ##
===========================================
+ Coverage     24.53%   24.6%   +0.06%     
  Complexity      804     804              
===========================================
  Files          1425    1426       +1     
  Lines         65026   64991      -35     
  Branches      21880   21824      -56     
===========================================
+ Hits          15954   15988      +34     
+ Misses        49072   49003      -69
Impacted Files Coverage Δ Complexity Δ
...agents/crawlers/microsoft/bingpreview-generic.json 0% <0%> (ø) 0 <0> (ø) ⬇️
...es/user-agents/crawlers/microsoft/bingpreview.json 0% <0%> (ø) 0 <0> (ø) ⬇️
...ts/crawlers/general-crawlers/general-crawlers.json 42.96% <100%> (+0.13%) 0 <0> (ø) ⬇️
...r-agents/crawlers/microsoft/bing-local-search.json 100% <100%> (ø) 0 <0> (?)
.../user-agents/crawlers/microsoft/msn-media-ios.json 42.85% <44.44%> (+32.85%) 0 <0> (ø) ⬇️
...ources/user-agents/crawlers/microsoft/bingbot.json 38.7% <46.66%> (+8.7%) 0 <0> (ø) ⬇️
...er-agents/crawlers/microsoft/bingpreview-beta.json 52% <50%> (+26.19%) 0 <0> (ø) ⬇️
...urces/user-agents/crawlers/microsoft/adidxbot.json 58.33% <71.42%> (+18.33%) 0 <0> (ø) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a39a997...2fc2f85. Read the comment docs.

@mimmi20 mimmi20 self-assigned this Apr 21, 2017
@mimmi20 mimmi20 added this to the 6023 milestone Apr 21, 2017
@mimmi20 mimmi20 merged commit e2c5733 into browscap:master Apr 21, 2017
@jaydiablo jaydiablo deleted the issue-1380 branch April 22, 2017 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing bing bots
2 participants