Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Voyager crawler #35

Closed
jpavello opened this issue Jan 17, 2017 · 1 comment
Closed

Voyager crawler #35

jpavello opened this issue Jan 17, 2017 · 1 comment

Comments

@jpavello
Copy link

I am using Voight-Kampf gem to identify wether a visit is a crawler and I recently noticed several non-bot requests detected as robots. They all match with the voyager crawler, corresponding to kosmix.com (a website that doesn't seem to be online anynore) but they do because the user agent includes the string "VOYAGER2 DG310" which happens to be a smartphone by Doogee.

I think this value should probably either be removed (as the site seems to be offline) or be changed to "voyager ", adding a blank space at the end of the string, but I'm not sure which option is the best solution and thats why I open a issue instead a PR.
@monperrus
Copy link
Owner

I think this value should probably either be removed

Sounds a good idea, thanks.

@neurolabs neurolabs mentioned this issue Apr 23, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@monperrus @jpavello