Add a full list of validated bots to skip the fake browser tests.

This should make it easier to maintain in the long run, and whitelists all
of the yandex bots too.

rules/flameeyes_60_fake_browsers.conf

@@ -16,7 +16,7 @@
 # agents because they will try to pass for browsers even when they are
 # not. We already validate these bots to be coming from a FcRDNS so we
 # don't need to apply more validation.
-SecRule REQUEST_HEADERS:User-Agent "@pm applebot bingbot bingpreview googlebot" \
+SecRule REQUEST_HEADERS:User-Agent "@pmfromfile flameeyes_validated_robots.data" \
 	"id:436998,t:lowercase,phase:2,skipAfter:FLAMEEYES_END_FAKE_BROWSERS_HEADERS,nolog"
 
 # A number of spambots tries to mimic known good User-Agent values,

rules/flameeyes_validated_robots.data

@@ -0,0 +1,39 @@
+adsbot-google
+applebot
+applebot
+archive.org_bot
+baiduspider
+bingbot
+bingpreview
+bloglines/
+blogscope
+friendfeedbot
+googlebot
+hailoobot
+linkedinbot
+mediapartners-google
+mojeekbot
+netvibes
+newsgator
+technoratibot
+yadirectfetcher
+yandexaccessibilitybot
+yandexblogs
+yandexbot
+yandexcalendar
+yandexdirectdyn
+yandexfavicons
+yandeximageresizer
+yandeximages
+yandexmedia
+yandexmetrika
+yandexmobilebot
+yandexnews
+yandexpagechecker
+yandexscreenshotbot
+yandexsitelinks
+yandexvertis
+yandexvideo
+yandexvideoparser
+yandexwebmaster
+yeti/