Adding fix for Chrome Lighthouse not being detected as a bot

[ankurp]

Since rails uses this gem for allow_browser It shows a 406 Browser not supported error when request is made by PageSpeed Insight. https://pagespeed.web.dev/

Stackoverflow discussing the issue where the user agent contains Chrome-Lighthouse starting March of 2024 https://stackoverflow.com/questions/16403295/what-is-the-name-of-the-google-pagespeed-user-agent

[ankurp]

@gshutler just bumping again to see if I can get a review and have this be merged as it is a dependency in rails framework and causing issue with Lighthouse by default.

[ankurp]

@gshutler Wanted to see if you have time to get this merged so can update the gem in Rails where it is showing a 406 page because lighthouse is using a specific user agent with older browser.

[gshutler]

Is this actually the case?

https://github.com/GoogleChrome/lighthouse/blob/main/core/config/constants.js#L42-L43

It appears to not be in their user agent for over a year.

[gshutler]

Realised I could run a test myself. Came through as:

Mozilla/5.0 (Linux; Android 7.0; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4590.2 Mobile Safari/537.36 Chrome-Lighthouse
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4590.2 Safari/537.36 Chrome-Lighthouse

[gshutler]

https://github.com/gshutler/useragent/releases/tag/v0.16.11 contains this change.

[tagliala]

Hello,

I ran into this but I think there was a change in Lighthouse and I cannot see the user agent "Chrome-Lighthouse" string anymore. In fact, this has been removed in GoogleChrome/lighthouse#14384

I'm now getting

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36

Ref: GoogleChrome/lighthouse#14917 (comment)