[FIX] web: detect googlebot to avoid redirects

[chienandalu]

Those redirects can kill how the site is fetched. If a website has its default language different to english and english is available as a secondary one, the crawler bot will be always redirected to the english version. For example: mysite.info/en_US.

The effect of that is that crawlers like the Googlebot-image aren't able to validate the site icon, as they don't use any site subfolder.

This is the relevant part where the redirection should be avoided

odoo/addons/http_routing/models/ir_http.py

Lines 452 to 455 in cce0b49

	# See /3, missing lang in url but user-agent is a bot
	elif not url_lang_str and request.env['ir.http'].is_a_bot():
	_logger.debug("%r (lang: %r) missing lang in url but user-agent is a bot, continue", path, request_url_code)
	request.lang = default_lang

cc @Tecnativa TT48630

Set to draft for the moment to test on customer

---

I confirm I have signed the CLA and read the PR guidelines at www.odoo.com/submit-pr

[robodoo]

[Julien00859]

Hello @chienandalu thank you for your work.

Set to draft for the moment to test on customer

Have you had the opportunity to test this PR with your customer?

[chienandalu]

It avoids the redirection indeed which is needed (tested with user-agent switcher), which is a good signal. Although we're still to receive confirmation of the correct results of the crawling

[Julien00859]

Hello @chienandalu, I'm willing to merge this PR once you confirmed me it was working as expected

Although we're still to receive confirmation of the correct results of the crawling

[chienandalu]

Looks like after a while it's finally working

[Julien00859]

Actually @chienandalu looking again at this change. Should "googlebot" be already considered a bot because "bot" in "googlebot"?

[chienandalu]

Oh, you're right... I didn't see that it's check agains every string not against the whole list... closing this then. sorry

[Julien00859]

No worry ;)