Does AppleBot still masquerate as GoogleBot?

[dlackty]

Hello, Alexander
Hope you're doing well!

While reading through the source code, I found there's a logic to detect AppleBot masquerateing as GoogleBot (Introduced here: 2d03063)

However, I cound't find any information regarding this behavior, even looked back to the doc in 2018, when the code was added.

So I'm wondering if this behavior and detection could be removed.

[alaz]

Hello Richard,

I am good, thank you! Hope you are doing well too!

I remember that I saw such behaviour in server logs: the requests were coming from Apple's IP range, but manifested themselves as Googlebot.

I searched some historical logs for ^17\..*Googlebot, but found zero results. Either such requests are rare or Applebot does not masquerade as Googlebot anymore. Do you have any log data to search for this case?

[dlackty]

I searched through the logs from 2020/1/1 ~ now, there's two different Applebot user-agent found on our site, based on ~5M requests:

• Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)
• Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)

I also did a searched for logs in 2018 and showed no Googlebot was found as well.

As we're not sure the reason why you saw that behavior, I guessed maybe Apple engineers were developing their bots and poking around your server?

[alaz]

Thank you. I agree that it makes sense to remove the workaround. I shall prepare a PR.