Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves version detection for Opera and Opera Mobile and improves browser version detection for client hints #7065

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
0651eb1
Adds detection for Wave Browser and improves version detection for cl…
liviuconcioiu Mar 25, 2022
d2a8f0d
Improves version detection for Opera Mobile
liviuconcioiu Mar 25, 2022
1c8953d
Improves version detection for Opera
liviuconcioiu Mar 25, 2022
515dd50
Fix test
liviuconcioiu Mar 25, 2022
43e1794
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Mar 25, 2022
f3a9655
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Mar 29, 2022
c7c1f06
Fix
liviuconcioiu Mar 29, 2022
8ccd5c3
Add client hints fixture for Coc Coc
liviuconcioiu Mar 31, 2022
36d48a2
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Mar 31, 2022
1674c2c
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Mar 31, 2022
f80ec90
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Apr 4, 2022
8fb2ccb
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Apr 7, 2022
c13d97b
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Apr 12, 2022
41a08c6
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Apr 25, 2022
e355c72
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Apr 25, 2022
c1859af
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu May 7, 2022
616b4b3
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu May 16, 2022
93bd022
Remove Wave Browser
liviuconcioiu May 19, 2022
182f63a
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu May 19, 2022
8aedfa6
Fix version
liviuconcioiu May 19, 2022
af60cdf
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Aug 16, 2022
3f61274
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Oct 18, 2022
327991d
Update version for Brave browser
liviuconcioiu Oct 18, 2022
d6e4b19
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Nov 16, 2022
945cc04
Update version for Brave browser
liviuconcioiu Nov 16, 2022
36c6d4d
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Dec 2, 2022
05a41d4
Merge remote-tracking branch 'upstream/master' into browsers
liviuconcioiu Dec 19, 2022
76cff82
Merge branch 'master' into browsers
liviuconcioiu Apr 10, 2023
737d26d
Update version for Wavebox
liviuconcioiu Apr 10, 2023
89b1dec
Merge branch 'master' into browsers
liviuconcioiu Jan 23, 2024
f6e0f04
Fix version for Crow Browser
liviuconcioiu Jan 23, 2024
118eed0
Merge branch 'master' into browsers
liviuconcioiu Feb 24, 2024
deb42fa
Fix version for Vewd Browser
liviuconcioiu Feb 24, 2024
4f36512
Merge branch 'master' into browsers
liviuconcioiu Mar 1, 2024
49d3069
Merge branch 'master' into browsers
liviuconcioiu Mar 6, 2024
ce6b520
Merge branch 'master' into browsers
liviuconcioiu Mar 6, 2024
f7ee8d9
Merge branch 'master' into browsers
liviuconcioiu Mar 7, 2024
24cde30
Merge branch 'master' into browsers
liviuconcioiu Mar 7, 2024
c662c2f
Merge branch 'master' into browsers
liviuconcioiu Mar 11, 2024
ecfe03b
Fix Catsxp version
liviuconcioiu Mar 11, 2024
85c63fb
Fix version detection for DuckDuckGo Privacy Browser
liviuconcioiu Mar 11, 2024
29c22c2
Add double quotes
liviuconcioiu Mar 24, 2024
ee2ea81
Merge branch 'master' into browsers
liviuconcioiu Mar 24, 2024
6e76c22
Improves version detection for MIUI Browser
liviuconcioiu Jul 14, 2024
45f23dc
Merge branch 'master' into browsers
liviuconcioiu Jul 14, 2024
695564d
Remove unneeded headers
liviuconcioiu Jul 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 17 additions & 9 deletions Parser/Client/Browser.php
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,7 @@ class Browser extends AbstractClientParser
'VV' => 'vivo Browser',
'VB' => 'Vision Mobile Browser',
'VM' => 'VMware AirWatch',
'WB' => 'Wave Browser',
'WI' => 'Wear Internet Browser',
'WP' => 'Web Explorer',
'WE' => 'WebPositive',
Expand Down Expand Up @@ -493,7 +494,7 @@ class Browser extends AbstractClientParser
'HP', 'IO', 'TP', 'CJ', 'HQ', 'HI', 'NA', 'BW', 'YO',
'DC', 'G8', 'DT', 'AP', 'AK', 'UI', 'SD', 'VN', '4S',
'2S', 'RF', 'LR', 'SQ', 'BV', 'L1', 'F0', 'KS', 'V0',
'C8', 'AZ', 'MM', 'BT', 'N0', 'P0', 'F3', 'VS',
'C8', 'AZ', 'MM', 'BT', 'N0', 'P0', 'F3', 'VS', 'WB',
],
'Firefox' => [
'AX', 'BI', 'BF', 'BH', 'BN', 'C0', 'CU', 'EI', 'F1',
Expand Down Expand Up @@ -656,7 +657,7 @@ public function parse(): ?array
$engine = '';
$engineVersion = '';

// If client hints report Chromium, but user agent detects a chromium based browser, we favor this instead
// If client hints report Chromium, but user agent detects a Chromium based browser, we favor this instead
if ('Chromium' === $name
&& !empty($browserFromUserAgent['name'])
&& 'Chromium' !== $browserFromUserAgent['name']
Expand Down Expand Up @@ -684,14 +685,21 @@ public function parse(): ?array
if ($name === $browserFromUserAgent['name']) {
$engine = $browserFromUserAgent['engine'] ?? '';
$engineVersion = $browserFromUserAgent['engine_version'] ?? '';
}

// In case the user agent reports a more detailed version, we try to use this instead
if (!empty($browserFromUserAgent['version'])
&& 0 === \strpos($browserFromUserAgent['version'], $version)
&& \version_compare($version, $browserFromUserAgent['version'], '<')
) {
$version = $browserFromUserAgent['version'];
}
// In case the user agent reports a more detailed version, we try to use this instead
if (!empty($browserFromUserAgent['version'])
&& 0 === \strpos($browserFromUserAgent['version'], $version)
&& \version_compare($version, $browserFromUserAgent['version'], '<')
) {
$version = $browserFromUserAgent['version'];
}
Comment on lines +989 to +994
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be moved outside of the if condition. Let's assume the client hints report a Firefox 16, but the useragent is spoofed by some tool and contains Safari 20.18. We would then end up using Firefox 20.18, which is wrong.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And, how can we detect the full version then? Some times, client hints don't provide Sec-CH-UA-Full-Version

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In most cases the browser reported in client hints and in user agent should match. In that case the version from user agent will be used. If not I'm not sure if we should simply use the version from useragent.

Copy link
Collaborator

@sanchezzzhak sanchezzzhak Apr 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it turns out to be interesting here, opera prints all versions in clienthints

04/04/2022
last version Opera Desktop Linux 85
last version Opera Mobile 68.2

Opera on Windows Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36 OPR/85.0.4341.18Mozilla/5.0 (Windows NT 10.0; WOW64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36 OPR/85.0.4341.18
Opera on Macos Mozilla/5.0 (Macintosh; Intel Mac OS X 12_3_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36 OPR/85.0.4341.18
Opera on Linux Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.60 Safari/537.36 OPR/85.0.4341.18
Opera on Android Mozilla/5.0 (Linux; Android 10; VOG-L29) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.58 Mobile Safari/537.36 OPR/63.3.3216.58675
-- --
Opera on Android Mozilla/5.0 (Linux; Android 10; SM-G970F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.58 Mobile Safari/537.36 OPR/63.3.3216.58675Mozilla/5.0 (Linux; Android 10; SM-N975F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.58 Mobile Safari/537.36 OPR/63.3.3216.58675

if you look at this list, they just printed versions for all OS

Sec-CH-UA: '"Chromium";v="98", "Opera";v="82", " Not A;Brand";v="99", "OperaMobile";v="68"'
Sec-CH-UA-Platform: 'Android'

the current tests I am satisfied with the result.

But in the future it will be necessary to come up with something else when useragent is gone.
as an option:
if Mobile OS + opera family get clienthint OperaMobile=68
if Desktop OS + opera family get Opera=82


// If client hints report Opera or Opera Mobile, we use the version from useragent
if (!empty($browserFromUserAgent['version'])
&& ('OP' === $short || 'OM' === $short)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it known that Opera always reports an incorrect version in the client hints? 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the data I have, everything is incorrect, just for those two browser.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I'm not sure how we should handle that. If the browsers might freeze the useragent at some point I'm not sure if the version in useragent will be even updated with new releases or if the number can only be fetched through client hints 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I close this PR then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sanchezzzhak what do you think about that?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current changes give an accurate result.
but if there is no user agent in the future, this condition will need to be rewritten.

) {
$version = $browserFromUserAgent['version'];
}
} else {
$name = $browserFromUserAgent['name'];
Expand Down
52 changes: 52 additions & 0 deletions Tests/Parser/Client/fixtures/browser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5207,6 +5207,45 @@
engine: Blink
engine_version: ""
family:
-
user_agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36
client:
type: browser
name: CCleaner
version: 99.0.4844.51
engine: Blink
engine_version: 99.0.4844.51
family: Chrome
headers:
Sec-CH-UA: '" Not A;Brand";v="99", "Chromium";v="99", "CCleaner Browser";v="99"'
Sec-CH-UA-Platform: 'Windows'
Sec-CH-UA-Mobile: '?0'
-
user_agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36
client:
type: browser
name: Wave Browser
version: 96.0.4664.45
engine: Blink
engine_version: 96.0.4664.45
family: Chrome
headers:
Sec-CH-UA: '" Not A;Brand";v="99", "WaveBrowser";v="96", "WaveBrowser";v="96"'
Sec-CH-UA-Platform: 'Windows'
Sec-CH-UA-Mobile: '?0'
-
user_agent: Mozilla/5.0 (Linux; Android 11; SM-M317F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.101 Mobile Safari/537.36 OPR/68.0.3540.63570
client:
type: browser
name: Opera Mobile
version: 68.0.3540.63570
engine: Blink
engine_version: 98.0.4758.101
family: Opera
headers:
Sec-CH-UA: '"Chromium";v="98", "Opera";v="82", " Not A;Brand";v="99", "OperaMobile";v="68"'
Sec-CH-UA-Platform: 'Android'
Sec-CH-UA-Mobile: '?1'
-
user_agent: Mozilla/5.0 (Linux; Android 11; SM-A022G Build/RP1A.200720.012; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/97.0.4692.98 Mobile Safari/537.36
client:
Expand Down Expand Up @@ -5333,6 +5372,19 @@
family: Chrome
headers:
x-requested-with: com.anc.web.browser
-
user_agent: Mozilla/5.0 (Linux; Android 10; PACT00) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.120 Mobile Safari/537.36
client:
type: browser
name: Coc Coc
version: 97.0.4692.120
engine: Blink
engine_version: 97.0.4692.120
family: Chrome
headers:
Sec-CH-UA: '" Not;A Brand";v="99", "CocCoc";v="97", "Chromium";v="97"'
Sec-CH-UA-Platform: 'Android'
Sec-CH-UA-Mobile: '?1'
-
user_agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0 Floorp/8.3.6
client:
Expand Down
6 changes: 3 additions & 3 deletions Tests/fixtures/clienthints.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
client:
type: browser
name: Opera Mobile
version: "96.0.4664.104"
version: "67.1.3508.63168"
engine: Blink
engine_version: "96.0.4664.104"
device:
Expand Down Expand Up @@ -160,7 +160,7 @@
client:
type: browser
name: Opera
version: "98.0.4758.82"
version: "84.0.4316.19"
engine: Blink
engine_version: "98.0.4758.82"
device:
Expand Down Expand Up @@ -422,7 +422,7 @@
client:
type: browser
name: Opera
version: "98.0.4758.102"
version: "84.0.4316.21"
engine: Blink
engine_version: "98.0.4758.102"
device:
Expand Down