Skip to content
Branch: master
Go to file
Code

README.md

Third Party Web

Check out the shiny new web UI https://www.thirdpartyweb.today/

Data on third party entities and their impact on the web.

This document is a summary of which third party scripts are most responsible for excessive JavaScript execution on the web today.

Table of Contents

  1. Goals
  2. Methodology
  3. NPM Module
  4. Updates
  5. Data
    1. Summary
    2. How to Interpret
    3. Third Parties by Category
      1. Advertising
      2. Analytics
      3. Social
      4. Video
      5. Developer Utilities
      6. Hosting Platforms
      7. Marketing
      8. Customer Success
      9. Content & Publishing
      10. CDNs
      11. Tag Management
      12. Mixed / Other
    4. Third Parties by Total Impact
  6. Future Work
  7. FAQs
  8. Contributing

Goals

  1. Quantify the impact of third party scripts on the web.
  2. Identify the third party scripts on the web that have the greatest performance cost.
  3. Give developers the information they need to make informed decisions about which third parties to include on their sites.
  4. Incentivize responsible third party script behavior.
  5. Make this information accessible and useful.

Methodology

HTTP Archive is an inititiave that tracks how the web is built. Every month, ~4 million sites are crawled with Lighthouse on mobile. Lighthouse breaks down the total script execution time of each page and attributes the execution to a URL. Using BigQuery, this project aggregates the script execution to the origin-level and assigns each origin to the responsible entity.

NPM Module

The entity classification data is available as an NPM module.

const {getEntity} = require('third-party-web')
const entity = getEntity('https://d36mpcpuzc4ztk.cloudfront.net/js/visitor.js')
console.log(entity)
//   {
//     "name": "Freshdesk",
//     "homepage": "https://freshdesk.com/",
//     "categories": ["customer-success"],
//     "domains": ["d36mpcpuzc4ztk.cloudfront.net"]
//   }

Updates

2019-02-01 dataset

Huge props to WordAds for reducing their impact from ~2.5s to ~200ms on average! A few entities are showing considerably less data this cycle (Media Math, Crazy Egg, DoubleVerify, Bootstrap CDN). Perhaps they've added new CDNs/hostnames that we haven't identified or the basket of sites in HTTPArchive has shifted away from their usage.

2019-03-01 dataset

Almost 2,000 entities tracked now across ~3,000+ domains! Huge props to @simonhearne for making this massive increase possible. Tag Managers have now been split out into their own category since they represented such a large percentage of the "Mixed / Other" category.

2019-05-06 dataset

Google Ads clarified that www.googletagservices.com serves more ad scripts than generic tag management, and it has been reclassified accordingly. This has dropped the overall Tag Management share considerably back down to its earlier position.

2019-05-13 dataset

A shortcoming of the attribution approach has been fixed. Total usage is now reported based on the number of pages in the dataset that use the third-party, not the number of scripts. Correspondingly, all average impact times are now reported per page rather than per script. Previously, a third party could appear to have a lower impact or be more popular simply by splitting their work across multiple files.

Third-parties that performed most of their work from a single script should see little to no impact from this change, but some entities have seen significant ranking movement. Hosting providers that host entire pages are, understandably, the most affected.

Some notable changes below:

Third-Party Previously (per-script) Now (per-page)
Beeketing 137 ms 465 ms
Sumo 263 ms 798 ms
Tumblr 324 ms 1499 ms
Yandex APIs 393 ms 1231 ms
Google Ads 402 ms 1285 ms
Wix 972 ms 5393 ms

Data

Summary

Across top ~4 million sites, ~2700 origins account for ~57% of all script execution time with the top 50 entities already accounting for ~47%. Third party script execution is the majority chunk of the web today, and it's important to make informed choices.

How to Interpret

Each entity has a number of data points available.

  1. Usage (Total Number of Occurrences) - how many scripts from their origins were included on pages
  2. Total Impact (Total Execution Time) - how many seconds were spent executing their scripts across the web
  3. Average Impact (Average Execution Time) - on average, how many milliseconds were spent executing each script
  4. Category - what type of script is this

Third Parties by Category

This section breaks down third parties by category. The third parties in each category are ranked from first to last based on the average impact of their scripts. Perhaps the most important comparisons lie here. You always need to pick an analytics provider, but at least you can pick the most well-behaved analytics provider.

Overall Breakdown

Unsurprisingly, ads account for the largest identifiable chunk of third party script execution.

breakdown by category

Advertising

These scripts are part of advertising networks, either serving or measuring.

Rank Name Usage Average Impact
1 GumGum 2,887 19 ms
2 ExoClick 1,212 43 ms
3 SublimeSkinz 1,094 58 ms
4 MailMunch 5,095 60 ms
5 Geniee 1,161 68 ms
6 Salesforce.com 1,017 70 ms
7 District M 2,606 72 ms
8 Rakuten Marketing 1,208 72 ms
9 BlueKai 2,302 75 ms
10 PushCrew 2,926 76 ms
11 FreakOut 3,826 76 ms
12 Constant Contact 1,314 80 ms
13 Scorecard Research 6,956 86 ms
14 Gemius 4,535 87 ms
15 Tynt 22,558 87 ms
16 Rubicon Project 39,353 90 ms
17 Popads 2,896 103 ms
18 Nativo 1,091 103 ms
19 Outbrain 8,189 104 ms
20 AppNexus 1,248 105 ms
21 Unbounce 3,591 105 ms
22 Amazon Ads 25,338 107 ms
23 i-mobile 3,227 110 ms
24 LINE Corporation 5,182 113 ms
25 Sharethrough 1,655 114 ms
26 Intent Media 8,019 116 ms
27 Bing Ads 15,786 116 ms
28 Adroll 47,959 117 ms
29 JuicyAds 1,498 125 ms
30 Smart AdServer 5,189 126 ms
31 DTSCOUT 12,679 132 ms
32 Index Exchange 4,946 133 ms
33 Attentive 1,205 146 ms
34 WordAds 18,478 149 ms
35 Skimbit 15,819 167 ms
36 Yahoo! 6,183 172 ms
37 OptiMonk 1,601 173 ms
38 Teads 3,457 177 ms
39 Cxense 4,374 179 ms
40 TrafficStars 2,040 187 ms
41 Adform 6,772 189 ms
42 sovrn 4,673 190 ms
43 Tribal Fusion 1,050 191 ms
44 Unruly Media 1,189 193 ms
45 OpenX 3,015 193 ms
46 Bizible 1,300 194 ms
47 Perfect Market 1,595 219 ms
48 GetResponse 1,073 219 ms
49 Sortable 1,551 220 ms
50 Criteo 62,125 237 ms
51 Adyoulike 1,234 239 ms
52 Permutive 9,556 243 ms
53 Refersion 1,320 249 ms
54 VigLink 7,097 309 ms
55 Privy 13,047 309 ms
56 Yandex Ads 25,906 313 ms
57 Pubmatic 52,613 315 ms
58 Media.net 14,529 341 ms
59 Taboola 24,583 364 ms
60 LongTail Ad Solutions 3,433 365 ms
61 Klaviyo 14,593 392 ms
62 piano 1,183 459 ms
63 DoubleVerify 1,772 548 ms
64 RevJet 1,210 550 ms
65 MGID 8,210 574 ms
66 Infolinks 3,699 579 ms
67 33 Across 1,188 615 ms
68 Vidible 1,146 625 ms
69 Admixer for Publishers 1,043 642 ms
70 MediaVine 6,490 655 ms
71 Bridgewell DSP 14,079 791 ms
72 Moat 6,862 817 ms
73 LoyaltyLion 1,300 830 ms
74 Adloox 5,759 921 ms
75 Integral Ad Science 8,013 1028 ms
76 Sizmek 1,484 1047 ms
77 LKQD 1,335 1449 ms
78 Google/Doubleclick Ads 680,948 1456 ms
79 fam 2,135 1807 ms

Analytics

These scripts measure or track users and their actions. There's a wide range in impact here depending on what's being tracked.

Rank Name Usage Average Impact
1 StatCounter 3,403 59 ms
2 Treasure Data 4,833 62 ms
3 WordPress Site Stats 3,732 64 ms
4 Roxr Software 1,623 66 ms
5 Amplitude Mobile Analytics 2,429 69 ms
6 Heap 2,605 72 ms
7 Mixpanel 7,513 75 ms
8 Google Analytics 1,200,666 77 ms
9 Chartbeat 4,492 79 ms
10 Quantcast 3,787 80 ms
11 Hotjar 177,468 84 ms
12 Parse.ly 3,278 91 ms
13 Searchanise 4,302 91 ms
14 Smart Insight Tracking 1,161 93 ms
15 etracker 1,643 94 ms
16 Snowplow 5,845 95 ms
17 CallRail 7,019 101 ms
18 Nielsen NetRatings SiteCensus 11,805 102 ms
19 Marchex 2,503 104 ms
20 Baidu Analytics 10,739 106 ms
21 Crazy Egg 4,988 110 ms
22 Evidon 1,061 111 ms
23 ContentSquare 1,367 130 ms
24 VWO 4,724 158 ms
25 Trust Pilot 15,202 164 ms
26 Net Reviews 2,537 176 ms
27 PageSense 1,294 180 ms
28 FullStory 7,654 184 ms
29 Segment 9,541 191 ms
30 Kampyle 1,094 206 ms
31 Optimizely 19,583 223 ms
32 Nosto 1,946 225 ms
33 UserReport 1,300 228 ms
34 BounceX 1,386 233 ms
35 mPulse 13,177 246 ms
36 PowerReviews 1,043 261 ms
37 Marketo 1,336 359 ms
38 Inspectlet 5,605 361 ms
39 Histats 13,537 361 ms
40 Bazaarvoice 1,845 397 ms
41 Snapchat 13,344 410 ms
42 Salesforce 10,892 478 ms
43 Lucky Orange 7,529 491 ms
44 Feefo.com 1,686 502 ms
45 Gigya 2,261 579 ms
46 Yandex Metrica 292,542 580 ms
47 Ezoic 1,329 582 ms
48 Revolver Maps 1,144 609 ms
49 AB Tasty 3,010 1594 ms

Social

These scripts enable social features.

Rank Name Usage Average Impact
1 AddToAny 24,490 87 ms
2 Pinterest 17,218 89 ms
3 LinkedIn 14,038 111 ms
4 VK 13,473 121 ms
5 Twitter 213,911 139 ms
6 Kakao 18,109 158 ms
7 Instagram 9,441 184 ms
8 Yandex Share 24,181 184 ms
9 Facebook 1,461,331 229 ms
10 ShareThis 40,133 234 ms
11 SocialShopWave 2,044 302 ms
12 AddThis 118,289 403 ms
13 Disqus 1,252 994 ms
14 LiveJournal 3,680 1327 ms
15 PIXNET 15,434 1508 ms
16 Tumblr 7,972 2048 ms

Video

These scripts enable video player and streaming functionality.

Rank Name Usage Average Impact
1 Vimeo 10,403 355 ms
2 Brightcove 6,615 809 ms
3 YouTube 408,326 849 ms
4 Wistia 13,083 928 ms
5 Twitch 1,068 1896 ms

Developer Utilities

These scripts are developer utilities (API clients, site monitoring, fraud detection, etc).

Rank Name Usage Average Impact
1 Key CDN 1,468 55 ms
2 LightWidget 2,325 66 ms
3 Siteimprove 1,510 76 ms
4 Trusted Shops 5,639 79 ms
5 New Relic 13,062 95 ms
6 Accessibe 3,561 99 ms
7 GetSiteControl 1,812 99 ms
8 Riskified 1,074 106 ms
9 Affirm 2,500 113 ms
10 iubenda 12,781 124 ms
11 Optanon 8,025 128 ms
12 Swiftype 1,544 135 ms
13 Seznam 1,654 135 ms
14 Bold Commerce 13,095 144 ms
15 Cookiebot 20,838 147 ms
16 Sift Science 1,080 148 ms
17 Other Google APIs/SDKs 580,415 168 ms
18 Amazon Pay 5,751 169 ms
19 TrustArc 1,370 180 ms
20 MaxCDN Enterprise 2,394 198 ms
21 GitHub 1,653 232 ms
22 Fraudlogix 2,244 258 ms
23 Fastly 6,694 275 ms
24 PayPal 15,436 342 ms
25 Stripe 23,538 388 ms
26 Cloudflare 53,319 465 ms
27 Google Maps 267,417 520 ms
28 AppDynamics 1,281 527 ms
29 Secomapp 2,150 546 ms
30 Bugsnag 8,686 569 ms
31 Rambler 7,698 691 ms
32 GoDaddy 6,687 700 ms
33 Sentry 9,061 715 ms
34 Signyfyd 1,691 867 ms
35 Mapbox 5,206 877 ms
36 Yandex APIs 27,480 1080 ms
37 POWr 16,407 1364 ms
38 Esri ArcGIS 1,692 4750 ms

Hosting Platforms

These scripts are from web hosting platforms (WordPress, Wix, Squarespace, etc). Note that in this category, this can sometimes be the entirety of script on the page, and so the "impact" rank might be misleading. In the case of WordPress, this just indicates the libraries hosted and served by WordPress not all sites using self-hosted WordPress.

Rank Name Usage Average Impact
1 Blogger 23,674 327 ms
2 Salesforce Commerce Cloud 2,623 342 ms
3 WordPress 92,546 404 ms
4 Dealer 8,637 433 ms
5 Ecwid 1,399 739 ms
6 Tilda 7,842 747 ms
7 Squarespace 39,492 911 ms
8 Adobe Business Catalyst 3,181 971 ms
9 Shopify 107,323 1058 ms
10 Weebly 13,271 1433 ms
11 CDK Dealer Management 3,429 1568 ms
12 Hatena Blog 13,205 1579 ms
13 Wix 54,909 5962 ms

Marketing

These scripts are from marketing tools that add popups/newsletters/etc.

Rank Name Usage Average Impact
1 Beeketing 3,984 74 ms
2 RD Station 6,545 76 ms
3 iZooto 1,370 83 ms
4 Ve 1,848 149 ms
5 Listrak 1,073 154 ms
6 Hubspot 33,125 181 ms
7 Yotpo 13,629 202 ms
8 Mailchimp 23,373 204 ms
9 OptinMonster 7,493 260 ms
10 Bronto Software 1,056 262 ms
11 Pardot 1,435 404 ms
12 Albacross 1,920 490 ms
13 Sumo 18,438 686 ms
14 Bigcommerce 10,096 976 ms
15 Drift 6,565 1279 ms
16 Judge.me 8,307 1375 ms
17 PureCars 2,697 1901 ms
18 Tray Commerce 3,173 2328 ms

Customer Success

These scripts are from customer support/marketing providers that offer chat and contact solutions. These scripts are generally heavier in weight.

Rank Name Usage Average Impact
1 SnapEngage 1,083 59 ms
2 Pure Chat 4,661 70 ms
3 Foursixty 1,411 81 ms
4 LivePerson 4,302 112 ms
5 iPerceptions 1,992 132 ms
6 Comm100 1,321 134 ms
7 Intercom 15,656 245 ms
8 Help Scout 2,183 258 ms
9 Tidio Live Chat 12,655 383 ms
10 Tawk.to 63,460 405 ms
11 LiveChat 19,468 411 ms
12 ContactAtOnce 3,247 491 ms
13 Jivochat 45,110 553 ms
14 Olark 6,903 656 ms
15 Smartsupp 14,862 782 ms
16 ZenDesk 69,488 892 ms

Content & Publishing

These scripts are from content providers or publishing-specific affiliate tracking.

Rank Name Usage Average Impact
1 Research Online 2,222 62 ms
2 Accuweather 1,491 72 ms
3 Booking.com 1,656 157 ms
4 Tencent 2,257 163 ms
5 OpenTable 1,563 165 ms
6 SnapWidget 2,607 174 ms
7 Covert Pics 1,007 184 ms
8 AMP 74,549 308 ms
9 Medium 1,157 474 ms
10 Embedly 5,513 514 ms
11 Spotify 3,225 602 ms
12 issuu 1,934 670 ms
13 SoundCloud 4,464 986 ms
14 Dailymotion 1,838 1244 ms

CDNs

These are a mixture of publicly hosted open source libraries (e.g. jQuery) served over different public CDNs and private CDN usage. This category is unique in that the origin may have no responsibility for the performance of what's being served. Note that rank here does not imply one CDN is better than the other. It simply indicates that the scripts being served from that origin are lighter/heavier than the ones served by another.

Rank Name Usage Average Impact
1 Bootstrap CDN 1,445 55 ms
2 Adobe TypeKit 18,274 94 ms
3 Yandex CDN 1,572 184 ms
4 JSDelivr CDN 97,319 193 ms
5 Microsoft Hosted Libs 4,941 204 ms
6 Monotype 4,073 212 ms
7 Azure Web Services 10,575 225 ms
8 jQuery CDN 167,533 269 ms
9 Unpkg 6,922 274 ms
10 Google CDN 947,402 288 ms
11 Akamai 7,316 336 ms
12 Cloudflare CDN 117,287 339 ms
13 FontAwesome CDN 47,278 413 ms
14 CreateJS CDN 3,516 2323 ms

Tag Management

These scripts tend to load lots of other scripts and initiate many tasks.

Rank Name Usage Average Impact
1 Google Tag Manager 721,295 115 ms
2 TagCommander 1,072 145 ms
3 BrightTag / Signal 3,666 158 ms
4 Adobe Tag Manager 32,466 311 ms
5 Tealium 12,103 362 ms
6 Ensighten 3,761 444 ms

Mixed / Other

These are miscellaneous scripts delivered via a shared origin with no precise category or attribution. Help us out by identifying more origins!

Rank Name Usage Average Impact
1 ResponsiveVoice 1,241 70 ms
2 Amazon Web Services 38,265 161 ms
3 All Other 3rd Parties 1,380,545 318 ms
4 Parking Crew 5,147 326 ms
5 Heroku 2,002 607 ms
6 uLogin 2,316 1223 ms

Third Parties by Total Impact

This section highlights the entities responsible for the most script execution across the web. This helps inform which improvements would have the largest total impact.

Name Popularity Total Impact Average Impact
Google/Doubleclick Ads 680,948 991,570 s 1456 ms
All Other 3rd Parties 1,380,545 438,448 s 318 ms
YouTube 408,326 346,661 s 849 ms
Facebook 1,461,331 334,392 s 229 ms
Wix 54,909 327,376 s 5962 ms
Google CDN 947,402 272,512 s 288 ms
Yandex Metrica 292,542 169,622 s 580 ms
Google Maps 267,417 138,999 s 520 ms
Shopify 107,323 113,506 s 1058 ms
Other Google APIs/SDKs 580,415 97,529 s 168 ms
Google Analytics 1,200,666 92,871 s 77 ms
Google Tag Manager 721,295 83,294 s 115 ms
ZenDesk 69,488 62,011 s 892 ms
AddThis 118,289 47,729 s 403 ms
jQuery CDN 167,533 45,063 s 269 ms
Cloudflare CDN 117,287 39,767 s 339 ms
WordPress 92,546 37,424 s 404 ms
Squarespace 39,492 35,967 s 911 ms
Twitter 213,911 29,771 s 139 ms
Yandex APIs 27,480 29,681 s 1080 ms
Tawk.to 63,460 25,679 s 405 ms
Jivochat 45,110 24,937 s 553 ms
Cloudflare 53,319 24,772 s 465 ms
PIXNET 15,434 23,272 s 1508 ms
AMP 74,549 22,928 s 308 ms
POWr 16,407 22,375 s 1364 ms
Hatena Blog 13,205 20,853 s 1579 ms
FontAwesome CDN 47,278 19,542 s 413 ms
Weebly 13,271 19,021 s 1433 ms
JSDelivr CDN 97,319 18,827 s 193 ms
Pubmatic 52,613 16,557 s 315 ms
Tumblr 7,972 16,326 s 2048 ms
Hotjar 177,468 14,877 s 84 ms
Criteo 62,125 14,701 s 237 ms
Sumo 18,438 12,647 s 686 ms
Wistia 13,083 12,145 s 928 ms
Smartsupp 14,862 11,628 s 782 ms
Judge.me 8,307 11,424 s 1375 ms
Bridgewell DSP 14,079 11,133 s 791 ms
Adobe Tag Manager 32,466 10,104 s 311 ms
Bigcommerce 10,096 9,854 s 976 ms
ShareThis 40,133 9,398 s 234 ms
Stripe 23,538 9,143 s 388 ms
Taboola 24,583 8,950 s 364 ms
Drift 6,565 8,400 s 1279 ms
Integral Ad Science 8,013 8,241 s 1028 ms
CreateJS CDN 3,516 8,169 s 2323 ms
Yandex Ads 25,906 8,113 s 313 ms
Esri ArcGIS 1,692 8,036 s 4750 ms
LiveChat 19,468 8,001 s 411 ms
Blogger 23,674 7,742 s 327 ms
Tray Commerce 3,173 7,388 s 2328 ms
Sentry 9,061 6,480 s 715 ms
Amazon Web Services 38,265 6,159 s 161 ms
Hubspot 33,125 6,001 s 181 ms
Tilda 7,842 5,858 s 747 ms
Klaviyo 14,593 5,713 s 392 ms
Adroll 47,959 5,625 s 117 ms
Moat 6,862 5,604 s 817 ms
Snapchat 13,344 5,466 s 410 ms
CDK Dealer Management 3,429 5,376 s 1568 ms
Brightcove 6,615 5,352 s 809 ms
Rambler 7,698 5,318 s 691 ms
Adloox 5,759 5,306 s 921 ms
PayPal 15,436 5,286 s 342 ms
Salesforce 10,892 5,206 s 478 ms
PureCars 2,697 5,127 s 1901 ms
Media.net 14,529 4,949 s 341 ms
Bugsnag 8,686 4,945 s 569 ms
Histats 13,537 4,884 s 361 ms
LiveJournal 3,680 4,882 s 1327 ms
Tidio Live Chat 12,655 4,849 s 383 ms
AB Tasty 3,010 4,798 s 1594 ms
Mailchimp 23,373 4,766 s 204 ms
MGID 8,210 4,716 s 574 ms
GoDaddy 6,687 4,678 s 700 ms
Mapbox 5,206 4,568 s 877 ms
Olark 6,903 4,528 s 656 ms
Yandex Share 24,181 4,455 s 184 ms
SoundCloud 4,464 4,404 s 986 ms
Tealium 12,103 4,385 s 362 ms
Optimizely 19,583 4,375 s 223 ms
MediaVine 6,490 4,251 s 655 ms
Privy 13,047 4,036 s 309 ms
fam 2,135 3,857 s 1807 ms
Intercom 15,656 3,829 s 245 ms
Dealer 8,637 3,737 s 433 ms
Lucky Orange 7,529 3,699 s 491 ms
Vimeo 10,403 3,689 s 355 ms
Rubicon Project 39,353 3,558 s 90 ms
WebsiteBuilder.com 738 3,441 s 4663 ms
mPulse 13,177 3,243 s 246 ms
Adobe Business Catalyst 3,181 3,089 s 971 ms
Cookiebot 20,838 3,062 s 147 ms
Kakao 18,109 2,853 s 158 ms
uLogin 2,316 2,832 s 1223 ms
Embedly 5,513 2,831 s 514 ms
Yotpo 13,629 2,754 s 202 ms
WordAds 18,478 2,745 s 149 ms
Amazon Ads 25,338 2,710 s 107 ms
Skimbit 15,819 2,647 s 167 ms
Trust Pilot 15,202 2,488 s 164 ms
Akamai 7,316 2,455 s 336 ms
Azure Web Services 10,575 2,377 s 225 ms
Permutive 9,556 2,326 s 243 ms
Dailymotion 1,838 2,287 s 1244 ms
VigLink 7,097 2,191 s 309 ms
Infolinks 3,699 2,143 s 579 ms
AddToAny 24,490 2,119 s 87 ms
Twitch 1,068 2,025 s 1896 ms
Inspectlet 5,605 2,022 s 361 ms
Tynt 22,558 1,960 s 87 ms
OptinMonster 7,493 1,950 s 260 ms
Spotify 3,225 1,943 s 602 ms
LKQD 1,335 1,935 s 1449 ms
Unpkg 6,922 1,896 s 274 ms
Bold Commerce 13,095 1,885 s 144 ms
Hotmart 810 1,878 s 2318 ms
Fastly 6,694 1,841 s 275 ms
Bing Ads 15,786 1,838 s 116 ms
Segment 9,541 1,825 s 191 ms
Instagram 9,441 1,734 s 184 ms
Adobe TypeKit 18,274 1,723 s 94 ms
DTSCOUT 12,679 1,678 s 132 ms
Parking Crew 5,147 1,677 s 326 ms
Ensighten 3,761 1,668 s 444 ms
VK 13,473 1,631 s 121 ms
ContactAtOnce 3,247 1,594 s 491 ms
iubenda 12,781 1,586 s 124 ms
LinkedIn 14,038 1,559 s 111 ms
Sizmek 1,484 1,554 s 1047 ms
Pinterest 17,218 1,536 s 89 ms
Signyfyd 1,691 1,466 s 867 ms
FullStory 7,654 1,408 s 184 ms
Gigya 2,261 1,310 s 579 ms
issuu 1,934 1,295 s 670 ms
Adform 6,772 1,279 s 189 ms
LongTail Ad Solutions 3,433 1,252 s 365 ms
Disqus 1,252 1,245 s 994 ms
New Relic 13,062 1,242 s 95 ms
Heroku 2,002 1,216 s 607 ms
Nielsen NetRatings SiteCensus 11,805 1,204 s 102 ms
Secomapp 2,150 1,174 s 546 ms
Baidu Analytics 10,739 1,141 s 106 ms
Okas Concepts 673 1,112 s 1652 ms
Dynamic Yield 964 1,095 s 1136 ms
LoyaltyLion 1,300 1,079 s 830 ms
Yahoo! 6,183 1,062 s 172 ms
Sekindo 376 1,054 s 2804 ms
Ecwid 1,399 1,033 s 739 ms
Optanon 8,025 1,028 s 128 ms
Microsoft Hosted Libs 4,941 1,006 s 204 ms
Amazon Pay 5,751 974 s 169 ms
DoubleVerify 1,772 970 s 548 ms
DigiTrust 485 969 s 1999 ms
Albacross 1,920 941 s 490 ms
Intent Media 8,019 931 s 116 ms
Salesforce Commerce Cloud 2,623 897 s 342 ms
sovrn 4,673 890 s 190 ms
Monotype 4,073 862 s 212 ms
Outbrain 8,189 854 s 104 ms
Kaltura Video Platform 506 852 s 1684 ms
Feefo.com 1,686 846 s 502 ms
Technorati 830 822 s 991 ms
Cxense 4,374 783 s 179 ms
Ezoic 1,329 773 s 582 ms
VWO 4,724 748 s 158 ms
Bazaarvoice 1,845 732 s 397 ms
33 Across 1,188 730 s 615 ms
AdMatic 793 719 s 906 ms
Vidible 1,146 716 s 625 ms
CallRail 7,019 710 s 101 ms
Revolver Maps 1,144 697 s 609 ms
Yieldify 513 683 s 1332 ms
AppDynamics 1,281 675 s 527 ms
Admixer for Publishers 1,043 669 s 642 ms
RevJet 1,210 665 s 550 ms
Index Exchange 4,946 655 s 133 ms
Smart AdServer 5,189 654 s 126 ms
StickyADS.tv 838 650 s 776 ms
SocialShopWave 2,044 617 s 302 ms
Teads 3,457 612 s 177 ms
Scorecard Research 6,956 600 s 86 ms
Vidyard 536 587 s 1094 ms
LINE Corporation 5,182 585 s 113 ms
OpenX 3,015 583 s 193 ms
Fraudlogix 2,244 580 s 258 ms
Pardot 1,435 579 s 404 ms
BrightTag / Signal 3,666 578 s 158 ms
ZEDO 794 576 s 726 ms
Mixpanel 7,513 567 s 75 ms
Help Scout 2,183 563 s 258 ms
Snowplow 5,845 555 s 95 ms
TINT 224 553 s 2470 ms
Crazy Egg 4,988 551 s 110 ms
Medium 1,157 549 s 474 ms
IBM Digital Analytics 806 546 s 677 ms
piano 1,183 543 s 459 ms
RD Station 6,545 496 s 76 ms
LivePerson 4,302 484 s 112 ms

Future Work

  1. Introduce URL-level data for more fine-grained analysis, i.e. which libraries from Cloudflare/Google CDNs are most expensive.
  2. Expand the scope, i.e. include more third parties and have greater entity/category coverage.

FAQs

I don't see entity X in the list. What's up with that?

This can be for one of several reasons:

  1. The entity does not have references to their origin on at least 50 pages in the dataset.
  2. The entity's origins have not yet been identified. See How can I contribute?

What is "Total Occurences"?

Total Occurrences is the number of pages on which the entity is included.

How is the "Average Impact" determined?

The HTTP Archive dataset includes Lighthouse reports for each URL on mobile. Lighthouse has an audit called "bootup-time" that summarizes the amount of time that each script spent on the main thread. The "Average Impact" for an entity is the total execution time of scripts whose domain matches one of the entity's domains divided by the total number of pages that included the entity.

Average Impact = Total Execution Time / Total Occurrences

How does Lighthouse determine the execution time of each script?

Lighthouse's bootup time audit attempts to attribute all toplevel main-thread tasks to a URL. A main thread task is attributed to the first script URL found in the stack. If you're interested in helping us improve this logic, see Contributing for details.

The data for entity X seems wrong. How can it be corrected?

Verify that the origins in data/entities.json5 are correct. Most issues will simply be the result of mislabelling of shared origins. If everything checks out, there is likely no further action and the data is valid. If you still believe there's errors, file an issue to discuss futher.

How can I contribute?

Only about 90% of the third party script execution has been assigned to an entity. We could use your help identifying the rest! See Contributing for details.

Contributing

Thanks

A huge thanks to @simonhearne and @soulgalore for their assistance in classifying additional domains!

Updating the Entities

The domain->entity mapping can be found in data/entities.json5. Adding a new entity is as simple as adding a new array item with the following form.

{
    "name": "Facebook",
    "homepage": "https://www.facebook.com",
    "categories": ["social"],
    "domains": [
        "*.facebook.com",
        "*.fbcdn.net"
    ],
    "examples": [
        "www.facebook.com",
        "connect.facebook.net",
        "staticxx.facebook.com",
        "static.xx.fbcdn.net",
        "m.facebook.com"
    ]
}

Updating Attribution Logic

The logic for attribution to individual script URLs can be found in the Lighthouse repo. File an issue over there to discuss further.

Updating the Data

The queries used to compute the data are in the sql/ directory.

  1. Edit all-observed-domains-query.sql to query the correct month's HTTPArchive run.
  2. Run all-observed-domains-query.sql in BigQuery.
  3. Download the results and check them in at data/YYYY-MM-01-observed-domains.json.
  4. Edit bootup-time-scripting.partial.sql to query the correct month's HTTPArchive run.
  5. Run origin-query.generated.sql in BigQuery.
  6. Download the results and check them in at data/YYYY-MM-01-origin-scripting.json.
  7. Run yarn build to regenerate the latest canonical domain mapping.
  8. Create a new table in lighthouse-infrastructure.third_party_web BigQuery table of the format YYYY_MM_01 with the csv in dist/domain-map.csv with three columns domain, canonicalDomain, and category.
  9. Edit bootup-time-scripting.partial.sql to join on the table you just created.
  10. Run yarn build to regenerate the queries.
  11. Run entity-per-page.generated.sql in BigQuery.
  12. Download the results and check them in at data/YYYY-MM-01-entity-scripting.json.
  13. Run web-almanac-all-observed-domains-identification.sql in BigQuery.
  14. Save the results to a BigQuery table YYYY_MM_01_all_observed_domains.

Updating this README

This README is auto-generated from the templates lib/ and the computed data. In order to update the charts, you'll need to make sure you have cairo installed locally in addition to yarn install.

# Install `cairo` and dependencies for node-canvas
brew install pkg-config cairo pango libpng jpeg giflib
# Build the requirements in this repo
yarn build
# Regenerate the README
yarn start

Updating the website

The web code is located in www/ directory of this repository. Open a PR to make changes.

You can’t perform that action at this time.