-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
core: add inline scripts to Scripts artifact #7065
Changes from 3 commits
fac5bed
a67cccb
a8debfb
8535f0b
a959a03
13067d4
0c28f25
7a2a75b
554d398
f02e0b6
9321d6a
0fe823a
80a062e
b5a0912
a38a548
9fc4e6d
cc8e401
e78d8b5
98509ba
8610f78
c01ca17
54f416c
9abd2f9
5c133cb
28bfe98
b9f82ae
0ecd754
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,6 +7,8 @@ | |
|
||
const Gatherer = require('./gatherer'); | ||
const NetworkRequest = require('../../lib/network-request'); | ||
const getElementsInDocumentString = require('../../lib/page-functions.js').getElementsInDocumentString; // eslint-disable-line max-len | ||
const URL = require('../../lib/url-shim.js'); | ||
|
||
/** | ||
* @fileoverview Gets JavaScript file contents. | ||
|
@@ -20,21 +22,46 @@ class Scripts extends Gatherer { | |
async afterPass(passContext, loadData) { | ||
const driver = passContext.driver; | ||
|
||
/** @type {Object<string, string>} */ | ||
const scriptContentMap = {}; | ||
/** @type {LH.Artifacts['Scripts']} */ | ||
brendankenny marked this conversation as resolved.
Show resolved
Hide resolved
|
||
const scripts = []; | ||
|
||
/** @type {string[]} */ | ||
const inlineScripts = await driver.evaluateAsync(`(() => { | ||
${getElementsInDocumentString}; | ||
|
||
return getElementsInDocument('script') | ||
.filter(meta => !meta.src && meta.text.trim()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. seems like this one isn't a |
||
.map(meta => meta.text); | ||
})()`, {useIsolation: true}); | ||
|
||
if (inlineScripts.length) { | ||
const mainResource = loadData.networkRecords.find( | ||
request => URL.equalWithExcludedFragments(request.url, passContext.url)); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the other instances of finding the main resource mention Agreed it'd be much preferable to call a centralized version of this. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done. after the two merge PRs merge maybe we should just move that check into There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
yeah it's a bit different to do generically because in these cases we know for a fact there are no fragments on the network records but agreed we should not have to do these things, the URL.* methods should just be fast by default :) |
||
if (!mainResource) { | ||
throw new Error('could not locate mainResource'); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm a teensy bit worried that this is going to start throwing errors into our perf category on edge cases we previously didn't really care about. We don't actually need a request ID for the whole thing to work. WDYT about making There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sounds alright, but what are these edge cases? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If I knew them I would fix them :) I just meant that we have seen this error popup in peoples reports before and it's mostly 🤷♂️and move on since it wasn't that big a deal. Now it'll be a bit more user-facing. I actually had to tackle this for canonical though and think it'll be solid. https://github.com/GoogleChrome/lighthouse/pull/7080/files#diff-2113a8215d43931339dbb50287d89dcbR450 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done. thoughts on handling reporting of unminified scripts in the JS audit? I just put There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. also, one of us should extract that function to somewhere else (whoever merges last) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. extract it off of though if you mean we should finally move There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Ideally we'd try to provide a snippet of the code like we do for CSS, but I'm OK with this for now until we see how common it is. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe when the fancy code snippet preview (#6901) lands we can do something cool here :D There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah having it in 💯 to snippets |
||
} | ||
scripts.push(...inlineScripts.map(code => ({ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we've generally preferred the explicit cool with adding an explicit |
||
code, | ||
requestId: mainResource.requestId, | ||
}))); | ||
} | ||
|
||
const scriptRecords = loadData.networkRecords | ||
.filter(record => record.resourceType === NetworkRequest.TYPES.Script); | ||
|
||
for (const record of scriptRecords) { | ||
try { | ||
const content = await driver.getRequestContent(record.requestId); | ||
if (content) { | ||
scriptContentMap[record.requestId] = content; | ||
scripts.push({ | ||
code: content, | ||
requestId: record.requestId, | ||
}); | ||
} | ||
} catch (e) {} | ||
} | ||
|
||
return scriptContentMap; | ||
return scripts; | ||
} | ||
} | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,35 +16,44 @@ const resourceType = 'Script'; | |
describe('Page uses optimized responses', () => { | ||
it('fails when given unminified scripts', () => { | ||
const auditResult = UnminifiedJavascriptAudit.audit_({ | ||
Scripts: { | ||
'123.1': | ||
` | ||
Scripts: [ | ||
{ | ||
requestId: '123.1', | ||
code: ` | ||
var foo = new Set(); | ||
foo.add(1); | ||
foo.add(2); | ||
|
||
if (foo.has(2)) { | ||
console.log('hello!') | ||
} | ||
`, | ||
'123.2': | ||
` | ||
`, | ||
}, | ||
{ | ||
requestId: '123.2', | ||
code: ` | ||
const foo = new Set(); | ||
foo.add(1); | ||
|
||
async function go() { | ||
await foo.has(1) | ||
console.log('yay esnext!') | ||
} | ||
`, | ||
'123.3': | ||
/* eslint-disable no-useless-escape */ | ||
`, | ||
}, | ||
{ | ||
requestId: '123.3', | ||
code: /* eslint-disable no-useless-escape */ | ||
` | ||
const foo = 1 | ||
/Edge\/\d*\.\d*/.exec('foo') | ||
`, | ||
'123.4': '#$*%dense', | ||
}, | ||
}, | ||
{ | ||
requestId: '123.4', | ||
code: '#$*%dense', | ||
}, | ||
], | ||
}, [ | ||
{requestId: '123.1', url: 'foo.js', transferSize: 20 * KB, resourceType}, | ||
{requestId: '123.2', url: 'other.js', transferSize: 50 * KB, resourceType}, | ||
|
@@ -58,30 +67,36 @@ describe('Page uses optimized responses', () => { | |
})); | ||
|
||
expect(results).toMatchObject([ | ||
{url: 'foo.js', wastedPercent: 57, wastedKB: 11}, | ||
{url: 'other.js', wastedPercent: 53, wastedKB: 27}, | ||
{url: 'foo.js', wastedPercent: 56, wastedKB: 11}, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why did this change? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. whitespace changes in the provided network record contents. the |
||
{url: 'other.js', wastedPercent: 53, wastedKB: 26}, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same here |
||
{url: 'valid-ish.js', wastedPercent: 39, wastedKB: 39}, | ||
]); | ||
}); | ||
|
||
it('passes when scripts are already minified', () => { | ||
const auditResult = UnminifiedJavascriptAudit.audit_({ | ||
Scripts: { | ||
'123.1': | ||
'var f=new Set();f.add(1);f.add(2);if(f.has(2))console.log(1234)', | ||
'123.2': | ||
` | ||
const foo = new Set(); | ||
foo.add(1); | ||
Scripts: [ | ||
{ | ||
requestId: '123.1', | ||
code: 'var f=new Set();f.add(1);f.add(2);if(f.has(2))console.log(1234)', | ||
}, | ||
{ | ||
requestId: '123.2', | ||
code: ` | ||
const foo = new Set(); | ||
foo.add(1); | ||
|
||
async function go() { | ||
await foo.has(1) | ||
console.log('yay esnext!') | ||
} | ||
`, | ||
'123.3': | ||
'for{(wtf', | ||
}, | ||
async function go() { | ||
await foo.has(1) | ||
console.log('yay esnext!') | ||
} | ||
`, | ||
}, | ||
{ | ||
requestId: '123.3', | ||
code: 'for{(wtf', | ||
}, | ||
], | ||
}, [ | ||
{requestId: '123.1', url: 'foo.js', transferSize: 20 * KB, resourceType}, | ||
{requestId: '123.2', url: 'other.js', transferSize: 3 * KB, resourceType}, // too small | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1998,41 +1998,10 @@ | |
"id": "unminified-javascript", | ||
"title": "Minify JavaScript", | ||
"description": "Minifying JavaScript files can reduce payload sizes and script parse time. [Learn more](https://developers.google.com/speed/docs/insights/MinifyResources).", | ||
"score": 0.88, | ||
"scoreDisplayMode": "numeric", | ||
"rawValue": 150, | ||
"displayValue": "Potential savings of 30 KB", | ||
"warnings": [], | ||
"details": { | ||
"type": "opportunity", | ||
"headings": [ | ||
{ | ||
"key": "url", | ||
"valueType": "url", | ||
"label": "URL" | ||
}, | ||
{ | ||
"key": "totalBytes", | ||
"valueType": "bytes", | ||
"label": "Size (KB)" | ||
}, | ||
{ | ||
"key": "wastedBytes", | ||
"valueType": "bytes", | ||
"label": "Potential Savings (KB)" | ||
} | ||
], | ||
"items": [ | ||
{ | ||
"url": "http://localhost:10200/zone.js", | ||
"totalBytes": 71654, | ||
"wastedBytes": 30470, | ||
"wastedPercent": 42.52388078488413 | ||
} | ||
], | ||
"overallSavingsMs": 150, | ||
"overallSavingsBytes": 30470 | ||
} | ||
"score": null, | ||
"scoreDisplayMode": "error", | ||
"rawValue": null, | ||
"errorMessage": "artifacts.Scripts is not iterable" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. update the artifacts.json? |
||
}, | ||
"unused-css-rules": { | ||
"id": "unused-css-rules", | ||
|
@@ -4549,7 +4518,6 @@ | |
"audits[uses-long-cache-ttl].details.headings[0].text", | ||
"audits[total-byte-weight].details.headings[0].text", | ||
"audits[render-blocking-resources].details.headings[0].label", | ||
"audits[unminified-javascript].details.headings[0].label", | ||
"audits[uses-webp-images].details.headings[1].label", | ||
"audits[uses-text-compression].details.headings[0].label" | ||
], | ||
|
@@ -4819,7 +4787,6 @@ | |
"audits[uses-long-cache-ttl].details.headings[2].text", | ||
"audits[total-byte-weight].details.headings[1].text", | ||
"audits[render-blocking-resources].details.headings[1].label", | ||
"audits[unminified-javascript].details.headings[1].label", | ||
"audits[uses-webp-images].details.headings[2].label", | ||
"audits[uses-text-compression].details.headings[1].label" | ||
], | ||
|
@@ -4872,13 +4839,19 @@ | |
"lighthouse-core/audits/byte-efficiency/unminified-javascript.js | description": [ | ||
"audits[unminified-javascript].description" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/unused-css-rules.js | title": [ | ||
"audits[unused-css-rules].title" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/unused-css-rules.js | description": [ | ||
"audits[unused-css-rules].description" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/uses-webp-images.js | title": [ | ||
"audits[uses-webp-images].title" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/uses-webp-images.js | description": [ | ||
"audits[uses-webp-images].description" | ||
], | ||
"lighthouse-core/lib/i18n/i18n.js | displayValueByteSavings": [ | ||
{ | ||
"values": { | ||
"wastedBytes": 30470 | ||
}, | ||
"path": "audits[unminified-javascript].displayValue" | ||
}, | ||
{ | ||
"values": { | ||
"wastedBytes": 8526 | ||
|
@@ -4893,22 +4866,9 @@ | |
} | ||
], | ||
"lighthouse-core/lib/i18n/i18n.js | columnWastedBytes": [ | ||
"audits[unminified-javascript].details.headings[2].label", | ||
"audits[uses-webp-images].details.headings[3].label", | ||
"audits[uses-text-compression].details.headings[2].label" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/unused-css-rules.js | title": [ | ||
"audits[unused-css-rules].title" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/unused-css-rules.js | description": [ | ||
"audits[unused-css-rules].description" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/uses-webp-images.js | title": [ | ||
"audits[uses-webp-images].title" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/uses-webp-images.js | description": [ | ||
"audits[uses-webp-images].description" | ||
], | ||
"lighthouse-core/audits/byte-efficiency/uses-optimized-images.js | title": [ | ||
"audits[uses-optimized-images].title" | ||
], | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -107,8 +107,8 @@ declare global { | |
RobotsTxt: {status: number|null, content: string|null}; | ||
/** Set of exceptions thrown during page load. */ | ||
RuntimeExceptions: Crdp.Runtime.ExceptionThrownEvent[]; | ||
/** The content of all scripts loaded by the page, keyed by networkRecord requestId. */ | ||
Scripts: Record<string, string>; | ||
/** The content of all scripts loaded by the page, and the networkRecord requestId that loaded them. Note, HTML documents will have one entry for the same requestId per script tag. */ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe reverse these clauses, |
||
Scripts: Array<{code: string, requestId: string}>; | ||
/** Version information for all ServiceWorkers active after the first page load. */ | ||
ServiceWorker: {versions: Crdp.ServiceWorker.ServiceWorkerVersion[], registrations: Crdp.ServiceWorker.ServiceWorkerRegistration[]}; | ||
/** The status of an offline fetch of the page's start_url. -1 and a explanation if missing or there was an error. */ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now takes into account these two inline scripts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could we assert more in here to give insight about which 3 these should be? (shouldn't be exhaustive, but presumably the sort should be stable so could do urls)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done. the data was stable too so I include it.