Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core(lhr): expose environment info #5871

Merged
merged 4 commits into from
Aug 23, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions lighthouse-core/gather/driver.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,14 @@ class Driver {
return this.evaluateAsync('navigator.userAgent');
}

/**
* Computes the ULTRADUMB™ benchmark index to get a rough estimate of device class.
* @return {Promise<number>}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will you add a simple docstring here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, done

*/
getBenchmarkIndex() {
return this.evaluateAsync(`(${pageFunctions.ultradumbBenchmark.toString()})()`);
}

/**
* @return {Promise<void>}
*/
Expand Down
15 changes: 14 additions & 1 deletion lighthouse-core/gather/gather-runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,9 @@ class GatherRunner {
return {
fetchTime: (new Date()).toJSON(),
LighthouseRunWarnings: [],
UserAgent: await options.driver.getUserAgent(),
HostUserAgent: await options.driver.getUserAgent(),
NetworkUserAgent: '', // updated later
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NetworkUserAgent doesn't seem self-obvious to me but I don't have a great suggestion for an alternative. Will brainstorm :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sg, I also tried RuntimeUserAgent but thought NetworkUserAgent was a bit more descriptive

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. I still have nothings, so that's fine with me. If @paulirish has ideas we can do a follow up PR quickly before we ship it :)

BenchmarkIndex: 0, // updated later
traces: {},
devtoolsLogs: {},
settings: options.settings,
Expand All @@ -397,6 +399,7 @@ class GatherRunner {
await driver.connect();
const baseArtifacts = await GatherRunner.getBaseArtifacts(options);
await GatherRunner.loadBlank(driver);
baseArtifacts.BenchmarkIndex = await options.driver.getBenchmarkIndex();
await GatherRunner.setupDriver(driver, options);

// Run each pass
Expand All @@ -420,6 +423,16 @@ class GatherRunner {
// Save devtoolsLog, but networkRecords are discarded and not added onto artifacts.
baseArtifacts.devtoolsLogs[passConfig.passName] = passData.devtoolsLog;

const userAgentEntry = passData.devtoolsLog.find(entry =>
entry.method === 'Network.requestWillBeSent' &&
!!entry.params.request.headers['User-Agent']
);

if (userAgentEntry && !baseArtifacts.NetworkUserAgent) {
// @ts-ignore - guaranteed to exist by the find above
baseArtifacts.NetworkUserAgent = userAgentEntry.params.request.headers['User-Agent'];
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also do baseArtifacts.NetworkUserAgent = await driver.getUserAgent() after GatherRunner.setupDriver() is called. It's shorter, but is it better to just look at the actual network traffic?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the number of ways that have popped up in issues as the way folks are trying to set the user agent string, I thought it was best to grab it directly from network records instead of assume it's done being manipulated after setupDriver

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SG. I guess HostUserAgent might end up weird too if people are overriding things


// If requested by config, save pass's trace.
if (passData.trace) {
baseArtifacts.traces[passConfig.passName] = passData.trace;
Expand Down
32 changes: 30 additions & 2 deletions lighthouse-core/lib/page-functions.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ function registerPerformanceObserverInPage() {
window.____lhPerformanceObserver = observer;
}


/**
* Used by _waitForCPUIdle and executed in the context of the page, returns time since last long task.
*/
Expand Down Expand Up @@ -114,7 +113,35 @@ function getElementsInDocument(selector) {
function getOuterHTMLSnippet(element) {
const reOpeningTag = /^.*?>/;
const match = element.outerHTML.match(reOpeningTag);
return match && match[0] || '';
return (match && match[0]) || '';
}

/**
* Computes a memory/CPU performance benchmark index to determine rough device class.
* @see https://docs.google.com/spreadsheets/d/1E0gZwKsxegudkjJl8Fki_sOwHKpqgXwt8aBAfuUaB8A/edit?usp=sharing
*
* The benchmark creates a string of length 100,000 in a loop.
* The returned index is the number of times per second the string can be created.
*
* - 750+ is a desktop-class device, Core i3 PC, iPhone X, etc
* - 300+ is a high-end Android phone, Galaxy S8, low-end Chromebook, etc
* - 75+ is a mid-tier Android phone, Nexus 5X, etc
* - <75 is a budget Android phone, Alcatel Ideal, Galaxy J2, etc
*/
/* istanbul ignore next */
function ultradumbBenchmark() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we just had a whole thing with page-functions, but I kind of wish this was in its own file for easier running :) I guess we can always add yarn ultradumbBenchmark later

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done :)
image

const start = Date.now();
let iterations = 0;

while (Date.now() - start < 500) {
let s = ''; // eslint-disable-line no-unused-vars
for (let j = 0; j < 100000; j++) s += 'a';

iterations++;
}

const durationInSeconds = (Date.now() - start) / 1000;
return iterations / durationInSeconds;
}

module.exports = {
Expand All @@ -123,4 +150,5 @@ module.exports = {
checkTimeSinceLastLongTask,
getElementsInDocument,
getOuterHTMLSnippet,
ultradumbBenchmark,
};
8 changes: 7 additions & 1 deletion lighthouse-core/report/html/renderer/report-renderer.js
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,14 @@ class ReportRenderer {
{name: 'URL', description: report.finalUrl},
{name: 'Fetch time', description: Util.formatDateTime(report.fetchTime)},
...envValues,
{name: 'User agent', description: report.userAgent},
{name: 'User agent (host)', description: report.userAgent},
{name: 'User agent (network)', description: report.environment &&
report.environment.networkUserAgent},
{name: 'CPU/Memory Power', description: report.environment &&
report.environment.benchmarkIndex.toFixed(0)},
].forEach(runtime => {
if (!runtime.description) return;

const item = this._dom.cloneTemplate('#tmpl-lh-env__items', env);
this._dom.find('.lh-env__name', item).textContent = `${runtime.name}:`;
this._dom.find('.lh-env__description', item).textContent = runtime.description;
Expand Down
7 changes: 6 additions & 1 deletion lighthouse-core/runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,12 @@ class Runner {

/** @type {LH.Result} */
const lhr = {
userAgent: artifacts.UserAgent,
userAgent: artifacts.HostUserAgent,
environment: {
networkUserAgent: artifacts.NetworkUserAgent,
hostUserAgent: artifacts.HostUserAgent,
benchmarkIndex: artifacts.BenchmarkIndex,
},
lighthouseVersion,
fetchTime: artifacts.fetchTime,
requestedUrl: requestedUrl,
Expand Down
24 changes: 24 additions & 0 deletions lighthouse-core/scripts/benchmark.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env node

/**
* @license Copyright 2018 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

/* eslint-disable no-console */

const ultradumbBenchmark = require('../lib/page-functions').ultradumbBenchmark;

console.log('Running ULTRADUMB™ benchmark 10 times...')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lint says you need a ; :)


let total = 0;
for (let i = 0; i < 10; i++) {
const result = ultradumbBenchmark();
console.log(`Result ${i + 1}: ${result.toFixed(0)}`);
total += result;
}

console.log('----------------------------------------');
console.log('Final result:', (total / 10).toFixed(0));
3 changes: 3 additions & 0 deletions lighthouse-core/test/gather/fake-driver.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ module.exports = {
getUserAgent() {
return Promise.resolve('Fake user agent');
},
getBenchmarkIndex() {
return Promise.resolve(125.2);
},
connect() {
return Promise.resolve();
},
Expand Down
29 changes: 25 additions & 4 deletions lighthouse-core/test/gather/gather-runner-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,16 +106,37 @@ describe('GatherRunner', function() {
});
});

it('collects user agent as an artifact', () => {
it('collects benchmark as an artifact', async () => {
const url = 'https://example.com';
const driver = fakeDriver;
const config = new Config({});
const settings = {};
const options = {url, driver, config, settings};

return GatherRunner.run([], options).then(results => {
assert.equal(results.UserAgent, 'Fake user agent', 'did not find expected user agent string');
});
const results = await GatherRunner.run([], options);
expect(Number.isFinite(results.BenchmarkIndex)).toBeTruthy();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like you must be trolling here...

});

it('collects host user agent as an artifact', async () => {
const url = 'https://example.com';
const driver = fakeDriver;
const config = new Config({});
const settings = {};
const options = {url, driver, config, settings};

const results = await GatherRunner.run([], options);
expect(results.HostUserAgent).toEqual('Fake user agent');
});

it('collects network user agent as an artifact', async () => {
const url = 'https://example.com';
const driver = fakeDriver;
const config = new Config({passes: [{}]});
const settings = {};
const options = {url, driver, config, settings};

const results = await GatherRunner.run(config.passes, options);
expect(results.NetworkUserAgent).toContain('Mozilla');
});

it('collects requested and final URLs as an artifact', () => {
Expand Down
4 changes: 3 additions & 1 deletion lighthouse-core/test/results/artifacts/artifacts.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
{
"LighthouseRunWarnings": [],
"UserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3358.0 Safari/537.36",
"BenchmarkIndex": 1000,
"HostUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3358.0 Safari/537.36",
"NetworkUserAgent": "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36(KHTML, like Gecko) Chrome/66.0.3359.30 Mobile Safari/537.36",
"fetchTime": "2018-03-13T00:55:45.840Z",
"URL": {
"requestedUrl": "http://localhost:10200/dobetterweb/dbw_tester.html",
Expand Down
5 changes: 5 additions & 0 deletions lighthouse-core/test/results/sample_v2.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
{
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3358.0 Safari/537.36",
"environment": {
"networkUserAgent": "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36(KHTML, like Gecko) Chrome/66.0.3359.30 Mobile Safari/537.36",
"hostUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3358.0 Safari/537.36",
"benchmarkIndex": 1000
},
"lighthouseVersion": "3.0.3",
"fetchTime": "2018-03-13T00:55:45.840Z",
"requestedUrl": "http://localhost:10200/dobetterweb/dbw_tester.html",
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
"update:sample-json": "yarn i18n:collect-strings && node ./lighthouse-cli -A=./lighthouse-core/test/results/artifacts --throttling-method=devtools --output=json --output-path=./lighthouse-core/test/results/sample_v2.json && node lighthouse-core/scripts/cleanup-LHR-for-diff.js ./lighthouse-core/test/results/sample_v2.json --only-remove-timing",
"diff:sample-json": "yarn i18n:checks && bash lighthouse-core/scripts/assert-golden-lhr-unchanged.sh",
"update:crdp-typings": "node lighthouse-core/scripts/extract-crdp-mapping.js",
"ultradumbBenchmark": "./lighthouse-core/scripts/benchmark.js",
"mixed-content": "./lighthouse-cli/index.js --chrome-flags='--headless' --preset=mixed-content",
"minify-latest-run": "./lighthouse-core/scripts/lantern/minify-trace.js ./latest-run/defaultPass.trace.json ./latest-run/defaultPass.trace.min.json && ./lighthouse-core/scripts/lantern/minify-devtoolslog.js ./latest-run/defaultPass.devtoolslog.json ./latest-run/defaultPass.devtoolslog.min.json"
},
Expand Down
8 changes: 6 additions & 2 deletions typings/artifacts.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,12 @@ declare global {
fetchTime: string;
/** A set of warnings about unexpected things encountered while loading and testing the page. */
LighthouseRunWarnings: string[];
/** The user agent string of the version of Chrome that was used by Lighthouse. */
UserAgent: string;
/** The user agent string of the version of Chrome used. */
HostUserAgent: string;
/** The user agent string that Lighthouse used to load the page. */
NetworkUserAgent: string;
/** The benchmark index that indicates rough device class. */
BenchmarkIndex: number;
/** A set of page-load traces, keyed by passName. */
traces: {[passName: string]: Trace};
/** A set of DevTools debugger protocol records, keyed by passName. */
Expand Down
11 changes: 11 additions & 0 deletions typings/lhr.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,15 @@ declare global {
[varName: string]: string;
}

export interface Environment {
/** The user agent string of the version of Chrome used. */
hostUserAgent: string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you copy the above doc strings down for these too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

/** The user agent string that was sent over the network. */
networkUserAgent: string;
/** The benchmark index number that indicates rough device class. */
benchmarkIndex: number;
}

/**
* The full output of a Lighthouse run.
*/
Expand Down Expand Up @@ -43,6 +52,8 @@ declare global {
runWarnings: string[];
/** The User-Agent string of the browser used run Lighthouse for these results. */
userAgent: string;
/** Information about the environment in which Lighthouse was run. */
environment: Environment;
/** Execution timings for the Lighthouse run */
timing: {total: number, [t: string]: number};
/** The record of all formatted string locations in the LHR and their corresponding source values. */
Expand Down