Allow Perf-X server Rerun lighthouse on POST request. #1393

weiwei-lin · 2017-01-04T12:32:16Z

Make Perf-X server be able to handle lighthouse rerun request. After receiving the request, Perf-X server will run lighthouse with same parameters plus some additional flags provided in the POST request body and then responds with generated results.
This will allow us to rerun lighthouse with some URLblocking patterns within lighthouse report in the future.
Part of Perf-X project #1143

paulirish · 2017-01-04T19:32:27Z

Looks like your rebase messed up, as the rename from a previous PR is reverted here. Can you take a look?

paulirish · 2017-01-04T19:38:23Z

Do you want to also add a button to the report to rerun? That way we can test out this flow easily.

weiwei-lin · 2017-01-04T22:54:01Z

Do you want to also add a button to the report to rerun? That way we can test out this flow easily.

Yes, I will do that. But probably in another PR. Because I think it's not just a rerun button. We also need to pass additional cli-flags (bolckedUrlPatterns and probably --perf as well). Then we need to design and implement a way for the users to configure blocking patterns as well as a way to surface the rerun results.

If we want something in this PR to ease the test process, I think we can include a function to trigger rerun in report page, which is not visible to users but can be manually invoked in dev-tool console.

weiwei-lin · 2017-01-04T22:57:41Z

Looks like your rebase messed up, as the rename from a previous PR is reverted here. Can you take a look?

Actually, it's intentional. Because I thought the server now does more than just serve and open report. But yeah, rerun is not used anywhere yet. I have renamed the function to serveAndOpenReport again.

paulirish · 2017-01-04T23:03:56Z

I think we can include a function to trigger rerun in report page, which is not visible to users but can be manually invoked in dev-tool console.

that sgtm. let's do that.

brendankenny · 2017-01-05T00:01:43Z

For a possible followup to this:

One thing this makes me think of is since there's a lighthouse --view flag that runs lighthouse and then launches the performance experiment, which can then re-run lighthouse after this PR, maybe we should drop the --view flag and have something like lighthouse --interactive (or whatever) which just launches the performance experiment, with no starting report but with a big start button that kicks off the initial run.

weiwei-lin · 2017-01-05T01:49:27Z

maybe we should drop the --view flag and have something like lighthouse --interactive (or whatever) which just launches the performance experiment, with no starting report but with a big start button that kicks off the initial run.

Agree. I was thinking about renaming --view to --perf-x, but --interactive seems more intuitive.
I also considered including the initial run in perf-x, even moving perf-x entirely out of cli and becomes a stand-alone branch (like extension). Because in --perf-x, users will interact with lighthouse in browser not command line. Currently, we are still thinking about how to surface results and what is the most ideal workflow for users. After I have a clearer answer, I will implement those changes .

weiwei-lin · 2017-01-05T01:50:01Z

Will come back later to address some coding style issues.

weiwei-lin

Ready for review again. PTAL. :)

weiwei-lin · 2017-01-05T03:45:02Z

lighthouse-cli/performance-experiment/server.js

@@ -100,9 +102,10 @@ function rerunRequestHandler(request, response) {

      // Add more to flags without changing the original flags
      const flags = Object.assign({}, lhParams.flags, additionalFlags);
-      lighthouse(lhParams.url, flags, lhParams.configs).then(results => {
+      lighthouse(lhParams.url, flags, perfOnlyConfig).then(results => {


When we rerun tests with configurations (e.g. URL blocking patterns), we only care about performance. So use perfOnlyConfig here to get faster result.

weiwei-lin · 2017-01-05T03:48:22Z

lighthouse-core/report/scripts/.eslintrc.js

@@ -0,0 +1,7 @@
+module.exports = {


Allow us to use /* exported rerunLighthouse */ in perf-x-api.js

paulirish · 2017-01-05T08:18:17Z

the more I think about it the more i like the --interactive flag. (happy to rename it later)

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR. (We can still review them simultaneously, even with the dependency)

paulirish

mostly good. I am liking this direction...

Right now rerunlighthouse just logs the new results to the console. Don't we want to update the report?

paulirish · 2017-01-05T08:12:53Z

lighthouse-core/report/report-generator.js

+      scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/perf-x-api.js')));
+    }
+
+    scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));


awkward logic here.
probably better to do a `reportContext !== 'devtools' conditional around this last push(), and nuke the early exit on 211

Yeah. That's much better. Fixed.

paulirish · 2017-01-05T08:14:11Z

lighthouse-cli/performance-experiment/server.js

+  if (request.method === 'GET') {
+    if (pathname === '/') {
+      reportRequestHandler(request, response);
+    } else if (pathname === '/rerun') {


let's nuke the GET handler for rerun completely, yah?

Sorry, didn't notice that it's not deleted yet. Fixed.

weiwei-lin

Right now rerunlighthouse just logs the new results to the console. Don't we want to update the report?

In my original plan, this PR will only include back-end support for rerunning lighthouse but not front-end blocking patterns input interface or report update. Because the way to surface results and to configure blocking patterns is not decided yet.

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR

Hmm. I think if we add support for users to input blocking patterns in the report page in this PR, then we can rename --view in this PR. Otherwise, we'd better stick with --view for now because this mode is not interactive for users yet.

weiwei-lin · 2017-01-05T08:32:25Z

lighthouse-cli/performance-experiment/server.js

+  if (request.method === 'GET') {
+    if (pathname === '/') {
+      reportRequestHandler(request, response);
+    } else if (pathname === '/rerun') {


Sorry, didn't notice that it's not deleted yet. Fixed.

weiwei-lin · 2017-01-05T08:32:47Z

lighthouse-core/report/report-generator.js

+      scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/perf-x-api.js')));
+    }
+
+    scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));


Yeah. That's much better. Fixed.

paulirish · 2017-01-05T08:58:39Z

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR

Hmm. I think if we add support for users to input blocking patterns in the report page in this PR, then we can rename --view in this PR. Otherwise, we'd better stick with --view for now because this mode is not interactive for users yet.

Ah.. I was thinking just UI to trigger a re-run, and viewing of those new results.

That wouldn't include configuration of blocked URLs or comparing different runs. Those two are separate and something i know our designer Chris has been thinking about, as well.

ebidel · 2017-01-06T00:02:50Z

lighthouse-core/report/report-generator.js

+    }
+
+    if (reportContext !== 'devtools') {
+      scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));


why was the 'utf8' removed?

Sorry. I didn't add this when I rewrite this method because I thought 'utf8' is default anyway : (
I will add it back in the next commit. Would you mind to tell me why it's necessary though?

I think null is the default. https://nodejs.org/api/fs.html#fs_fs_readfilesync_file_options

My bad. Fixed in the next update! Thanks for pointing out!

weiwei-lin

Added a button in the top bar of the report page (only in --interactive mode).
Currently, clicking rerun button will just give user a new link to the new report. This will be changed later in the project.

weiwei-lin · 2017-01-06T04:29:52Z

lighthouse-cli/bin.ts

@@ -83,18 +83,17 @@ const cliFlags = yargs
    'port': 'The port to use for the debugging protocol. Use 0 for a random port',
    'skip-autolaunch': 'Skip autolaunch of Chrome when already running instance is not found',
    'select-chrome': 'Interactively choose version of Chrome to use when multiple installations are found',
+    'interactive': 'Open Lighthouse in interactive mode'


Guess it's time to change --view to --interactive now. Becuase report page now is somewhat interactive.

weiwei-lin · 2017-01-06T04:34:56Z

lighthouse-cli/performance-experiment/server.js

+const lighthouse = require('../../lighthouse-core');
+const perfOnlyConfig = require('../../lighthouse-core/config/perf.json');
+
+const hostedExperiments = [];


Right now the server stores all the results / configs here (in the memory). I'm aware of the potential memory issue. After we have a front end that can compare performance data, we can directly forward the results to that front end page and don't need to store them in memory (at least not in cli).

Edit: Now only stores one instance of rerun results.

weiwei-lin · 2017-01-06T04:36:47Z

lighthouse-cli/performance-experiment/server.js

 module.exports = {
-  serveAndOpenReport
+  hostExperiment


Now this function does more than just opening report. Guess it's time to change this name as well.

weiwei-lin · 2017-01-06T04:38:19Z

lighthouse-core/report/styles/report.css

@@ -151,6 +151,11 @@ body {
  display: none;
 }

+.report-body__icon.rerun-button {
+  background-image: url('data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="24px" height="24px" viewBox="0 0 24 24" fill="#000000"><defs><path id="a" d="M0 0h24v24H0V0z"/></defs><clipPath id="b"><use xlink:href="#a" overflow="visible"/></clipPath><path clip-path="url(#b)" d="M21 10.12h-6.78l2.74-2.82c-2.73-2.7-7.15-2.8-9.88-.1-2.73 2.71-2.73 7.08 0 9.79 2.73 2.71 7.15 2.71 9.88 0C18.32 15.65 19 14.08 19 12.1h2c0 1.98-.88 4.55-2.64 6.29-3.51 3.48-9.21 3.48-12.72 0-3.5-3.47-3.53-9.11-.02-12.58 3.51-3.47 9.14-3.47 12.65 0L21 3v7.12zM12.5 8v4.25l3.5 2.08-.72 1.21L11 13V8h1.5z"/></svg>');


not sure about this icon choice. Haven't found a better one yet.

weiwei-lin · 2017-01-06T05:00:24Z

Ready for review again. PTAL

brendankenny · 2017-01-10T00:48:34Z

lighthouse-core/report/scripts/perf-x.js

@@ -0,0 +1,57 @@
+/**


does it make sense to move this under lighthouse-cli/performance-experiment/ to keep everything in one place?

hmm. What about this approach:

Move perf-x.js to lighthouse-cli/performance-experiment/

In lighthouse-cli/performance-experiment/server.js, override reportGenerator.getReportJS.
So we can keep perf-x.js in lighthouse-cli/performance-experiment/ while do not need to import something outside of core to report-generator.js

paulirish · 2017-01-10T01:40:04Z

lighthouse-cli/performance-experiment/server.js

+    const additionalFlags = JSON.parse(message);
+    const flags = Object.assign({}, hostedExperiments.original.params.flags, additionalFlags);
+
+    lighthouse(url, flags, perfOnlyConfig).then(results => {


let's assume that other clients (like devtools) will also want to rerun lighthouse. in that case, can we split out this bit into a separate file?

Could you explain in a little bit more detail? I don't know how to make it reusable by extension or devtools. Thanks : )

paulirish · 2017-01-10T01:41:33Z

lighthouse-core/report/templates/report-template.html

+          <button class="report-body__icon rerun-button js-rerun-button"></button>
+          <div class="rerun-popup js-rerun-popup" status="idle">
+            <p class="rerun-message">Rerunning Lighthouse</p>
+            <a class="rerun-report-link js-rerun-report-link" href="/?id=rerun">New Report</a>


this UX is a little funky. maybe @samuelli has some ideas.

Let's just go with the button rotating while its running. After its finished, load the new report in the same page. Also add a title to the button.

Optionally, gray out the background of the report while its running.

Thanks for the advice. The button now rotates while rerunning. Also tried graying out the background, but that does not look very nice.

rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed\n

weiwei-lin · 2017-01-10T23:36:43Z

Ready for review again. PTAL.

paulirish

There will be more churn here, but this is a solid start. let's get this in place so we can iterate on the UX.

) * perf-x server now can rerun lighouse on request * support post request to rerun lighthouse with addtional flags * Added a rerun button on report page * moved perf-x.js to perf-x project folder * changed UX design rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed

weiwei-lin and others added 4 commits January 5, 2017 09:29

perf-x server now can rerun lighouse on request

42b6378

fixed a bug caused by rebase and added some inline commits

f209126

support post request to rerun lighthouse with addtional flags

70c3634

rename hostExperiment to serveAndOpenReport

eef6acb

weiwei-lin force-pushed the perf-x-rerun-lighthouse branch from cde16c8 to eef6acb Compare January 4, 2017 22:36

weiwei-lin added 2 commits January 5, 2017 11:58

added a support function in report page for sending rerun request

33cb613

fixed ReportGenerator.getRortJS does not return correct JS file

c7d93c8

fixed coding errors

d65748b

weiwei-lin commented Jan 5, 2017

View reviewed changes

paulirish requested changes Jan 5, 2017

View reviewed changes

weiwei-lin added 2 commits January 5, 2017 19:38

fixed coding sytle and removed rerun GET hanlder

189177b

removed trailing spaces

415cdf7

weiwei-lin commented Jan 5, 2017

View reviewed changes

ebidel reviewed Jan 6, 2017

View reviewed changes

weiwei-lin added 2 commits January 6, 2017 15:25

Added a rerun button on report page

b6765f9

fixed a typo

d25a85f

weiwei-lin commented Jan 6, 2017

View reviewed changes

no longer store multiple rerun results

571724c

weiwei-lin force-pushed the perf-x-rerun-lighthouse branch from 076c925 to 571724c Compare January 8, 2017 23:42

weiwei-lin and others added 2 commits January 9, 2017 13:09

fixed coding style

2b76c26

deleted unnecessary response text

7891430

brendankenny reviewed Jan 10, 2017

View reviewed changes

paulirish reviewed Jan 10, 2017

View reviewed changes

weiwei-lin and others added 4 commits January 10, 2017 13:08

moved perf-x.js to perf-x project folder

54167f7

add missing files

ba6572e

added missing use strict

e8d85d9

changed UX design

0b65e48

rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed\n

weiwei-lin force-pushed the perf-x-rerun-lighthouse branch from 7094039 to 0b65e48 Compare January 10, 2017 06:02

removed uncessary url params

896d40d

Merge branch 'master' into perf-x-rerun-lighthouse

4dd5b6e

paulirish approved these changes Jan 11, 2017

View reviewed changes

paulirish merged commit 8d0d671 into GoogleChrome:master Jan 11, 2017

weiwei-lin deleted the perf-x-rerun-lighthouse branch January 12, 2017 04:31

Allow Perf-X server Rerun lighthouse on POST request. #1393

Allow Perf-X server Rerun lighthouse on POST request. #1393

Conversation

weiwei-lin commented Jan 4, 2017 • edited Loading

paulirish commented Jan 4, 2017

paulirish commented Jan 4, 2017

weiwei-lin commented Jan 4, 2017

weiwei-lin commented Jan 4, 2017

paulirish commented Jan 4, 2017

brendankenny commented Jan 5, 2017

weiwei-lin commented Jan 5, 2017

weiwei-lin commented Jan 5, 2017

weiwei-lin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulirish commented Jan 5, 2017

paulirish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weiwei-lin left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulirish commented Jan 5, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weiwei-lin left a comment • edited Loading

Choose a reason for hiding this comment

weiwei-lin Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

weiwei-lin Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

weiwei-lin Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

weiwei-lin Jan 6, 2017 • edited Loading

Choose a reason for hiding this comment

weiwei-lin commented Jan 6, 2017

Choose a reason for hiding this comment

weiwei-lin Jan 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

weiwei-lin commented Jan 10, 2017

paulirish left a comment

Choose a reason for hiding this comment

weiwei-lin commented Jan 4, 2017 •

edited

Loading

weiwei-lin left a comment •

edited

Loading

weiwei-lin left a comment •

edited

Loading

weiwei-lin Jan 6, 2017 •

edited

Loading

weiwei-lin Jan 6, 2017 •

edited

Loading

weiwei-lin Jan 6, 2017 •

edited

Loading

weiwei-lin Jan 6, 2017 •

edited

Loading

weiwei-lin Jan 10, 2017 •

edited

Loading