Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Perf-X server Rerun lighthouse on POST request. #1393

Merged
merged 20 commits into from
Jan 11, 2017

Conversation

weiwei-lin
Copy link
Contributor

@weiwei-lin weiwei-lin commented Jan 4, 2017

Make Perf-X server be able to handle lighthouse rerun request. After receiving the request, Perf-X server will run lighthouse with same parameters plus some additional flags provided in the POST request body and then responds with generated results.
This will allow us to rerun lighthouse with some URLblocking patterns within lighthouse report in the future.
Part of Perf-X project #1143

@paulirish
Copy link
Member

Looks like your rebase messed up, as the rename from a previous PR is reverted here. Can you take a look?

@paulirish
Copy link
Member

Do you want to also add a button to the report to rerun? That way we can test out this flow easily.

@weiwei-lin
Copy link
Contributor Author

Do you want to also add a button to the report to rerun? That way we can test out this flow easily.

Yes, I will do that. But probably in another PR. Because I think it's not just a rerun button. We also need to pass additional cli-flags (bolckedUrlPatterns and probably --perf as well). Then we need to design and implement a way for the users to configure blocking patterns as well as a way to surface the rerun results.

If we want something in this PR to ease the test process, I think we can include a function to trigger rerun in report page, which is not visible to users but can be manually invoked in dev-tool console.

@weiwei-lin
Copy link
Contributor Author

Looks like your rebase messed up, as the rename from a previous PR is reverted here. Can you take a look?

Actually, it's intentional. Because I thought the server now does more than just serve and open report. But yeah, rerun is not used anywhere yet. I have renamed the function to serveAndOpenReport again.

@paulirish
Copy link
Member

I think we can include a function to trigger rerun in report page, which is not visible to users but can be manually invoked in dev-tool console.

that sgtm. let's do that.

@brendankenny
Copy link
Member

For a possible followup to this:

One thing this makes me think of is since there's a lighthouse --view flag that runs lighthouse and then launches the performance experiment, which can then re-run lighthouse after this PR, maybe we should drop the --view flag and have something like lighthouse --interactive (or whatever) which just launches the performance experiment, with no starting report but with a big start button that kicks off the initial run.

@weiwei-lin
Copy link
Contributor Author

maybe we should drop the --view flag and have something like lighthouse --interactive (or whatever) which just launches the performance experiment, with no starting report but with a big start button that kicks off the initial run.

Agree. I was thinking about renaming --view to --perf-x, but --interactive seems more intuitive.
I also considered including the initial run in perf-x, even moving perf-x entirely out of cli and becomes a stand-alone branch (like extension). Because in --perf-x, users will interact with lighthouse in browser not command line. Currently, we are still thinking about how to surface results and what is the most ideal workflow for users. After I have a clearer answer, I will implement those changes .

@weiwei-lin
Copy link
Contributor Author

Will come back later to address some coding style issues.

Copy link
Contributor Author

@weiwei-lin weiwei-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ready for review again. PTAL. :)

@@ -100,9 +102,10 @@ function rerunRequestHandler(request, response) {

// Add more to flags without changing the original flags
const flags = Object.assign({}, lhParams.flags, additionalFlags);
lighthouse(lhParams.url, flags, lhParams.configs).then(results => {
lighthouse(lhParams.url, flags, perfOnlyConfig).then(results => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we rerun tests with configurations (e.g. URL blocking patterns), we only care about performance. So use perfOnlyConfig here to get faster result.

@@ -0,0 +1,7 @@
module.exports = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allow us to use /* exported rerunLighthouse */ in perf-x-api.js

@paulirish
Copy link
Member

the more I think about it the more i like the --interactive flag. (happy to rename it later)

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR. (We can still review them simultaneously, even with the dependency)

Copy link
Member

@paulirish paulirish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly good. I am liking this direction...

Right now rerunlighthouse just logs the new results to the console. Don't we want to update the report?

scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/perf-x-api.js')));
}

scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awkward logic here.
probably better to do a `reportContext !== 'devtools' conditional around this last push(), and nuke the early exit on 211

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. That's much better. Fixed.

if (request.method === 'GET') {
if (pathname === '/') {
reportRequestHandler(request, response);
} else if (pathname === '/rerun') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's nuke the GET handler for rerun completely, yah?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, didn't notice that it's not deleted yet. Fixed.

Copy link
Contributor Author

@weiwei-lin weiwei-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now rerunlighthouse just logs the new results to the console. Don't we want to update the report?

In my original plan, this PR will only include back-end support for rerunning lighthouse but not front-end blocking patterns input interface or report update. Because the way to surface results and to configure blocking patterns is not decided yet.

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR

Hmm. I think if we add support for users to input blocking patterns in the report page in this PR, then we can rename --view in this PR. Otherwise, we'd better stick with --view for now because this mode is not interactive for users yet.

if (request.method === 'GET') {
if (pathname === '/') {
reportRequestHandler(request, response);
} else if (pathname === '/rerun') {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, didn't notice that it's not deleted yet. Fixed.

scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/perf-x-api.js')));
}

scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. That's much better. Fixed.

@paulirish
Copy link
Member

Ideally, I'd like it in this PR, but to keep this straightforward for review, I'm okay with splitting it into its own PR

Hmm. I think if we add support for users to input blocking patterns in the report page in this PR, then we can rename --view in this PR. Otherwise, we'd better stick with --view for now because this mode is not interactive for users yet.

Ah.. I was thinking just UI to trigger a re-run, and viewing of those new results.

That wouldn't include configuration of blocked URLs or comparing different runs. Those two are separate and something i know our designer Chris has been thinking about, as well.

}

if (reportContext !== 'devtools') {
scriptList.push(fs.readFileSync(path.join(__dirname, './scripts/lighthouse-report.js')));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was the 'utf8' removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. I didn't add this when I rewrite this method because I thought 'utf8' is default anyway : (
I will add it back in the next commit. Would you mind to tell me why it's necessary though?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad. Fixed in the next update! Thanks for pointing out!

Copy link
Contributor Author

@weiwei-lin weiwei-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a button in the top bar of the report page (only in --interactive mode).
Currently, clicking rerun button will just give user a new link to the new report. This will be changed later in the project.

@@ -83,18 +83,17 @@ const cliFlags = yargs
'port': 'The port to use for the debugging protocol. Use 0 for a random port',
'skip-autolaunch': 'Skip autolaunch of Chrome when already running instance is not found',
'select-chrome': 'Interactively choose version of Chrome to use when multiple installations are found',
'interactive': 'Open Lighthouse in interactive mode'
Copy link
Contributor Author

@weiwei-lin weiwei-lin Jan 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess it's time to change --view to --interactive now. Becuase report page now is somewhat interactive.

const lighthouse = require('../../lighthouse-core');
const perfOnlyConfig = require('../../lighthouse-core/config/perf.json');

const hostedExperiments = [];
Copy link
Contributor Author

@weiwei-lin weiwei-lin Jan 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now the server stores all the results / configs here (in the memory). I'm aware of the potential memory issue. After we have a front end that can compare performance data, we can directly forward the results to that front end page and don't need to store them in memory (at least not in cli).

Edit: Now only stores one instance of rerun results.

module.exports = {
serveAndOpenReport
hostExperiment
Copy link
Contributor Author

@weiwei-lin weiwei-lin Jan 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now this function does more than just opening report. Guess it's time to change this name as well.

@@ -151,6 +151,11 @@ body {
display: none;
}

.report-body__icon.rerun-button {
background-image: url('data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="24px" height="24px" viewBox="0 0 24 24" fill="#000000"><defs><path id="a" d="M0 0h24v24H0V0z"/></defs><clipPath id="b"><use xlink:href="#a" overflow="visible"/></clipPath><path clip-path="url(#b)" d="M21 10.12h-6.78l2.74-2.82c-2.73-2.7-7.15-2.8-9.88-.1-2.73 2.71-2.73 7.08 0 9.79 2.73 2.71 7.15 2.71 9.88 0C18.32 15.65 19 14.08 19 12.1h2c0 1.98-.88 4.55-2.64 6.29-3.51 3.48-9.21 3.48-12.72 0-3.5-3.47-3.53-9.11-.02-12.58 3.51-3.47 9.14-3.47 12.65 0L21 3v7.12zM12.5 8v4.25l3.5 2.08-.72 1.21L11 13V8h1.5z"/></svg>');
Copy link
Contributor Author

@weiwei-lin weiwei-lin Jan 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about this icon choice. Haven't found a better one yet.

@weiwei-lin
Copy link
Contributor Author

Ready for review again. PTAL

@@ -0,0 +1,57 @@
/**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it make sense to move this under lighthouse-cli/performance-experiment/ to keep everything in one place?

Copy link
Contributor Author

@weiwei-lin weiwei-lin Jan 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm. What about this approach:

  1. Move perf-x.js to lighthouse-cli/performance-experiment/
  2. In lighthouse-cli/performance-experiment/server.js, override reportGenerator.getReportJS.
    So we can keep perf-x.js in lighthouse-cli/performance-experiment/ while do not need to import something outside of core to report-generator.js

const additionalFlags = JSON.parse(message);
const flags = Object.assign({}, hostedExperiments.original.params.flags, additionalFlags);

lighthouse(url, flags, perfOnlyConfig).then(results => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's assume that other clients (like devtools) will also want to rerun lighthouse. in that case, can we split out this bit into a separate file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain in a little bit more detail? I don't know how to make it reusable by extension or devtools. Thanks : )

<button class="report-body__icon rerun-button js-rerun-button"></button>
<div class="rerun-popup js-rerun-popup" status="idle">
<p class="rerun-message">Rerunning Lighthouse</p>
<a class="rerun-report-link js-rerun-report-link" href="/?id=rerun">New Report</a>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this UX is a little funky. maybe @samuelli has some ideas.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just go with the button rotating while its running. After its finished, load the new report in the same page. Also add a title to the button.

Optionally, gray out the background of the report while its running.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the advice. The button now rotates while rerunning. Also tried graying out the background, but that does not look very nice.

weiwei-lin and others added 4 commits January 10, 2017 13:08
rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed\n
@weiwei-lin
Copy link
Contributor Author

Ready for review again. PTAL.

Copy link
Member

@paulirish paulirish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be more churn here, but this is a solid start. let's get this in place so we can iterate on the UX.

@paulirish paulirish merged commit 8d0d671 into GoogleChrome:master Jan 11, 2017
weiwei-lin pushed a commit to weiwei-lin/lighthouse that referenced this pull request Jan 12, 2017
)

* perf-x server now can rerun lighouse on request
* support post request to rerun lighthouse with addtional flags
* Added a rerun button on report page
* moved perf-x.js to perf-x project folder
* changed UX design
  rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed
@weiwei-lin weiwei-lin deleted the perf-x-rerun-lighthouse branch January 12, 2017 04:31
andrewrota pushed a commit to andrewrota/lighthouse that referenced this pull request Jan 13, 2017
)

* perf-x server now can rerun lighouse on request
* support post request to rerun lighthouse with addtional flags
* Added a rerun button on report page
* moved perf-x.js to perf-x project folder
* changed UX design
  rerun button now reruns lighthouse with the same config and refresh the page after rerun is completed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants