Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update the benchmark script (upgrade lighthouse to v3) #13994

Merged
merged 1 commit into from
Nov 5, 2018
Merged

Update the benchmark script (upgrade lighthouse to v3) #13994

merged 1 commit into from
Nov 5, 2018

Conversation

keyz
Copy link
Contributor

@keyz keyz commented Oct 27, 2018

I wanted to run some benchmark tests but I couldn't get the current script to work (lighthouse threw an obscure error of Cannot read property 'args' of undefined). I debugged it for a bit but didn't have much luck.

So I tried to upgrade lighthouse to v3 and it seems to work fine. Here's a full yarn start run -- does the output seem reasonable?

bench $ yarn start

Using globally installed version of Yarn
yarn run v1.12.1
warning package.json: No license field
$ node runner.js
Comparing Local (Current Branch) to Remote (Merge Base)
Running benchmarks for Local (Current Branch)
- Building React bundles...
(node:43790) ExperimentalWarning: The http2 module is an experimental API.
- Building benchmark "hacker-news"...
- Running benchmark "hacker-news"...
- Building benchmark "pe-class-components"...
- Running benchmark "pe-class-components"...
- Building benchmark "pe-functional-components"...
- Running benchmark "pe-functional-components"...
- Building benchmark "pe-no-components"...
- Running benchmark "pe-no-components"...
Running benchmarks for Remote (Merge Base)
- Building React bundles...
- Merge base commit 275e76e83bc2be5dc0be9185ff747f383969289a
- Building benchmark "hacker-news"...
- Running benchmark "hacker-news"...
- Building benchmark "pe-class-components"...
- Running benchmark "pe-class-components"...
- Building benchmark "pe-functional-components"...
- Running benchmark "pe-functional-components"...
- Building benchmark "pe-no-components"...
- Running benchmark "pe-no-components"...
┌──────────────────────────┬─────────────────────┬────────────────────────┬──────────────────┐
│                          │ Remote (Merge Base) │ Local (Current Branch) │                  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ hacker-news              │ Time                │ Time                   │ Diff             │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React               │ 250.06 ms +- 2.07   │ 250.33 ms +- 0.82      │ +0.1 % +- 0.9 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React+ReactDOM      │ 825.97 ms +- 3.53   │ 822.59 ms +- 9.08      │ -0.4 % +- 1.2 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load ReactDOM            │ 574.99 ms +- 3.66   │ 571.36 ms +- 9.50      │ -0.6 % +- 1.8 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Initial Render           │ 139.76 ms +- 10.28  │ 136.48 ms +- 13.98     │ -2.5 % +- 12.3 % │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Update Render            │ 57.16 ms +- 4.23    │ 59.1 ms +- 14.45       │ +3.2 % +- 26.4 % │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ First Meaningful Paint   │ 2628.86 ms +- 22.68 │ 2632.64 ms +- 5231.89  │ +0.1 % +- 199 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ pe-class-components      │ Time                │ Time                   │ Diff             │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React               │ 766.38 ms +- 6.55   │ 767.13 ms +- 5.63      │ +0.1 % +- 1.1 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React+ReactDOM      │ 1732.78 ms +- 3.92  │ 1737.1 ms +- 8.01      │ +0.2 % +- 0.5 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load ReactDOM            │ 967.24 ms +- 4.76   │ 972.25 ms +- 7.21      │ +0.5 % +- 0.9 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Initial Render           │ 447.1 ms +- 43.69   │ 446.19 ms +- 12.21     │ -0.5 % +- 10.1 % │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Update Render            │ 69.16 ms +- 11.60   │ 68.99 ms +- 2.72       │ -1 % +- 17.2 %   │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ First Meaningful Paint   │ 3311.59 ms +- 60.99 │ 3316.27 ms +- 20.41    │ +0.1 % +- 1.9 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ pe-functional-components │ Time                │ Time                   │ Diff             │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React               │ 767.43 ms +- 7.42   │ 766.45 ms +- 6.94      │ -0.1 % +- 1.3 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React+ReactDOM      │ 1732.48 ms +- 3.04  │ 1732.33 ms +- 3.80     │ 0 % +- 0.3 %     │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load ReactDOM            │ 964.39 ms +- 4.87   │ 966.78 ms +- 5.51      │ +0.2 % +- 0.8 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Initial Render           │ 429.35 ms +- 17.12  │ 400.2 ms +- 39.30      │ -6.8 % +- 9.9 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Update Render            │ 56.89 ms +- 4.02    │ 53.32 ms +- 2.60       │ -6.4 % +- 8 %    │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ First Meaningful Paint   │ 3217.38 ms +- 27.47 │ 3161.2 ms +- 43.06     │ -1.7 % +- 1.6 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ pe-no-components         │ Time                │ Time                   │ Diff             │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React               │ 767.04 ms +- 4.43   │ 764.01 ms +- 5.82      │ -0.4 % +- 1 %    │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load React+ReactDOM      │ 1729.96 ms +- 4.54  │ 1735.27 ms +- 4.83     │ +0.3 % +- 0.4 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Load ReactDOM            │ 966.66 ms +- 3.77   │ 970.69 ms +- 4.43      │ +0.4 % +- 0.6 %  │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Initial Render           │ 367.04 ms +- 9.68   │ 359.34 ms +- 38.53     │ -2.1 % +- 10.8 % │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ Update Render            │ 36.84 ms +- 2.54    │ 36.7 ms +- 1.50        │ -0.5 % +- 8 %    │
├──────────────────────────┼─────────────────────┼────────────────────────┼──────────────────┤
│ First Meaningful Paint   │ 3001.86 ms +- 18.74 │ 3019.33 ms +- 44.04    │ +0.6 % +- 1.6 %  │
└──────────────────────────┴─────────────────────┴────────────────────────┴──────────────────┘
✨  Done in 1231.85s.

keyz
keyz commented Oct 27, 2018
View reviewed changes
scripts/bench/benchmark.js
const results = await Lighthouse(
`http://localhost:8080/${benchmark}/`,
{
output: 'json',
disableCpuThrottling: false,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config enables throttling by default

@sizebot
Copy link

sizebot commented Oct 27, 2018

Details of bundled changes.

Comparing: 275e76e...07f0db4

scheduler

File Filesize Diff Gzip Diff Prev Size Current Size Prev Gzip Current Gzip ENV
scheduler.development.js n/a n/a 0 B 19.17 KB 0 B 5.74 KB UMD_DEV
scheduler.production.min.js n/a n/a 0 B 3.16 KB 0 B 1.53 KB UMD_PROD

Generated by 🚫 dangerJS

@gaearon
Copy link
Collaborator

gaearon commented Nov 5, 2018

i trust you

@gaearon gaearon merged commit ce90ffd into facebook:master Nov 5, 2018
jetoneza pushed a commit to jetoneza/react that referenced this pull request Jan 23, 2019
@keyz @jetoneza
update the benchmark script (facebook#13994)
932777f
n8schloss pushed a commit to n8schloss/react that referenced this pull request Jan 31, 2019
@keyz @n8schloss
update the benchmark script (facebook#13994)
1075736
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sophiebits sophiebits Awaiting requested review from sophiebits

@gaearon gaearon Awaiting requested review from gaearon

@trueadm trueadm Awaiting requested review from trueadm

Assignees
No one assigned
Labels
CLA Signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@keyz @sizebot @gaearon @facebook-github-bot