diff --git a/README.md b/README.md index 22a43ee..330360c 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ $ apt install time hyperfine ``` You have also to install [Google Chrome](https://www.google.com/chrome/) and -Lightpanda browser, but the code is not publicly available yet. +[Lightpanda browser](https://github.com/lightpanda-io/browser/releases/tag/nightly). ### Demo web page @@ -65,28 +65,30 @@ This bench is a very basic test to compare the two software. We start the browser and request the fake web page once with full JS execution. The final DOMTree is rendered in stdout. -We use Google Chrome version 122.0.6261.94. +We use Google Chrome version 130.0.6723.58. ```console $ google-chrome --version -Google Chrome 122.0.6261.94 +Google Chrome 130.0.6723.58 ``` +And Lightpanda commit [1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2](https://github.com/lightpanda-io/browser/commit/1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2). + ### Execution time ```console -$ hyperfine --warmup 3 --runs 20 --shell=none "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/" "./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/" -Benchmark 1: google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/ - Time (mean ± σ): 556.7 ms ± 10.2 ms [User: 360.8 ms, System: 170.6 ms] - Range (min … max): 538.2 ms … 571.6 ms 20 runs +$ hyperfine --warmup 3 --runs 20 --shell=none "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/" "./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/" +Benchmark 1: google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/ + Time (mean ± σ): 618.9 ms ± 12.8 ms [User: 363.0 ms, System: 162.0 ms] + Range (min … max): 598.2 ms … 659.2 ms 20 runs -Benchmark 2: ./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/ - Time (mean ± σ): 8.6 ms ± 0.2 ms [User: 5.0 ms, System: 3.2 ms] - Range (min … max): 8.3 ms … 9.0 ms 20 runs +Benchmark 2: ./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/ + Time (mean ± σ): 9.9 ms ± 0.3 ms [User: 5.8 ms, System: 3.7 ms] + Range (min … max): 9.3 ms … 10.4 ms 20 runs Summary - './browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/' ran - 64.48 ± 1.74 times faster than 'google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/' + './lightpanda --dump http://127.0.0.1:1234/campfire-commerce/' ran + 62.58 ± 2.32 times faster than 'google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:124/campfire-commerce/' ``` ![aws.m5 hyperfine](./img/aws_m5_hyperfine.png) @@ -96,47 +98,48 @@ Summary ```console $ /usr/bin/time -v google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/ Command being timed: "google-chrome --user-data-dir=/tmp/bench_chrome --headless=new --dump-dom http://127.0.0.1:1234/campfire-commerce/" - User time (seconds): 0.38 - System time (seconds): 0.14 - Percent of CPU this job got: 96% - Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.55 + User time (seconds): 0.34 + System time (seconds): 0.19 + Percent of CPU this job got: 94% + Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.57 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 - Maximum resident set size (kbytes): 169924 + Maximum resident set size (kbytes): 174096 Average resident set size (kbytes): 0 - Major (requiring I/O) page faults: 5 - Minor (reclaiming a frame) page faults: 20535 - Voluntary context switches: 2664 - Involuntary context switches: 1655 + Major (requiring I/O) page faults: 17 + Minor (reclaiming a frame) page faults: 20609 + Voluntary context switches: 2563 + Involuntary context switches: 1618 Swaps: 0 - File system inputs: 0 - File system outputs: 1624 + File system inputs: 1048 + File system outputs: 4576 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 + ``` ```console -$ /usr/bin/time -v ./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/ - Command being timed: "./browsercore-get --dump http://127.0.0.1:1234/campfire-commerce/" +$ /usr/bin/time -v ./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/ + Command being timed: "./lightpanda --dump http://127.0.0.1:1234/campfire-commerce/" User time (seconds): 0.00 System time (seconds): 0.00 - Percent of CPU this job got: 100% + Percent of CPU this job got: 92% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.01 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 - Maximum resident set size (kbytes): 14348 + Maximum resident set size (kbytes): 21276 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 - Minor (reclaiming a frame) page faults: 751 + Minor (reclaiming a frame) page faults: 925 Voluntary context switches: 6 - Involuntary context switches: 70 + Involuntary context switches: 11 Swaps: 0 File system inputs: 0 File system outputs: 0 @@ -147,60 +150,149 @@ $ /usr/bin/time -v ./browsercore-get --dump http://127.0.0.1:1234/campfire-comme Exit status: 0 ``` -## Multiple requests using Playwright +## Multiple requests using Puppeteer We compare now multiple page loads and js evaluations using -[Playwright](https://playwright.dev), which connects to the browser using CDP (Chrome Debug Protocol). +[Puppeteer](https://https://pptr.dev/), which connects to the browser using CDP +(Chrome Debug Protocol). ### Dependencies To run the benchmark, you need ti install [nodejs](https://nodejs.org/en/download). Once `nodejs` is installed, please run a `npm install` to install nodejs -dependencies, mainly Playwright. +dependencies, mainly Puppeteer. You have also to install [Google Chrome](https://www.google.com/chrome/) and Lightpanda browser, but the code is not publicly available yet. ### Running the benchmark -The `playwright/cdp.js` benchmark accepts multiple env vars to be configured. +The `puppeteer/cdp.js` benchmark accepts multiple env vars to be configured. * `BROWSER_ADDRESS` is the address of the running browser listening the CDP protocol, by default `http://127.0.0.1:9222`. * `BASE_URL` is the base url of the running web reser to request, by default `http://127.0.0.1:1234`. * `RUNS` is the number of pages loaded by the benchmark, default is `100`. -`npm run bench-playwright-cdp` starts a playwright process +`npm run bench-puppeteer-cdp` starts a Puppeteer process instance and load the page to extract data 100 times. ```console -$ npm run bench-playwright-cdp +$ npm run bench-puppeteer-cdp ``` ### Results **Google Chrome** -We use Google Chrome version 123.0.6312.105. +We use Google Chrome version 130.0.6723.58. You have to start the browser first. ```console -$ google-chrome --headless=new --disable-gpu --remote-debugging-port=9222 +$ /usr/bin/time -v google-chrome --headless=new --remote-debugging-port=9222 ``` Then you can run the benchmark. ```console -$ npm run bench-playwright-cdp +$ npm run bench-puppeteer-cdp -> demo@1.0.0 bench-playwright-cdp -> node playwright/cdp.js +> demo@1.0.0 bench-puppeteer-cdp +> node puppeteer/cdp.js ................................................................................ .................... total runs 100 -total duration (ms) 18792 -avg run duration (ms) 184 -min run duration (ms) 168 -max run duration (ms) 323 +total duration (ms) 25218 +avg run duration (ms) 248 +min run duration (ms) 216 +max run duration (ms) 451 +``` + +![aws.m5 Puppeteer with Google Chrome](./img/aws_m5_puppeteer_chrome.png) + +```console + Command being timed: "google-chrome --headless=new --remote-debugging-port=9222" + User time (seconds): 16.19 + System time (seconds): 6.59 + Percent of CPU this job got: 41% + Elapsed (wall clock) time (h:mm:ss or m:ss): 0:54.51 + Average shared text size (kbytes): 0 + Average unshared data size (kbytes): 0 + Average stack size (kbytes): 0 + Average total size (kbytes): 0 + Maximum resident set size (kbytes): 207828 + Average resident set size (kbytes): 0 + Major (requiring I/O) page faults: 138 + Minor (reclaiming a frame) page faults: 265165 + Voluntary context switches: 143625 + Involuntary context switches: 82912 + Swaps: 0 + File system inputs: 35680 + File system outputs: 169688 + Socket messages sent: 0 + Socket messages received: 0 + Signals delivered: 0 + Page size (bytes): 4096 + Exit status: 0 +``` + +**Lightpanda browser** + +We use Lightpanda commit [1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2](https://github.com/lightpanda-io/browser/commit/1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2). + +You have to start the Lightpanda Gateway. +```console +./gateway ``` -![aws.m5 Playwright with Google Chrome](./img/aws_m5_playwright_chrome.png) +And Lightpanda browser itself. +```console +/usr/bin/time -v ./lightpanda +``` + +Then you can run the benchmark. +```console +$ npm run bench-puppeteer-cdp + +> demo@1.0.0 bench-puppeteer-cdp +> node puppeteer/cdp.js + +................................................................................ +.................... +total runs 100 +total duration (ms) 3456 +avg run duration (ms) 32 +min run duration (ms) 23 +max run duration (ms) 105 +``` + +![aws.m5 Puppeteer with Lightpanda browser](./img/aws_m5_puppeteer_lightpanda.png) + +```console + Command being timed: "./lightpanda" + User time (seconds): 41.78 + System time (seconds): 14.15 + Percent of CPU this job got: 99% + Elapsed (wall clock) time (h:mm:ss or m:ss): 0:56.14 + Average shared text size (kbytes): 0 + Average unshared data size (kbytes): 0 + Average stack size (kbytes): 0 + Average total size (kbytes): 0 + Maximum resident set size (kbytes): 100940 + Average resident set size (kbytes): 0 + Major (requiring I/O) page faults: 0 + Minor (reclaiming a frame) page faults: 29326 + Voluntary context switches: 235 + Involuntary context switches: 4099 + Swaps: 0 + File system inputs: 0 + File system outputs: 0 + Socket messages sent: 0 + Socket messages received: 0 + Signals delivered: 0 + Page size (bytes): 4096 + Exit status: 0 +``` + +--- + +Console images generated with [Carbon](https://carbon.now.sh). diff --git a/img/aws_m5_hyperfine.png b/img/aws_m5_hyperfine.png index 423e0c5..8eb7b4c 100644 Binary files a/img/aws_m5_hyperfine.png and b/img/aws_m5_hyperfine.png differ diff --git a/img/aws_m5_playwright_chrome.png b/img/aws_m5_playwright_chrome.png deleted file mode 100644 index 217dda0..0000000 Binary files a/img/aws_m5_playwright_chrome.png and /dev/null differ diff --git a/img/aws_m5_puppeteer_chrome.png b/img/aws_m5_puppeteer_chrome.png new file mode 100644 index 0000000..3f896a4 Binary files /dev/null and b/img/aws_m5_puppeteer_chrome.png differ diff --git a/img/aws_m5_puppeteer_lightpanda.png b/img/aws_m5_puppeteer_lightpanda.png new file mode 100644 index 0000000..ba44e51 Binary files /dev/null and b/img/aws_m5_puppeteer_lightpanda.png differ