Skip to content

[wasm] Update benchmarks with Pixel 3 and Macbook Pro 15 numbers #2573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 17, 2019
Merged

[wasm] Update benchmarks with Pixel 3 and Macbook Pro 15 numbers #2573

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1bb878a
save
dsmilkov Dec 17, 2019
05753b6
save
dsmilkov Dec 17, 2019
40b3189
save
dsmilkov Dec 17, 2019
068fed7
save
dsmilkov Dec 17, 2019
700b639
save
dsmilkov Dec 17, 2019
fd0e813
save
dsmilkov Dec 17, 2019
cf20422
Merge branch 'master' into wasm-update-bench
dsmilkov Dec 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 15 additions & 12 deletions tfjs-backend-wasm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,28 +98,31 @@ CPU backend, and ~5.3-7.7X slower than the WebGL backend.
<img src="./mobilenet-v2-bench.svg">

| MobileNet inference (ms) | WASM | WebGL | CPU |
|-------------------|-------|-------|-------|
| iPhone X | 147.1 | 20.3 | 941.3 |
| iPhone XS | 140 | 18.1 | 426.4 |
| Desktop Linux | 91.5 | 17.1 | 1049 |
| Macbook Pro | | | |
|--------------------------|-------|-------|-------|
| iPhone X | 147.1 | 20.3 | 941.3 |
| iPhone XS | 140 | 18.1 | 426.4 |
| Pixel 3 | 266.2 | 77.3 | 2345.2|
| Desktop Linux | 91.5 | 17.1 | 1049 |
| Macbook Pro | 98.4 | 19.6 | 893.5 |



### Face Detector

Face detector is a lite model with 0.1M params and ~20M multiply-adds. For this model,
the WASM backend is between ~8.2-19.8X faster than the vanilla CPU backend, and
only 1X-1.7X slower than the WebGL backend.
the WASM backend is between ~8.2-19.8X faster than the vanilla CPU backend and
comparable to the WebGL backend (up to ~1.7X faster, or 2X slower, depending on
the device).

<img src="./face-detector-bench.svg">

| Face Detector inference (ms) | WASM | WebGL | CPU |
|---------------|------|-------|-------|
| iPhone X | 23 | 13.5 | 318 |
| iPhone XS | 21.4 | 10.5 | 176.9 |
| Desktop Linux | 12.6 | 12.7 | 249.5 |
| Macbook Pro | | | |
|------------------------------|------|-------|-------|
| iPhone X | 22.4 | 13.5 | 318 |
| iPhone XS | 21.4 | 10.5 | 176.9 |
| Pixel 3 | 40.7 | 31.8 | 535.2 |
| Desktop Linux | 12.6 | 12.7 | 249.5 |
| Macbook Pro 15 2019 | 13.6 | 22.7 | 209.1 |

# FAQ

Expand Down
2 changes: 1 addition & 1 deletion tfjs-backend-wasm/face-detector-bench.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion tfjs-backend-wasm/mobilenet-v2-bench.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion tfjs-core/benchmarks/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ <h2>TensorFlow.js Model Benchmark</h2>
await sleep(200);
await measureAveragePredictTime();
await sleep(200);
if (queryTimerIsEnabled()) {
if (state.backend != 'webgl' || queryTimerIsEnabled()) {
await profileKernelTime();
} else {
showMsg('Skipping kernel times since query timer extension is not ' +
Expand Down