Clone the repo and install the dependecies:
git clone https://github.com/etsy/DeviceTiming.git cd DeviceTiming npm install
Setup the test
So first, backup your code:
cp /path/to/your/js /path/to/your/js-unmodified
./devicetiming server /path/to/your/js
This will first recursively instrument all the code in the path provided.
Then it will start listening at port 8537 for beacons from the client code.
You can provide a differnt port with
devicetiming -h for more.
When you're done you can restore your backed up copy:
cp /path/to/your/js-unmodified /path/to/your/js
Running The Tests
To run the tests, visit your development server hosting the instrumented code from each device you wish to test. We use Adobe Edge Inspect in our device lab to perform these tests across all the devices at once. Once you've loaded a page with instrumented code in the browser, it will continuously reload the page to send additional runs to the server. It will continue until you stop it by manually closing or leaving that page.
The instrumentation assumes that the DeviceTiming server can be found at the same hostname as the locations you hit in the browser, if you need to use a different hostname for those beacons, use the
--hostname argument with
Wrap that string in an
Add the timing code -- first we mark time before the eval call, then from there to the one at the beginning of the eval string we call the "parse" time. From the inside eval call timer, until a timer just after eval runs captures the "exec" time. This is functionally equivalent to the original, unmodified file executing in the browser.
What's the deal with intermittant 0ms parse times?
It seems that some combinations of devices / browsers have an optimization that caches the bytecode generated by the parsing step and then skips it whenever possible. Additionally, it's possible that browsers that don't exhibit this behavior in our tests still have this optimization, but only for scripts that are parsed through a
<script> tag and not strings run through eval. In other words, a cache keyed by URI rather than content hash.
Tell me more
See slides and resources from Unpacking the Black Box: Benchmarking JS Parsing and Execution on Mobile Devices, a talk Daniel Espeset gave about this work at Velocity NYC 2014.
Questions and comments welcome
Open an issue, submit a pull request or tweet at @danielespeset on twitter.
Made by Performance and Frontend Infrastructure at Etsy.