Benchmark prototype

[mzgoddard]

Resolves

Related #554

Proposed Changes

• Add benchmark suite
• The individual benchmark at playground/index.html displays a link to the suite page after running
• Control individual benchmark warm up and recording time as url components in the hash
• Serialize individual benchmark data into its URL
• A share link is displayed after the run with that encoded URL
• Suite reports can be opened in the running frame for easy viewing
• Save Reports as a JSON download and Compare Reports to load that JSON in a future run to compare changes made to the VM

Reason for Changes

The individual benchmark page was a stepping stone to this to provide a more representative view of performance across varying Scratch projects. The suite executes multiple projects in order with varying warm up and recording times.

Two high level performance statistics are immediately presented, steps per second and blocks per second. The steps per second are the number of times any thread is run until it yields or completes during one second. The blocks per second are the number of VM script blocks executed in a second. VM bound projects regularly show a similar number of blocks per second with steps per second being a representation of the complexity of the threads. Lower steps per second with a VM bound number of blocks per seconds relates to a higher number of blocks per step. Blocks per second lower than the VM bound value likely indicates a currently expensive block function being executed like pen_stamp. Lower blocks per second may also indicate a simple project or that the project is render bound.

More details can be viewed by click the steps per second and blocks per second values. Either opening in the benchmark frame or in a new tab.

All of the benchmark suite's reports can be saved together with the Save Reports link. This creates a JSON download to the local system that can be loaded with Compare Reports in a future run, during or after the run, to see side by side steps and blocks per second values. Or the second values can be clicked to see the other recorded performance values. This is great for recording a run and comparing later against the VM with an optimization implemented. There will be some margin for error between any two runs depending on other work the testing system may be doing.

Example benchmark run:

Example saving reports and re-running against the loaded report:

Example navigating detailed reports between two executions:

[boazsender]

Pulled and tested, LGTM.

Also, love the bench and perma link links. Nice work!

[thisandagain]

This looks great at first glance running locally @mzgoddard. 👍

[thisandagain]

/cc @cwillisf @kchadha @paulkaplan

[rschamp]

This is an amazing amount of work that looks and runs great! Thank you @mzgoddard!