Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Octane-mandreel-latency inappropriately penalizes lazy parsing #29

bhackett1024 opened this issue May 7, 2015 · 0 comments


Copy link

bhackett1024 commented May 7, 2015

AFAIK most web browsers these days use lazy parsing to some degree, where a function is not parsed until it first executes. This is an important feature for improving browser latency and memory usage. The pause time sampling done for mandreel-latency test ends up measuring all the time spent lazy parsing during its first sample. Since there is so much code in the test, parsing alone can take hundreds of milliseconds. For a browser with off thread JIT compilation and incremental GC (like Firefox) the remaining pause times will be tiny, so the benchmark effectively only measures the time spent in that first sample. This time can be easily gamed: by avoiding lazy parsing and doing more work up front (which will hurt browser latency and memory usage) the benchmark score sees a huge boost. v8 seems to be doing this now, as of [1], though the associated bug is not public so I can't really tell what's going on. [2] is a bug I filed against Firefox for this issue, and has more details.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

1 participant