WebPageTest 3.0

@pmeenan pmeenan released this Dec 16, 2016 · 171 commits to master since this release

Configuration Instructions

Changes from WebPageTest 2.19:

With a year of commits there are a LOT of bug fixes, support for updated browsers, etc. This will just cover the main feature changes.

  • Video, Chrome Trace and tcpdump processing has been pused to the agents and is no longer done on the server (for better scaling).

Web UI

  • Added support for PHP 7
  • The UI now supports reporting the output of multiple steps for multi-step scripts (huge thanks the the massive amount of work that sburnicki put in on ot).
  • JS Parse and Execution time is displayed in the waterfall on a per-request basis when Chrome timelines are captured.
  • The times when the main thread is blocked/interactive is displayed below the waterfall when Chrome timelines are captured.
  • When tcpdump is captured with a test the bandwidth display in the waterfall will be calculated from the packet capture and will also display spurious retransmits (duplicated data).
  • There are several new settings but two in particular that you should consider changing on your existing install if you are doing an update:
    • Switch the waterfalls to the new scheme where they are colored by mime type:
      • mime_waterfalls=1
    • Enable auto-update of the test agents as new ones are rolled out to the public instance:

Test Agent

  • URLBlast support (Testing IE <= 8) has discontinued and is no longer included in the releases. Existing installs will continue to work but there will be no more updates.
  • Windows Safari support has been discontinued. It was last updated at Safari 5 which is ancient and unused at this point.

JS (Mobile testing)

  • The mobile agent is now stable and should work reliably (more reliably for Android than iOS testing though)
  • Limited support for testing several browsers as "black-box" browsers (visual metrics but no waterfalls)
    • UC Mini
    • Opera Mini
    • Opera
    • Samsung Browser

wptdriver (Desktop Testing:

  • Added support for testing Microsoft Edge.
  • Added Windows 10 support and now requires Windows 7 at a minimum.
  • New traffic-shaping driver (winshaper) is now used for Windows 8.1 or later.
  • Added support for testing 64-bit versions of Chrome and Firefox.
  • Added support for decoding TLS streams for Chrome so response bodies and optimization checks now work.
  • Added support for decoding and reporting HTTP/2
  • Added support for decoding Brotli compression
  • Improved the image optimization check to look for bloat in PNGs and any non-animated GIFs that should be PNGs.

Downloads

WebPageTest 2.19

@pmeenan pmeenan released this Jan 22, 2016 · 1272 commits to master since this release

Configuration Instructions

Changes from WebPageTest 2.18:

Web Server

  • Added support for emulating arbitrary mobile devices (Chrome)
  • Updated embedded Chrome Dev Tools and Tracing UI
  • Added support for reporting user timing measures(API only)
  • Fixed target TTFB logic/grading
  • Added reporting of the agent Windows version in getTesters.php to prepare for Chrome deprecating < Windows 7 (6.1 actual version)
  • Updated EC2 AMI ID's to new/refreshed 2016 versions of the AMI's
  • Added support for per-connectivity or per-location test timeout settings

Test Agent

js (Mobile testing)

  • Improved iOS testing with 60fps video capture, tcpdump, feature compatible with Android testing
  • Changed test end conditions to match desktop (at onload or after 2 seconds of inactivity after onload)
  • Added support for testing Chrome on Android Marshmallow
  • Improved reliability
  • Updated to support Node JS 4+
  • Error test if Chrome crashed during testing
  • Added support for collecting user timing measures
  • Added support for the setDns script command

wptdriver:

  • Updated to support latest Chrome and Firefox releases
  • Added HTTP/2 decode support for Firefox and IE 11 (including server push)
  • Support talking HTTPS to WebPageTest server (including client certificates if configured)
  • Reduced overhead when testing Chrome
  • Added support for emulating arbitrary mobile devices (Chrome)
  • Fixed mobile emulation viewport size to match actual device
  • Block automatic browser downloads
  • Added support for collecting user timing measures
  • Added reporting of the current Windows version to prepare for Chrome deprecating < Windows 7 (6.1 actual version)

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

Downloads

WebPageTest 2.18

@pmeenan pmeenan released this Jul 20, 2015 · 1574 commits to master since this release

Configuration Instructions

Changes from WebPageTest 2.17:

Web Server

  • Fixed a critical remote code execution vulnerability (linux servers)

Test Agent

js (Mobile testing)

  • Added support for processing video captures on the node JS agent system instead of the server
  • Added support for custom metrics (JS that executes after the test completes)
  • Added reporting of user timing marks
  • Added reporting of navigation timing
  • Added support for the "ignore certificate errors" option

wptdriver:

  • Added support for decoding HTTP/2 for Firefox and IE (including server push)
  • Added support for capturing the SSL Key log file when running a tcpdump capture (Chrome and Firefox only)
  • Switched to mozjpeg for the jpeg compression check and for saved images (screen shots and video frames)
  • Fixed support for the setEventName script command

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

Downloads

WebPageTest 2.17

@pmeenan pmeenan released this Jun 24, 2015 · 1641 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.16:

Web Server

  • Added custom metrics to the result page
  • Added (optional) rate limiting for video rendering based on server load
  • Added support for connectivity-specific test timeout limits
  • Added support for newer versions of ffmpeg
  • Improved performance of mobile video processing
  • Improved EC2 auto-scaling

Test Agent

js (Mobile testing)

  • Added support for ignoring certificate errors
  • SPOF, Content blocking and setDNS support added
  • Added support for Chrome Dev Channel on Android
  • Added support for running custom chrome_shell builds on Android
  • Allow android tests to run for longer than 3 minutes with truncated video
  • Fixed support for Android Lollipop
  • Fixed packet loss support in traffic shaping
  • Fixed Chrome timeline capture and reporting (Chrome 42+)
  • Improved mobile agent reliability (particularly on Windows)

wptdriver:

  • Fixed support for setUserAgent for Chrome HTTPS requests
  • Added support for eventName script command
  • Added support for using HTTP basic auth over HTTPS between the agents and server
  • Improved chrome mobile emulation (disable flash)
  • Fixed packet loss support in traffic shaping
  • Improvements to Chrome HTTPS waterfall processing
  • Fixed Chrome timeline capture and reporting (Chrome 42+)
  • Added support for calculating speed index without video capture
  • Disabled the Firefox "browser upgraded" startup page

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

Downloads

WebPageTest 2.16

@pmeenan pmeenan released this Dec 18, 2014 · 1912 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.15:

There are quite a few big additions/improvements in this release:

  • Added support for an easy-deploy server running on EC2.
  • Mobile agent now (correctly) closes the browser between first and repeat view to match the desktop behavior.
  • Moved to server-side video rendering.
  • StatsD integration.

Web Server

  • Moved to server-side video rendering.
    • Significantly improved video layouts.
    • Support for 60fps mobile videos.
    • Supports huge numbers of tests (up to 150 in the same video was tested)
  • Eliminated dependency on the specific version of ffmpeg for processing mobile video (now works with 1.x and 2.x).
  • Several additions to the install check:
    • Added the version of the running release.
    • Added a check for freetype (needed for server-side video rendering).
  • Added "error rate" reporting to the getTesters.php view.
  • Significantly improved the performance of the test history view if running on Linux.
  • Removed the Progressive JPEG recommendation (currently controversial).
  • Added support for an easy-deploy server running on EC2.
    • Added support for automatically launching EC2 test agents.
  • Added support for automatically pulling the latest agent updates from the public WebPageTest instance.
  • Improved performance of the post-processing of mobile tests.
  • Added support for sharded WebPageTest servers (server ID as part of the test ID)
  • Added support for posting results to StatsD.
  • Integrated with Google Safe Browsing API for filtering of test requests (optional).
  • Added a hybrid work queue where beanstalkd can be used for API tests but the legacy mode (with test position) can be used for interactive tests.

Test Agent

js (Mobile testing)

  • Added support for reporting the DOM Content Loaded time
  • Agent now (correctly) closes the browser between first and repeat view to match the desktop behavior.

wptdriver:

  • Fixes to processing of Chrome requests (HTTPS in particular)
  • Fixed the Chrome mobile emulation
  • Fixed video capture dor latest Chrome releases

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Improved IE testing thruput (eliminated 5 second delay when running tests).

Downloads

WebPagetest 2.15

@pmeenan pmeenan released this Jul 22, 2014 · 2123 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.14:

There are quite a few big additions/improvements in this release:

  • The addition of custom metrics.
  • Logs integration for test results.
  • Significant improvements to the new mobile agents.
  • The mobitest android agents are no longer supported (security concerns with the results processing).
  • Added support for capturing and viewing Chrome traces.

Web Server

  • Added support for writing test results to a log file (see logTestResults and logTestRequests in settings.ini.sample)
  • Significant improvements to the processing of mobile video captures.
  • Fixed the display of console log messages from mobile tests.
  • Fixed the waterfall translucency slider support in filmstrip view for Chrome 30+.
  • Improved test reliability (reduced instances of corrupt results).
  • Added sample benchmark configurations.
  • Fixed timeline processing for Chrome 36+.
  • Added support for viewing Chrome traces.
  • Added installer checks for php's memory limit, for tmp being on tmpfs and for being able to write to the dat directory.

Test Agent

js (Mobile testing)

  • Significant improvements to video and waterfall processing (done on the server but mobile-specific).
  • Changed the logic to close the browser between first and repeat view to match desktop testing.
  • Added support for running against an emulator (though it is very much not recommended for anything but dev).

wptdriver:

  • Added support for capturing Chrome traces.
  • Fixed support for multi-step tests.
  • Fixed the reporting of HTTPS 204 responses.
  • Added support for overrideHost, setHeader and addHeader commands for IE HTTPS testing.
  • Improved the support for the exec and execAndWait commands in Chrome so they no longer run in a sandbox and can access the page variables.
  • Added support for capturing the page title and titleTime for Chrome (already worked for IE and Firefox).

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Added support for capturing custom metrics.
  • Improved the adult site detection logic.
  • Changed 401 response codes (auth challenge) to be treated like redirects.

Downloads

WebPagetest 2.14

@pmeenan pmeenan released this Apr 9, 2014 · 2342 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.13:

The biggest news for this release is the first stable release of the new mobile Android testing agent which includes high-speed video capture and full timeline reporting.

Web Server

  • Significant improvements to test management, eliminating a lot of cases of "missing data".
  • Added a timeline CPU utilization graph below the waterfall indicating JS execution, layout, etc.
  • Fonts added as a separate mime type for content views (waterfall and breakdown).
  • Added support for displaying 60fps video frames in the filmstrip UI (from mobile).
  • Added content bodies to HAR export if bodies were captured at the time of the test.
  • Added support for Appurify Kindle testing.
  • Added support for importing a dev tools trace as a stand-alone test (for integration with other tooling).
  • Added error messages from test agents to the test results UI for better diagnosing of failed tests.
  • Added support for editing test labels.
  • Improved thumbnail tracking in the filmstrip view.

Test Agent

js (Mobile testing)

  • Initial release. Setup instructions are here.

wptdriver:

  • Fixed IE 11 support.
  • Updated Firefox and Chrome support to support the latest releases.
  • Fixed traceroute test support.
  • Significant changes to how code injection is done for significantly improved reliability (eliminating cases of tests with missing data).
  • Improved reporting of HTTPS resources when testing with Chrome.
  • Switched ignoring of cert errors to only be enabled if explicitly requested with the test.

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Video capture (when enabled) runs continuously instead of only when paints are detected which enables better GPU animation and video playback capture.
  • Added support for pulling tests from multiple locations. i.e. "location=wptdriver,IE_10".

Downloads

WebPagetest 2.13

@pmeenan pmeenan released this Nov 19, 2013 · 2664 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.12:

First, a big thank you to the community, there were several patches from external contributors in this release.

Web Server

  • Integration with Appurify for mobile testing as a service (including traffic shaping, native browsers, full timeline capture and high-framerate video).
  • Integration with TSViewDB for time-series data storage and reporting.
  • Added a multi-waterfall view to the filmstrip comparisons to present overlaid waterfalls.
  • Added a main-thread breakdown view for Chrome timelines to show where the browser spent it's time.
  • Added an "Effective Bps" metric that calculates the bandwidth used from the first byte to the end of the load to show how efficiently a site is filling the pipe.
  • Fixed the EC2 auto-scaling logic to account for pending spot instance requests.
  • Added support for overriding the default connectivity on a per-location basis.
  • Added a link to the request details popup to open the response body in a new window (if bodies were captured).
  • Significantly improved the error messaging when tests fail (will now indicate why it failed to run the test).

Test Agent

wptdriver:

  • Significant fixes to IE testing (some of those "how did it ever work" kind of fixes).
  • Updates to support newer Chrome and Firefox releases.
  • Added support for arbitrary command-line options for Chrome.
  • Improvements to the emulated mobile testing.

URLBlast (legacy IE testing):

  • Removed support for automatically rebooting.
  • Significantly simplified the agent code and stripped out all of the legacy non-WebPagetest code.
  • Fixes to make agent more stable (keep running without intervention).

Improvements common to both agents:

  • Added reporting of msFirstPaint from window.performance (IE only).
  • Added a CPU Time metric to report how much CPU-busy time was consumed while loading the page.
  • Added reporting of agent CPU utilization (for tracking down under-provisioned agents).
  • Improved IE's cache clearing to be more reliable.

Downloads

WebPagetest 2.12

@pmeenan pmeenan released this Jul 11, 2013 · 2934 commits to master since this release

Configuration Instructions

Changes from WebPagetest 2.11:

Web Server

  • Added support for reporting user timing marks.
  • Added an optimization check to see if JPEGs were progressive or not.
  • Added detailed JPEG info to the page images UI.
  • Added support for turning arbitrary videos into comparison results pages.
  • Updated the embedded Chrome timeline viewer so new features like the frames UI are available.

Test Agent

wptdriver:

  • Added support for reporting DOM Element count and if the viewport is fixed.
  • Significant improvements to the IE support (navigation timing, crash fixes).
  • Console logs from Chrome are now only available when timeline capture is enabled.
  • Added support for ignoring SSL certificate errors for IE and Firefox (Chrome already had it).
  • Improved the quality of screen shots and video when the browser viewport is small.
  • Tweaks to the end state detection to allow for measuring an ajax event after a page load (where no document navigation happens).
  • Fixed issues with Firefox 22+ SSL capture.
  • Fixed video capture for Chrome 28+ where it was displaying a ruler grid instead of a blank page.
  • Fixed support for capturing video when Chrome is using GPU acceleration.
  • Fixed double-counting of bytes for encrypted requests.
  • Fixed reporting of navigation errors (IE).

URLBlast (legacy IE testing):

  • No significant unique changes.

Improvements common to both agents:

  • Added support for reporting user timing marks.
  • Added an optimization check to see if JPEGs were progressive or not.
  • Added support for expiring cache entries that are going to expire within X seconds (expireCache script command). Only supports IE right now.
  • Added support for regex filtering in the setHeader command (had only been implemented for the addHeader command).

Downloads