@pmeenan pmeenan released this Dec 13, 2017 · 256 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPageTest 17.08:

WebPageTest and wptagent now have "release" branches in github that will be updated any time a new release is made (making it easy to git clone/pull a stable release build).

Web UI

  • Added Lighthouse scores to the top-level grades
  • Added video as a distinct content type (waterfall, breakdown)
  • Expanded the filmstrip UI and admin pages to full screen width
  • Fixes for multi-step test support
    • Video rendering
    • HAR export
  • Added reporting of agent uptime
  • Fixes to EC2 agent auto-scaling logic
  • Improvements in "Time to Interactive" measurement

Agent (wptagent)

  • Added support for Microsoft Edge and Internet Explorer (full support except for header modification on IE)
  • Added support for testing Opera (full support on desktop and android, behaves like Chrome)
  • Fixes for testing Chrome 63
  • Fixes for testing Firefox 57
  • Fixed User agent overriding in Chrome and Firefox
  • Improvements to reliability of Lighthouse testing and error reporting
  • Fixed setviewportsize and setuseragent script commands
  • Added support for using vpn-reverse-tether for revers-tethering android devices (up through 6.0 for now)
  • Improvements to the optimization check processing (faster processing, improved CDN detection)

wptdriver (Legacy Desktop Testing)

wptdriver is deprecated and in mid-2018 will no longer be able to test Chrome. All testing should migrate to using wptagent.

@pmeenan pmeenan released this Aug 25, 2017 · 410 commits to master since this release

Assets 3

Configuration Instructions

Releases are moving to a YY.MM numbering scheme and will (hopefully) be released quarterly.

Changes from WebPageTest 3.0:

With close to a year of commits there are a LOT of bug fixes, support for updated browsers, etc. This will just cover the main feature changes.

The biggest change by far is a new cross-platform agent (wptagent) that supports Linux, Windows, Mac and Android testing. Eventually wptdriver will be deprecated and all testing will be moved to the new agent.

Web UI

  • Added display for browser main-thread blocking to the waterfall and Time to Interactive metrics (Chrome)
  • Added per-request JavaScript execution times to the waterfall (Chrome)
  • Added reporting of uncompressed resource sizes (JavaScript and CSS in particular)

Agents

  • Added support for running Lighthouse tests as part of wptagent (on both mobile and desktop agents)
  • Added collection of First Contentful and First Meaningful paint timings (Chrome)

JS (Mobile testing - legacy)

  • The JS agent has been discontinued in favor of wptagent (and the public instance has been running the new code exclusively for several months).

wptdriver (Legacy Desktop Testing)

  • Added collection of Time to interactive and interactivity metrics
  • Switched how SSL Hooking is done in order to support Chrome 61+
  • Added support for new Firefox extension model (required for multi-process Firefox)

@pmeenan pmeenan released this Dec 16, 2016 · 885 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPageTest 2.19:

With a year of commits there are a LOT of bug fixes, support for updated browsers, etc. This will just cover the main feature changes.

  • Video, Chrome Trace and tcpdump processing has been pused to the agents and is no longer done on the server (for better scaling).

Web UI

  • Added support for PHP 7
  • The UI now supports reporting the output of multiple steps for multi-step scripts (huge thanks the the massive amount of work that sburnicki put in on ot).
  • JS Parse and Execution time is displayed in the waterfall on a per-request basis when Chrome timelines are captured.
  • The times when the main thread is blocked/interactive is displayed below the waterfall when Chrome timelines are captured.
  • When tcpdump is captured with a test the bandwidth display in the waterfall will be calculated from the packet capture and will also display spurious retransmits (duplicated data).
  • There are several new settings but two in particular that you should consider changing on your existing install if you are doing an update:
    • Switch the waterfalls to the new scheme where they are colored by mime type:
      • mime_waterfalls=1
    • Enable auto-update of the test agents as new ones are rolled out to the public instance:

Test Agent

  • URLBlast support (Testing IE <= 8) has discontinued and is no longer included in the releases. Existing installs will continue to work but there will be no more updates.
  • Windows Safari support has been discontinued. It was last updated at Safari 5 which is ancient and unused at this point.

JS (Mobile testing)

  • The mobile agent is now stable and should work reliably (more reliably for Android than iOS testing though)
  • Limited support for testing several browsers as "black-box" browsers (visual metrics but no waterfalls)
    • UC Mini
    • Opera Mini
    • Opera
    • Samsung Browser

wptdriver (Desktop Testing:

  • Added support for testing Microsoft Edge.
  • Added Windows 10 support and now requires Windows 7 at a minimum.
  • New traffic-shaping driver (winshaper) is now used for Windows 8.1 or later.
  • Added support for testing 64-bit versions of Chrome and Firefox.
  • Added support for decoding TLS streams for Chrome so response bodies and optimization checks now work.
  • Added support for decoding and reporting HTTP/2
  • Added support for decoding Brotli compression
  • Improved the image optimization check to look for bloat in PNGs and any non-animated GIFs that should be PNGs.

@pmeenan pmeenan released this Jan 22, 2016 · 1986 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPageTest 2.18:

Web Server

  • Added support for emulating arbitrary mobile devices (Chrome)
  • Updated embedded Chrome Dev Tools and Tracing UI
  • Added support for reporting user timing measures(API only)
  • Fixed target TTFB logic/grading
  • Added reporting of the agent Windows version in getTesters.php to prepare for Chrome deprecating < Windows 7 (6.1 actual version)
  • Updated EC2 AMI ID's to new/refreshed 2016 versions of the AMI's
  • Added support for per-connectivity or per-location test timeout settings

Test Agent

js (Mobile testing)

  • Improved iOS testing with 60fps video capture, tcpdump, feature compatible with Android testing
  • Changed test end conditions to match desktop (at onload or after 2 seconds of inactivity after onload)
  • Added support for testing Chrome on Android Marshmallow
  • Improved reliability
  • Updated to support Node JS 4+
  • Error test if Chrome crashed during testing
  • Added support for collecting user timing measures
  • Added support for the setDns script command

wptdriver:

  • Updated to support latest Chrome and Firefox releases
  • Added HTTP/2 decode support for Firefox and IE 11 (including server push)
  • Support talking HTTPS to WebPageTest server (including client certificates if configured)
  • Reduced overhead when testing Chrome
  • Added support for emulating arbitrary mobile devices (Chrome)
  • Fixed mobile emulation viewport size to match actual device
  • Block automatic browser downloads
  • Added support for collecting user timing measures
  • Added reporting of the current Windows version to prepare for Chrome deprecating < Windows 7 (6.1 actual version)

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

@pmeenan pmeenan released this Jul 20, 2015 · 2288 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPageTest 2.17:

Web Server

  • Fixed a critical remote code execution vulnerability (linux servers)

Test Agent

js (Mobile testing)

  • Added support for processing video captures on the node JS agent system instead of the server
  • Added support for custom metrics (JS that executes after the test completes)
  • Added reporting of user timing marks
  • Added reporting of navigation timing
  • Added support for the "ignore certificate errors" option

wptdriver:

  • Added support for decoding HTTP/2 for Firefox and IE (including server push)
  • Added support for capturing the SSL Key log file when running a tcpdump capture (Chrome and Firefox only)
  • Switched to mozjpeg for the jpeg compression check and for saved images (screen shots and video frames)
  • Fixed support for the setEventName script command

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

@pmeenan pmeenan released this Jun 24, 2015 · 2355 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPagetest 2.16:

Web Server

  • Added custom metrics to the result page
  • Added (optional) rate limiting for video rendering based on server load
  • Added support for connectivity-specific test timeout limits
  • Added support for newer versions of ffmpeg
  • Improved performance of mobile video processing
  • Improved EC2 auto-scaling

Test Agent

js (Mobile testing)

  • Added support for ignoring certificate errors
  • SPOF, Content blocking and setDNS support added
  • Added support for Chrome Dev Channel on Android
  • Added support for running custom chrome_shell builds on Android
  • Allow android tests to run for longer than 3 minutes with truncated video
  • Fixed support for Android Lollipop
  • Fixed packet loss support in traffic shaping
  • Fixed Chrome timeline capture and reporting (Chrome 42+)
  • Improved mobile agent reliability (particularly on Windows)

wptdriver:

  • Fixed support for setUserAgent for Chrome HTTPS requests
  • Added support for eventName script command
  • Added support for using HTTP basic auth over HTTPS between the agents and server
  • Improved chrome mobile emulation (disable flash)
  • Fixed packet loss support in traffic shaping
  • Improvements to Chrome HTTPS waterfall processing
  • Fixed Chrome timeline capture and reporting (Chrome 42+)
  • Added support for calculating speed index without video capture
  • Disabled the Firefox "browser upgraded" startup page

URLBlast (legacy IE testing):

  • No significant changes (deprecated - use wptdriver where possible).

@pmeenan pmeenan released this Dec 18, 2014 · 2626 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPagetest 2.15:

There are quite a few big additions/improvements in this release:

  • Added support for an easy-deploy server running on EC2.
  • Mobile agent now (correctly) closes the browser between first and repeat view to match the desktop behavior.
  • Moved to server-side video rendering.
  • StatsD integration.

Web Server

  • Moved to server-side video rendering.
    • Significantly improved video layouts.
    • Support for 60fps mobile videos.
    • Supports huge numbers of tests (up to 150 in the same video was tested)
  • Eliminated dependency on the specific version of ffmpeg for processing mobile video (now works with 1.x and 2.x).
  • Several additions to the install check:
    • Added the version of the running release.
    • Added a check for freetype (needed for server-side video rendering).
  • Added "error rate" reporting to the getTesters.php view.
  • Significantly improved the performance of the test history view if running on Linux.
  • Removed the Progressive JPEG recommendation (currently controversial).
  • Added support for an easy-deploy server running on EC2.
    • Added support for automatically launching EC2 test agents.
  • Added support for automatically pulling the latest agent updates from the public WebPageTest instance.
  • Improved performance of the post-processing of mobile tests.
  • Added support for sharded WebPageTest servers (server ID as part of the test ID)
  • Added support for posting results to StatsD.
  • Integrated with Google Safe Browsing API for filtering of test requests (optional).
  • Added a hybrid work queue where beanstalkd can be used for API tests but the legacy mode (with test position) can be used for interactive tests.

Test Agent

js (Mobile testing)

  • Added support for reporting the DOM Content Loaded time
  • Agent now (correctly) closes the browser between first and repeat view to match the desktop behavior.

wptdriver:

  • Fixes to processing of Chrome requests (HTTPS in particular)
  • Fixed the Chrome mobile emulation
  • Fixed video capture dor latest Chrome releases

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Improved IE testing thruput (eliminated 5 second delay when running tests).
Sep 23, 2014
Added a bunch of tips

@pmeenan pmeenan released this Jul 22, 2014 · 2837 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPagetest 2.14:

There are quite a few big additions/improvements in this release:

  • The addition of custom metrics.
  • Logs integration for test results.
  • Significant improvements to the new mobile agents.
  • The mobitest android agents are no longer supported (security concerns with the results processing).
  • Added support for capturing and viewing Chrome traces.

Web Server

  • Added support for writing test results to a log file (see logTestResults and logTestRequests in settings.ini.sample)
  • Significant improvements to the processing of mobile video captures.
  • Fixed the display of console log messages from mobile tests.
  • Fixed the waterfall translucency slider support in filmstrip view for Chrome 30+.
  • Improved test reliability (reduced instances of corrupt results).
  • Added sample benchmark configurations.
  • Fixed timeline processing for Chrome 36+.
  • Added support for viewing Chrome traces.
  • Added installer checks for php's memory limit, for tmp being on tmpfs and for being able to write to the dat directory.

Test Agent

js (Mobile testing)

  • Significant improvements to video and waterfall processing (done on the server but mobile-specific).
  • Changed the logic to close the browser between first and repeat view to match desktop testing.
  • Added support for running against an emulator (though it is very much not recommended for anything but dev).

wptdriver:

  • Added support for capturing Chrome traces.
  • Fixed support for multi-step tests.
  • Fixed the reporting of HTTPS 204 responses.
  • Added support for overrideHost, setHeader and addHeader commands for IE HTTPS testing.
  • Improved the support for the exec and execAndWait commands in Chrome so they no longer run in a sandbox and can access the page variables.
  • Added support for capturing the page title and titleTime for Chrome (already worked for IE and Firefox).

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Added support for capturing custom metrics.
  • Improved the adult site detection logic.
  • Changed 401 response codes (auth challenge) to be treated like redirects.

@pmeenan pmeenan released this Apr 9, 2014 · 3056 commits to master since this release

Assets 3

Configuration Instructions

Changes from WebPagetest 2.13:

The biggest news for this release is the first stable release of the new mobile Android testing agent which includes high-speed video capture and full timeline reporting.

Web Server

  • Significant improvements to test management, eliminating a lot of cases of "missing data".
  • Added a timeline CPU utilization graph below the waterfall indicating JS execution, layout, etc.
  • Fonts added as a separate mime type for content views (waterfall and breakdown).
  • Added support for displaying 60fps video frames in the filmstrip UI (from mobile).
  • Added content bodies to HAR export if bodies were captured at the time of the test.
  • Added support for Appurify Kindle testing.
  • Added support for importing a dev tools trace as a stand-alone test (for integration with other tooling).
  • Added error messages from test agents to the test results UI for better diagnosing of failed tests.
  • Added support for editing test labels.
  • Improved thumbnail tracking in the filmstrip view.

Test Agent

js (Mobile testing)

  • Initial release. Setup instructions are here.

wptdriver:

  • Fixed IE 11 support.
  • Updated Firefox and Chrome support to support the latest releases.
  • Fixed traceroute test support.
  • Significant changes to how code injection is done for significantly improved reliability (eliminating cases of tests with missing data).
  • Improved reporting of HTTPS resources when testing with Chrome.
  • Switched ignoring of cert errors to only be enabled if explicitly requested with the test.

URLBlast (legacy IE testing):

  • No significant changes (will eventually be deprecated and replaced by wptdriver).

Improvements common to both desktop agents:

  • Video capture (when enabled) runs continuously instead of only when paints are detected which enables better GPU animation and video playback capture.
  • Added support for pulling tests from multiple locations. i.e. "location=wptdriver,IE_10".