Skip to content

Yeti Next

Jump to bottom
reid edited this page Nov 11, 2011 · 11 revisions

As existing open source testing solutions prove inadequate, Yeti will now compete in the Continuous Integration space.

Problems to Solve

After discussions with our customers, a few problem spaces have been identified:

  • Commanding other browsers to open and close Yeti's test page.
  • Exporting test results.
  • Stateless server.
  • Integration with (mocked) server-side applications.
  • Ability to work with other browser test frameworks.
  • Ability to test server-side JavaScript.

Constants to Maintain

Yeti does a few things very well. These things must be maintained:

  • Simplicity.
  • The existing command-line interface.
  • Ability to run browser tests quickly by "capturing" a browser.
  • Speedy test execution using the socket.io system for browser and internal communication.
  • Mobile browser compatibility, including Android and iOS.

Priority

The problem areas are being addressed in order of priority.

  1. Yahoo!: Integration into Node.js-based browser launcher. Easy access to test results.
  2. Canonical Ltd.: Easy access to test results.
  3. YUI Team: Replacement of YUI Test Selenium Driver and possibly Selenium itself.

Our releases will be timed to maintain this priority order; however, the design changes made will take all of these things into account.

Backlog

  • User-facing interface remains unchanged.
  • Developer API. require("yeti") and script your own additions.
    • Provide the ability to "plug-in" to various parts of Yeti's lifecycle.
    • Export test results using this system.
    • Events will be used, see API section below.
  • Integration with Browser Control System of choice.
  • Ability to replace YUI Test Selenium Driver in our own CI system.

API

Browsers

These APIs will use a unique ID per browser during Yeti's server lifetime. They also provide the User-Agent.

  • Browser seen event, if successfully loaded, you'll get:
    • Browser connected event.
  • Browser disconnected event.

Test Management

These APIs are for managing tests.

  • Batch added. Perhaps per device. Fire with testfiles:
    • Sanity checked? Fire dispatch event.
    • Fire invalid test event with failing testfiles.
  • Batch complete. Per device. This API provides device information.

This API will return test results.

  • Batch completed - for all devices.
  • Batch completed - for a browser ID.

Yeti

These APIs are for managing Yeti itself.

  • Add your own routes.

Browser Control System

In order to more efficiently use Yeti with a CI system, a script using the Yeti developer API should be able to command browsers to open to Yeti's test page. Possibilities include:

  • Integration with Soda, the old-school Selenium RC API for Node.js.
  • Contributing WebDriver additions to Soda.
  • Developing a lightweight agent program to open and close browsers for iOS and desktop browsers.

If Selenium integration is chosen, existing Selenium solutions are inadequate for running for the YUI team's own use. Better solutions would include:

  • Simpler software. Ability to hit a URL, open an executable and "capture" the entire system as a Yeti slave.
  • BrowserBox, Yeti edition. A Linux VM that contains many different browsers with software pre-installed that make it work with Selenium or Yeti.
  • Selenium-as-a-service. Internal projects, SauceLabs, etc. that provide a Selenium Hub as a service.

The Browser Control System should be assigned to another person to be worked on in parallel to Yeti work.

Clone this wiki locally