Quality Assurance Principles and Procedure

Principle #1: Whiteboard does not ship code that doesn't meet our quality assurance standards.

There is no amount of timeline pressure, no item too small, to exempt from Principle #1. No matter how good the rest of our work is, if there is a blatant oversight that we could have prevented, that kills confidence. The only exception to this rule is if a direct order comes from a director or principle partner at Whiteboard to override.

Note that QA takes time, so if you aren't performing QA already, you will see an increase in your estimates for a given task. However, over time, this avoids major cost both from a relational standpoint AND a time standpoint, as fixing bugs before shipping is much less complicated than backtracking to fix them.

Principle #2: Interface bugs trump all.

If you have to choose between fixing an interface bug or speeding up your code, start with the interface, every time. A visual bug is far more important than a performance issue. Sometimes they can be fixed at the same time, but start by fixing the interface, always.

Principle #3: Test the whole machine

Most often, our bugs are a result of a change made in one area of a site or application, and something breaking in an entirely different part of the application. For that reason, err on the side of testing the entire site. Writing good code will help avoid issues caused at a global scale, but sometimes all it takes is a few more minutes of running through seemingly unrelated parts of the site or application.

If you have automated tests, run the entire test suite before every push. It doesn't matter if your feature test passes for one feature; you need to be certain that you aren't causing regressions in other points of the code.

If you don't have automated tests, check everything you have done and everything that is possibly connected to that change. This is exhaustive and time consuming, but is not optional.

Testing Process for Websites

• Test all code in an equivalent environment to the live server environment. This requires setting up your machine to mirror that environment as much as possible. (This is a work in progress at Whiteboard, but this will also eliminate "works on my machine" problems)

• Test all code on all of the following classes of devices:

  • iPhone 5 and later
  • iPad Mini, iPad
  • Nexus (or similar Android based device)
  • Windows, latest 2 versions
  • Mac OS, latest 2 versions

• Test all code in these browsers and situations on desktops:

  • Browsers (Latest 2 versions on all, both OS's):
    • Chrome
    • Firefox
    • Safari
    • IE (Windows only)
  • Laptop, mobile width
  • Laptop, partial screen
  • Laptop or desktop, full screen

Note: Mobile issues are incredibly important. Be particularly aware of layering and touch-event issues on mobile, as these tend to frustrate the client the most. Note that the most common issues will occur in Firefox and IE, with some more minor issues in Safari. Chrome has become relatively stable, and is the most common browser for our development processes, but don't assume that Chrome on Windows will always render the same as Chrome on Mac OS X.

In order of priority, the top 10-15 scenarios of testing should occur like this:

• Mac OS X Chrome, latest (this is priority because it will act as your baseline)
• Windows Chrome, latest
• iPhone, latest
• Android, latest (Chrome)
• IE, last 2
• Firefox, latest (both OSs)
• Safari, Last 2 (both OSs)
• Previous version of all (if not already covered above)

What is, and what isn't, a bug?

What is a bug:

• Any and all issues related to brand mark display
  • Any issues related to favicons or logos
• Misspellings or incorrect grammar
• Visual overlapping issues
• Broken or incorrect buttons, links, images
• Form submissions not working
• 500 errors on normal actions
• Unreadable text
• Fonts or other assets not loading correctly
• Scrolling/container issues
• Inaccessible content
• Extremely long load times

What is not a bug:

• New features
• Requested features
• Features built one way, when the client meant for them to be built a different way.
  • (This is a product of communication breakdown, not broken or technically incorrect code)
• Changes to existing copy
• Changes to existing layout
• Load-time Optimization (sub 1s)