Apples or Oranges: When to Test


By Marli Mesibov

A few weeks ago we looked at why user testing is an important part of any design process. But answering why is only half the equation. It’s equally important to think about when usability testing is necessary.

When we ask our clients why usability testing is necessary, two of the most frequent responses are:

  • The end user is the one using the product, and therefore, can most accurately speak to what parts of the design are working or not.
  • Usability testing can help us determine the clarity of our messaging and copy.

Though these statements are accurate, they suggest that testing happens only after the application is built.  This method of testing only evaluates a final product, giving ample room for going astray along the way. What’s wrong with that? By only testing a completed application, you greatly increase the chances that your product won’t meet your customer’s needs.  You may think the customers want oranges, but after the tree is grown, you discover your customer’s fruit of choice is apples! Assuming that usability testing only fits into the final stages of a project overlooks a key element, and a big benefit, of user testing: constant user feedback. If you ask us why usability testing is necessary, our most frequent response is:

  • Usability testing allows you to determine, at every step of your process, how to optimize your project for your users’ needs.

User testing is not something that should wait. Though it might seem superfluous, testing early and often through the creation process will positively affect your product.

Usability Testing in Practice

I recently encountered some clients who were having trouble making a decision on their registration form.  They were split between two options: typing in a date of birth manually, or providing a set of dropdown menus for the month, day and year.  Since there was an even division, both among the clients and within the Above the Fold staff, I suggested a usability test to see what the target users preferred.

We created a list of three open ended questions for our testers to answer, and found ten remote testers, each with ten minutes to devote to testing in the next 48 hours. I made a point of finding testers who fit the demographic of the customer’s target end user.  Of the ten, six found time to complete the tasks we created and respond to my questions right away, and we were able to immediately implement their feedback.

To my clients’ surprise, the test results answered more than just our original question.  We found that some users did not even feel comfortable entering their date of birth, regardless of the method.  This began a completely different discussion about whether the date of birth was a necessity for the application. When we determined that it was, we set about finding ways to gain the user’s trust before asking for such personal information. With further testing, we were able to find a suitable solution which appealed to the end users.  Without usability testing, the client might never have discovered this problem and may have lost potential customers down the line.

I am not my User

Items like this are not overlooked through carelessness or disinterest.  Usability testing is designed to allow you access to the user’s perspective. That mantra “I am not my user” comes back into play here – let’s look at what it really means.

  • Personal Attachment: You have more background with this app than your user does, and risk giving a less objective critique.
  • Benefit of the Doubt: You know your app will benefit your users. They don’t know this when they first see it.
  • Love at first site is Rare: Even the target audience needs to be shown, via good usability, how your app benefits them.

Usability testing adds the most important perspective: that of your users. The sooner you get this perspective, the more time you have to tailor your app to their needs.  And let’s face it; we all prefer personalized products to generic ones.

Usability Testing: Good Team Mojo

Saving user testing for the end of the project can result in significant wasted effort, time, and money on the project. A usability test can break down your assumptions, allowing you to see what your end users will see and learn what they really need in your application.  If your product is already 90% developed, this can send your entire business model back to square one. User testing needs to happen early and often.

By continuously allowing user feedback to inform design, rather than validating or invalidating assumptions after the fact, a team can create a truly usable app. And through continuous testing, you will build up your team’s confidence in your product, service and business model. When it comes down to it, there’s nothing more important than that.

Twitter Digg Delicious Stumbleupon Technorati Facebook Email