Usability Testing

USS believes that usability testing is a core discipline in our field. We try to practice it and to continue refining our skills in doing usability tests. The discussions that led to this statement have been useful in clarifying our thinking and describe the state of the art as we see it today. Perhaps this statement can be helpful to you. Send us your comments if this is not the case or if you see ways of improving the statement.

Usability testing is a core skill because it is the principal means of finding out whether a system meets its intended purpose. All other skills that we deploy or cultivate aim to make usability (and, ultimately, use) successful.

Think about why you will be doing a usability test.

How do people interact with the system you are testing? What is difficult or easy for people to do? What makes sense about it? What is exciting about it? What changes would users like to see? What do they really hate about it? The way subjects actually use your system may reveal bugs that are invisible to you. It also may suggest enhancements that do not make sense or seem necessary when you test the system.

Consider the system as a whole.

Think of a usability test as a way of examining your thinking about the system. What are the boundaries around it? That is, a usability test is a means of thinking about the purpose and function of the "whole product" or "whole system." What additional tools, information, skills, and support will people really need to use your system? In a client/server context, you need to test "the whole" even though some elements may be fixed and beyond your control. (Of course, if you can't influence the behavior or structure of a system at all, there isn't much point in a usability test.)

Make sure the system is ready to test.

Convince yourself that the system is functional and usable from your own perspective. Usability testing is expensive and your subject's time is precious. Have you used all the mechanical tests that are available to you? For example, have you run a spell-checker on all of the text that the user will see? Have co-workers or others on the design team checked out the system?

List several tasks that a user should be able to accomplish with the system.

The tasks should be simple so that first-time users have a chance of success. The tasks should be specific, and should be stated in a very clear way. (For example: "Determine the total of annualized salaries for classified staff in your department.") Describe those tasks in writing so you can give users something to work from. Write each task on a separate page to avoid foreshadowing effects. Do not include hints in the task descriptions. If the test lasts more than a half hour your subjects will be exhausted and so will you.

Make a list of potential usability test subjects.

Pick people completely unfamiliar with the system. (This implies that once someone has participated in a usability test, they are no longer useful in that capacity. They've been indoctrinated or exposed to the system.) The subjects could be people whom you know very well or don't know at all. There are different issues involved with each scenario. With friends, a rapport has already been established, they trust you, and they know you're not passing judgment on them. With strangers, this rapport needs to be established.

Plan for data collection

How will you write down what you hear and what you observe? Would thumb-nail sketches of the screens the user will see be a good armature for your notes?

Schedule the test.

It is best if you can go to the users' own environment where they are most comfortable and it will take a minimum of their time. This allows them to focus on the tasks and the system, rather than on getting comfortable with a new mouse, a different version of the software, or a different context of whatever sort. These usability test notes assume that there will be one subject and one observer; sometimes a team of observers makes sense.

Prepare yourself to be objective.

It takes a lot of emotional and intellectual energy to listen and receive feedback that may be negative.

Don't take this "how" file with you.

If this testing ritual isn't intuitive and natural, it probably should be re-written. You've been nominated!