Back in 2003 we adapted techniques used in formal, lab-based usability testing to the “field” — where we’d watch users accomplish tasks on their own computers wherever they normally do their work (See Usability testing of Community Data & Mapping Systems).
This spring, we took this a step further in testing our our new Google-map based Repopulation Mapping System, but let the users define their own tasks. I had a twinge of guilt about not having formal tasks written down on index cards, but felt that we needed to let the neighborhood leaders who were testing the system show us what they’d do with the site. The information we gathered during testing resulted in several critical, concrete design improvements (though I still felt like we’d fallen short of a formal usability testing methodology).
I was relieved to find a label applied to this departure from structured tasks in the article by Mark Hurst, Asking customers for what you don’t already know. He talks about a “listening lab” where “the facilitator’s job is to invite the customer to demonstrate their own actual customer experience with the site. Rather than concocting an arbitrary set of tasks, the listening lab allows customers to use the site “for real,” as they would at home or work.”
We found that this listening lab approach combined with testing in the field, was a perfect fit for testing our new interactive map of New Orleans. It was perfect because every New Orleans neighborhood is different and has its own unique issues — ones we’d be hard pressed to create canned tasks for.
One neighborhood association president in NO East talked about a subdivision that had some subsidence issues, and she saw in the newspaper a lot of properties being sold to the Louisiana Land Trust (homeowners choosing options 2 and 3 in the Road Home program). She created her own task of zooming in to Oak Island to view how many Road Home applicants have chosen to sell to the state vs stay and rebuild. Her understanding of local conditions created a much better task than we would have ever created… and I loved hearing the story behind the statistics.
And as always, the “field” component of our testing was crucial in informing our design. And humbling. For example, our web stats show that only 5% of our visitors have 800 x 600 screen resolution, but 3 of our 5 testers were using that resolution and our map was only an inch and a half tall on their machines! These users are our core audience, and if they can’t use the system, then we’ve failed. Had they come into a lab to do the testing, we would have never been able to observe the way that they navigate the map and compensate for the small window size.
Next time we do usability testing, we’ll formalize this idea of “listening in the field” and not feel guilty about leaving the tasks on note cards behind!
Though I started in web design early in 1994, and had some early successes with a Webby nomination in 1999 and an article published at A List Apart, babies and hurricanes had frozen me in 1.0-land.