Re: Usability Testing and Documentation

Subject: Re: Usability Testing and Documentation
From: Jean Ostrem <jostrem -at- tdl -dot- com>
To: Cindy Hudson <chudson -at- ECS-INC -dot- com>, techwr-l -at- lists -dot- raycomm -dot- com
Date: Tue, 14 Mar 2000 10:22:16 -0800

At 11:42 AM -0500 3/13/00, Cindy Hudson wrote:

After some years of development, our documentation has never been put
through any usability testing. Now, the boss is willing to let me at least
try to set up a plan to see if it's feasible. There are some great Web sites
with good information, including User Interface Engineering
(http://world.std.com/~uieweb/), but I'd like to get some real world info.

Have you done usability testing on documentation? Care to make some
suggestions? The budget is limited, which means no consultant or even an
extra person. We'll have to make it work with our already busy staff.

Thanks in advance!

Cindy Hudson
Technical Communicator
Enterprise Computer Systems, Inc.

You can definitely do a low-budget usability test and get useful information. I did a usability test of an on-line help system a few years ago at a small company where we were as overworked as you can get, so I know it can be done.

My boss at the time was one of those theory guys currently being trashed on the list. (IMO, it takes both kinds to make the world.) He insisted that hardly anyone -wants- to read documentation, particularly when it is on-line help. They just want to use the product. I totally agree with this assessment. So to make the usability test as real world as possible, we did not label it a documentation usability test. We said it was a usability test of the application.

I strongly encourage you to use this same tactic. You'll not only get a usability test of your book, you'll get a usability test of the product it documents. More bang for the buck.

Our company had a training class to orient new users to the toolset. What we did was ask for volunteers out of the training class attendees to usability test a new version of the application. We took about 4 or 6 people. We did the two tests simultaneously in two separate rooms. In each room, we had one sort of objective third-party person (a guy not directly involved with the project) be the guy to talk to the test subject, and the writer sat as far off in a corner as possible (while still being able to see the screen) and took notes. We also videotaped the test, which was superfluous in my opinion. No one ever looked at the tapes.

There was one task with this application that everyone has to learn -- it's just not an intuitive thing you'd think the application could do. We figured this task forced even the most documentation-phobic users you could imagine to look in the manual. So we told people in the usability test to do a bunch of things, and one of them was that difficult task. We did tell them in advance that the on-line help system was available if they got stuck. When they got stuck, we watched what they did. Some turned to the help. Some did not. When people did turn to the help, we got our usability test.

This was my one chance to do usability testing of documentation, and it was fascinating. Among the things I learned:

- Most people want to be nice. They don't want to tell you the product is crappy and hard to learn. They will be far too apologetic about not understanding how to do the tasks in the test. Try to convince them that you are an uninterested third party so that you'll get more of an honest opinion from them. At the very least, never volunteer that you are the author of the book being tested. Then, you'll never get an honest response from them.

- People honestly feel stupid and frustrated when they don't get something. This should be something I know, but it honestly was a surprise to me. You get used to hearing from your fellow employees, who are not shy about telling you the difficulties they had learning the app by reading your stuff.

- You only have to test about 4 people. After 4, a pattern starts to emerge, and it's almost worthless to continue testing because it gets repetitious. Ideally, given all the time in the world, you should test 4 people, fix the problems they identify, and test again to find more problems.

Good luck!

Jean





Previous by Author: How to create a .pdf file from FrameMaker 5.5 (Education Version)
Next by Author: Fwd: Re: What sayest me... on Worthless TC Degrees
Previous by Thread: RE: Usability Testing and Documentation
Next by Thread: UNIX questions


What this post helpful? Share it with friends and colleagues:


Sponsored Ads