Re: Usability Testing--Interim (I hope) summary
Sandra Charker tells the sad story of usability testing that pointed up even
more flaws in a documentation set than were expected by the writers. Then she
goes on to inform us that the testing she did was a wasted exercise: "One sad
part of this is that there's very little we can do about it. Another sad part
is that this dismal documentation story comes from a very good software shop
that would smother its collective head in ashes if its software failed even
half as badly." (I might argue that if the software shop was so good, its
documentation would be better...or that testing without implementing a
correction plan is almost criminal negligence. Why bother with testing if
you're not going to respond to the results? You make A. Plato seem positively
brilliant.)
Except that in this case a correction plan for the documents is not the best plan for the product or the users, IMO. They will be better served by automating the stuff that is at present badly documented than by improving the documentation. It is a very good software shop; it's just that documentation is the weakest section. In the circumstances, I think the appropriate response to these usability test results would be to stop trying to maintain "traditional" or "standard" documentation, and settle for technical notes that are written and maintained by SMEs with the support of a good technical editor (not necessarily full-time or on staff). That would enable the company to apply more resources to its strengths in software development, including interface design.
Incidentally, I didn't do the testing or write up the results. The usability engineer did.
Forgive me for seeming so cynical, but if we are writing to meet the needs of
an audience, should we not be able to (a) define those needs, and (b) test our
documentation to determine how well we are meeting those needs with an eye
toward doing even better as we move forward?
Yes we should, but one of the needs of most software users is not to need documents at all. Tim's description of usability testing including testing "how long it takes each [test subject] to find the text telling the subject how to do perform a given task." So far so good. But if I were usability testing my product, rather than the documentation, I'd also want to test how long it takes a subject to perform a given task without consulting the documentation. If they took less time to figure out the software than to consult the documentation, then I'd be getting rid of the document, or at least that part of it.
That's why I don't think TWs should usability test documents any more than software developers should usability test software. It's too easy for us to assume that fixing the documents is the best solution to fixing the user's problems.
Apparently, where usability testing IS occurring, it is a mere bureaucratic
exercise. You all have made me feel at least honest in not pushing it. It
sounds like a waste of time.
That's not the point I meant to make. True the writers knew there were problems with the documentation, but usability testing showed us problems that neither we nor anyone else knew about, and it gave us and management some objective data for decision-making. That's not a waste of time.
Does anyone have any success stories for usability testing?
I think this is a success story, Tom, even if as a writer I don't much like the ending. It's sad that a lot of hard work has gone into those failed documents, but there's no law saying that usability testing shall always find that a product or a document is worth fixing. The usability tests provided evidence that the current documentation is passing the point of cost-effective maintenance. Other software products have documentation in the same state, but they don't know it.
Sandra Charker
mailto:scharker -at- connectives -dot- com
http://www.connectives.com
Previous by Author:
Re: Usability testing
Next by Author:
Request for a metadocument
Previous by Thread:
Re: Usability Testing--Interim (I hope) summary
Next by Thread:
RE: Usability Testing--Interim (I hope) summary
Search our Technical Writing Archives & Magazine
Visit TechWhirl's Other Sites
Sponsored Ads