Re. Quality/validation

[geoff-h -at- MTL -dot- FERIC -dot- CA]

 A few thoughts for the anonymous poster who asked about a
 disfunctional quality assurance process.

 The QA people aren't doing their jobs, and the QA manager
 doesn't understand this. Try to demonstrate how QA is
 falling down on the job. Collect lots of examples of the
 important things that they're missing, and a parallel list
 of the trivia that they're catching instead. Try to
 convince them that the former list is the important one,
 and explain why. This is delicate interpersonal stuff,
 because you must convince them of the problem without
 rubbing their noses in it.

 If that doesn't work, you probably have to take the same
 evidence one step further up the chain of command. Request
 the review in person, not in writing, and ask for
 confidentiality; ideally, the manager should present this
 to the QA manager as a management-sponsored review of the
 effectiveness of the review process. It may be obvious that
 you initiated the review, particularly if you couldn't
 maintain amicable disagreement in the previous stage, but
 don't make any enemies you don't have to make by being a
 public whistle blower.

 The manager of the QA manager should be willing to
 intervene in the process. In a case like this, you need to
 cover your ass pretty thoroughly, which is why you need
 hard evidence. The two lists I mentioned above are a good
 starting point; other list members may have additional
 suggestions.

 Another weak link in the chain is your expert reviewers.
 You noted that the quality of the reviews is highly
 variable, and two things can help here:

 First, get the QA manager or someone higher up the chain to
 make QA part of the performance appraisals or use some
 other trick to make it important to the reviewers; if
 reviewers don't do a good job, and you can demonstrate
 this, it will reflect on them and someone else can crack
 the whip. You're not necessarily trying to get them in
 trouble; if you can spot a problem here, such as poor time
 management, too much work, or an intractable programming
 problem, you may be able to get management to devote
 additional resources and lighten that person's load.
 Everyone benefits that way. Again, don't rub anyone's nose
 in a problem... this only works for puppies, and not well
 then.

 Second, work on how you present material for review.
 Dropping 300 pages unannounced on someone's desk on Friday
 afternoon and asking for comments by Monday won't get you
 too far. (Ridiculous extreme, but you see the point!)
 Giving a week's notice that two small chapters will arrive
 on Monday with a Friday deadline is more effective. A few
 more tips on getting a better review:

 1. Edit the text first so that reviewers aren't tempted to
 spend more time on your typos and style than on the facts.

 2. Provide concrete instructions about what to review, not
 a generic review request. "Review Chapter 12" is useless;
 you'll get far better results from "review pages 12-34 for
 the following technical details: all steps technically
 correct, no new features added since last design meeting...
 etc."

 3. Don't ask for reviews of anything that you can review
 yourself. For ex., if you have a product/design spec.
 sheet, don't ask reviewers to spot if you've missed
 documenting anything on the sheet. Develop a good,
 comprehensive style guide for your authors if you don't
 already have one, and do an internal check against the
 guide before you submit anything for review. There's little
 excuse for typos in a review manuscript.

 --Geoff Hart    @8^{)}
 geoff-h -at- mtl -dot- feric -dot- ca

 Disclaimer: If I didn't commit it in print in one of our
 reports, it don't represent FERIC's opinion.