What judging is like (WAS: Best examples of ...)

[Scott Miller <smiller -at- PORTAL -dot- COM>]

Regarding finding out what a good quality help system is by judging
them, Beth sez...

> Not to say that winning entries didn't deserve their awards -- I'm
> sure many
> of them did. However, I don't think that volunteering as a JUDGE is
> the best
> way to learn what constitutes quality. It scares me to think that
> entries
> might be judged by people who don't know what they're looking for.
> Sure,
> you're provided criteria and given limited "training" but without
> experience
> in the genre, you cannot differentiate between what "meets criteria"
> and
> what is innovative. (Outstanding vs. dismal is much easier to
> discern.) As
> an example, how many people complained about the lack of capitals in
> headings until someone said it was "downstyle", and new, so it's okay?
> What
> if someone's entry is based on new research that you know nothing
> about?
> "Different" doesn't mean "bad".
>
Most online judges that I've worked with are very experienced. There are
also beginners, but they are teamed with someone with a lot of
experience, both as a judge and as a writer. Not that it makes any
difference, however, since there are plenty of highly-experienced help
writers who aren't very good and don't know what quality help is. It's
possible to get a better, more fresh perspective from a less-experienced
writer than from someone who is more experienced, but less open-minded.

Also, the way it works is (in our competition anyway), you get the
software and the help, and basically do a usability study on it. The
important criteria is finding information quickly, which is a very
quantifiable property. You just put yourself in the role of a software
user, not a writer, and try to complete a task. If the help works, it's
a good help system.

As for new, innovative things, that's what most of us experienced judges
are in it for. It's a great way to steal ideas. Plus, there is a lot of
discussion among all the judges about what makes an entry an
award-winner, and what makes an entry less than successful.

As for the validity of the results... don't get me started.

- Scott Miller
smiller -at- portal -dot- com


 From ??? -at- ??? Sun Jan 00 00:00:00 0000=