Reprint: Tutorial Lessons We've Learned

["Jared M. Spool" <jspool -at- UIE -dot- COM>]

People have recently shown interest in developing on-line tutorials
to help users learn the key concepts of their applications. The
following is a reprint of an article that was printed in the
September/October 1996 issue of Eye For Design.  I thought you might
find this of interest.

(There is information on how to get a sample issue of Eye For Design
at the end of this message.)

     - o - o - o -

Tutorial Lessons We've Learned
by Carolyn Snyder

In testing printed and online tutorials, we learned some lessons of
our own!  We learned what users expect from a tutorial, and how they
work with it.  We also saw some problems a designer can encounter when
putting a tutorial online.

_____________________
Users Go Out of Order

In our tests, users chose their own path through the tutorials.  One
tutorial was presented in a particular sequence because it involved
building an application. If users completed sections out of order,
subsequent sections were not possible because of missing
prerequisites.   Despite a strong recommendation at the start of the
tutorial to complete all sections in order, some users still skipped
ahead to topics of greatest interest.  The designers realized that it
would have been better to make each section stand alone to support the
users' exploratory behavior.

_________________
Frequent Feedback

Two tutorials we tested were initially designed so that the user spent
the first several chapters constructing an application, and the
verification step of actually running the application wasn't
introduced until much later.  Users weren't comfortable with this --
they wanted feedback early and often on their work.  It's more
gratifying to get something working really quickly, even if it's just
a skeleton, like the classic "Hello world" application often used to
introduce a new programming language.

_______________________
Repetition Isn't Useful

The purpose of the tutorials we tested was to introduce product
concepts, not to improve user proficiency at completing tasks.  We saw
that users tended to skip sections of a tutorial that involved purely
mechanical repetition.  For example, when building a database form
containing five similar edit fields, users created the first field or
two, and then skipped ahead to the next unique activity.  They
commented that they didn't perceive any value in the repetition.

_______________________
Been There?  Done That?

When we tested an online tutorial, users had trouble knowing what
portions they had and hadn't covered, especially if they traversed
sections out of order.  In a couple cases, users accidentally skipped
sections.  The designers responded by adding checkmarks to completed
sections and by highlighting the recommended next section -- this
helped.

__________
Can't Skim

In an online tutorial, users found it much harder to skim or skip
portions of the content than with a printed book.  In early tests of
one online tutorial, we found that users were annoyed when the
tutorial covered material they already knew.  In some cases, their
annoyance was strong enough to color their perception of the entire
product.  This was actually good news for the development team -- they
took out the annoying parts, shortening the tutorial and improving its
effectiveness at the same time.

__________________________
Sharing Screen Real Estate

Sometimes, an online tutorial has to share screen real estate with the
software.  In one tutorial we tested, both the tutorial and the
application needed the whole screen, so the user had to switch back
and forth.  We found two issues with this:

Chunking of material: Users sometimes didn't remember when they were
supposed to switch back to the tutorial.  In testing, we found that
some of the switches came at awkward times.  The switches felt more
natural to users when they occurred at a point of closure, such as
clicking OK in a dialog box.  If the tutorial expected the user to
leave the dialog box open, we saw that the user would often get out
of sync with the instructions.  Based on this feedback, the designers
re-chunked the material to fit the patterns that were more natural to
users.

Remembering data: Almost every time users had to remember data (such
as the name of the file they were supposed to open) they either went
back to the tutorial to look at it again, or they entered it wrong.
The design team discussed ways to show the user the data right on the
screen.

____________________________
Auto-pacing Impairs Learning

Printed tutorials are inherently user-paced, but the designer of an
online tutorial can make it automatically paced by having it go to the
next screen after a pre-determined delay.  We saw that users may
absorb less content in an automatically-paced tutorial, when the pace
is not under their control. The online tutorial we tested started out
auto-paced, moving to the next screen as soon as the audio finished.
Users were quite passive.  Rather than utilizing the Pause button,
users opted to let the tutorial run at the pace the designer had set.

When we tested a self-paced design with Back and Next buttons, we
found that users' learning style became more active.  They spent
noticeably more time on the screens that contained new concepts,
taking the time to study illustrations and re-read text before going
on.  This had a positive effect on the amount of information they
retained -- users who tested the self-paced design did better at
answering technical questions about the material covered by the
tutorial.

____________________
When Animation Helps

One online tutorial displayed some complex visuals and devoted several
screens of text to explaining them.  We found that the first time the
visual appeared, users spent a fair amount of time studying the whole
thing, not realizing that the next several screens of the tutorial
would explain everything they were looking at.

The designers added some simple animation -- the tutorial highlighted
the section currently under discussion with a briefly flashing
outline.  This helped the users realize that the tutorial was going to
discuss parts of the illustration in sequence, so they didn't have to
figure out everything on their own.

         - o - o - o -

Eye For Design is published six times a year with articles on a
variety of product design and usability issues.

If you would like a complimentary issue mailed to you, just send
your postal address to efd -at- uie -dot- com -dot-   (Sorry, we do not have an email
version available, yet.)

Hope you found this to be of interest.

Jared

p.s. We'll ship your complimentary issue anywhere in the world,
as long as you tell us what country your from.

==========================================================
 Jared M. Spool                User Interface Engineering
 mailto:jspool -at- uie -dot- com     800 Turnpike Street, Suite 101
 (508) 975-4343                   North Andover, MA 01845
 fax: (508) 975-5353                                  USA
==========================================================