CHI-WEB Archives

ACM SIGCHI WWW Human Factors (Open Discussion)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Condense Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
"ACM SIGCHI WWW Human Factors (Open Discussion)" <[log in to unmask]>
Jurgen Koenemann <[log in to unmask]>
Fri, 25 Jul 1997 09:46:16 -0500
text/plain; charset="us-ascii"
Jurgen Koenemann <[log in to unmask]>
text/plain (75 lines)
>The debate this week at Web Review is whether graphic design matters
>in terms of usability on the Web.
>I start off by presenting some of Jared Spool's results from his user
>testing of 9 Web sites. One of his findings is that the graphics did
>not appear to help users find what they were looking for.
>Jennifer Fleming offers a counterpoint with examples where graphics
>can provide a visual hierarchy to improve usability.
>So, what's really happening? Is the current state of graphic design on
>the Web just so bad that the benefits are hard to see? Are bandwidth
>limitations making it impossible to use graphics well? Was Jared's study
>flawed? Does anyone really have a clue what is going on?

Let's open the can of worms here...

[ I DID attend Jared's presentation (SIG) at CHI'97 and
just got the copy of his slides (thanks Jared).
I did read the chapter on the web, but haven't gotten the book,
it may have the answers I'm looking for]

Jared set users up with "Scavenger Hunt" tasks to find information,
collected a large number of of behavioral data (completion times, link
following, ...) as well as other data (preferences,...) and he rated the
sites along many, many variables.

For analysis, they computed as far as I know, all possible pair-wise
correlations (tens or hundreds) between use data and site characteristics.
Not surprisingly, some turned out to be significant (e.g. his
counter-intuitive findings on readability), but most did not (incl.
"graphics"). It's the law of statistics independent of the factors tested.

This study is a typical fishing expedition and Jared himself says "the
results are very preliminary". Unfortunately, these prelimanry results get
now condensed to "findings" and are now touted as the latest "truth" about
web site design.
I suggest to relax and wait for some well designed experiments and their
outcomes that focus on some of the interesting factors.

Some multi-variate analysis may help too, e.g. "more text" and "less
graphics" and "number of links" are probably not independent and the
correlations Jared found may indeed measure something else. Similiarily,
some sites heavy on his "graphics" scale also had specific problems with
graphics-related elements like buttons (Inc, Travelocity). There are also
issues of familiarity (Edmunds car
site versus investments), and the list goes on and on...

It is worth repeating that correlation -> causation doesn't work, and that
no correlation -> no effect is not a valid conclusion either.

I found the value in his study NOT in the overall results but in the
detailed analysis of individual cases that worked or didn't work.

The general conclusions that good design can't rescue bad content and
structure and that bad graphics can mess otherwise good things up are not
unexpected. It calls for follow-up studies that isolate individual

Jurgen Koenemann

Jurgen Koenemann, Ph.D.                 Phone: (540) 231-5723

Virginia Tech (VPI&SU)                  Fax:   (540) 231-6075
Department of Computer Science          Email: [log in to unmask]
660 McBryde Hall
Blacksburg, VA 24061-0106