Just to add the Sharon's warning about the use of a think aloud
protocol, I have a colleague who uses layered voice stress analysis to
analyze respondent comments. He normally does it for market research
work but was recently asked to perform an analysis on a think aloud
protocol usability evaluation. His findings were that people
performing a think aloud protocol while trying to perform a task
showed signs of a loss of concentration for both activities (not
really unexpected). He also found that people who had to talk while
working out a problem showed "relatively high levels of physical
tension... anticipation, fear, embarrassment, and guilt." In other
words, if participants are required to talk while working through a
process, they cannot filter their responses and end up embarrassing
themselves or fearing they will. This may seem like a good think to
some (the ability to know their thought process without filtering),
but the resultant effect on their emotional state and performance
makes this more of an artifact of the protocol than data.
We only use think aloud in evaluations of conceptual designs (e.g.,
storyboards or static prototype) or only after task completion. We
also know (recently proven again in a for Sharon as a matter of fact)
that time-on-task does not correlate with performance so time a
measure of interest only in special cases. And TOT is never real if
taken during a think aloud protocol.
Bill
On Sep 15, 2008, at 9:24 AM, Sharon Laskowski wrote:
> I would also encourage you to read some of the classic books. Rubin
> and Chisnell have just published the 2nd edition of "Handbook of
> Usability Testing", for example. And consider joing the Usability
> Professionals' Association and attend their conference.
>
> I'm a bit concerned that you said you would be measuring time in
> your earlier note, but here you say you expect a think-aloud. Time
> would not be very informative if they are thinking aloud--too much
> variation in how people discuss. Think alouds can also affect error
> rates. The above book has some good advice about using think-
> alouds.
>
> Sharon
> Quoting [log in to unmask]:
>
>> Hello All,
>>
>> I appreciate all the quick response; all the materials were really
>> helpful
>> to get an idea and gathering knowledge about usability testing;
>> however
>> would still be waiting if someone would like to share their
>> experiences
>> regarding any major problems faced during the user testing; as while
>> conducting few of my tests, I noticed that few of the users were very
>> conscious when they knew that they are being noticed and the
>> sessions are
>> being recorded. I believe this would not lead me to the results I am
>> targeting for; also I noticed that they are unable to think a loud
>> as I
>> see them being conscious and hastate to do so in presence of a
>> representative taking notes; any guidance or shearing from the
>> members
>> would be great help.
>>
>> Thanks once again.
>>
>> Best Regards,
>> Sandeep Rathod
>> Sr. HCI Designer
>> --------------------------------------------------------------------------------------------------
>>
>> Zycus - SAVE WITH EASE
>> Email: [log in to unmask] | Tel. Desk: +91 22 6640 7676
>> (Extn: 7689)
>> Website: http://www.zycus.com
>> --------------------------------------------------------------------------------------------------
>>
>>
>
> --------------------------------------------------------------
> Tip of the Day: Suspend your subscription if using auto replies
> CHI-WEB: www.sigchi.org/web POSTINGS: mailto:[log in to unmask]
> MODERATORS: mailto:[log in to unmask]
> SUBSCRIPTION CHANGES & FAQ: www.sigchi.org/web/faq.html
> --------------------------------------------------------------
>
Bill
---------------------------------------------------
Bill Killam, MA CHFP
President, User-Centered Design, Inc.
20548 Deerwatch Place
Ashburn, VA 20147
703-729-0998 (Office)
703-626-6318 (Mobile)
http://www.user-centereddesign.com
--------------------------------------------------------------
Tip of the Day: Email mailto:[log in to unmask]
with any comments, questions or problems
CHI-WEB: www.sigchi.org/web POSTINGS: mailto:[log in to unmask]
MODERATORS: mailto:[log in to unmask]
SUBSCRIPTION CHANGES & FAQ: www.sigchi.org/web/faq.html
--------------------------------------------------------------
|
