CHI-WEB Archives

ACM SIGCHI WWW Human Factors (Open Discussion)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Anindita Paul <[log in to unmask]>
Reply To:
Anindita Paul <[log in to unmask]>
Wed, 29 Sep 2010 15:10:19 -0500
text/plain (100 lines)

We recently did a study where we put three interface prototypes on the same
page from top to bottom (as there was not enough real estate to put those
side by side as well as making it clearly visible to the user) in a survey
to ask the users to rank their preferences.

The survey was a second phase of the study. In the first phase users came to
our usability lab and went through a "Think Aloud" study where they had to
compare three variations of the prototypes of a home page . This allowed us
to get their subjective impressions and based on their preferred interface
three more interfaces were drawn out for the second phase. The second phase
(survey) was largely to evaluate the aesthetics of the three prototypes.
Since the phase 2 required users opinion we went with a larger sample rather
than getting in-depth feedback from a small sample of users in the lab.

I think the survey was useful for us to get quick feedback from a large
sample especially to get their opinion. The in-depth part was covered in the
first phase and we did not want to get repetetions.

Hope this helps,

On Wed, Sep 29, 2010 at 9:18 AM, Susan Price <[log in to unmask]> wrote:

> Kay,
> Your proposed test is not what I think of as A/B testing. Our A/B testing
> is showing one or the other variant to the same user segment (usually in the
> live production environment) and watching which performs better.
> Presenting on the same screen side by side is an attempt to not lead the
> participants? This seems to invite careful perusal of the differences, and
> that's usually not what we're going for. What we seek in the usability lab
> are the untutored, knee-jerk reactions that we can't anticipate. Inviting
> careful perusal invites participants to play "designer." Our team doesn't
> suffer from a lack of such opinions - what we seek are the revelations that
> we weren't able to anticipate.
> There will be unavoidable left-right bias I believe, and the test
> experience doesn't compare to the "one solution per screen" actual
> experience.
> Seems like it might be better to show each variant (and perhaps some
> dummies) several times and have the participant rank each with a numerical
> scale, with duplications of each to weed out the biases.
> -Susan
> On Sep 29, 2010, at 7:54 AM, Kay Corry Aubrey wrote:
> > Hi - can anyone on the  Chi-web list suggest innovative usability test
> > design approaches for A/B testing? Both A and B will be presented on the
> > same screen and the user needs to peruse each and choose the one they
> > prefer. I am looking for ideas on the best UIs to present text and
> pictures
> > most effectively for this type of purpose and was wondering if folks on
> the
> > list have suggestions.
> >
> > Please send your ideas to me and I will collect, summarize, and send out
> to
> > the list.
> >
> > Thanks!
> >
> > Kay
> >
> > Kay Corry Aubrey, user-centered research and design
> > Usability Resources Inc | |
> > [log in to unmask]
> > Phone: 781-275-3020 | Fax: 1-781-998-0325
> >
> >    --------------------------------------------------------------
> >    Tip of the Day: Suspend your subscription if using auto replies
> >            CHI-WEB:
> >              MODERATOR: mailto:[log in to unmask]
> >    --------------------------------------------------------------
>    --------------------------------------------------------------
>    Tip of the Day: Suspend your subscription if using auto replies
>            CHI-WEB:
>              MODERATOR: mailto:[log in to unmask]
>    --------------------------------------------------------------

    Tip of the Day: Suspend your subscription if using auto replies
              MODERATOR: mailto:[log in to unmask]