CHI-WEB Archives

ACM SIGCHI WWW Human Factors (Open Discussion)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
"Nick RAGOUZIS (Interfacility)" <[log in to unmask]>
Reply To:
Nick RAGOUZIS (Interfacility)
Thu, 1 Jun 2000 20:22:30 -0700
text/plain (198 lines)
> From: Joe Clark <[log in to unmask]>
> Date: Thu, 1 Jun 2000 16:26:01 -0500
> To: [log in to unmask]
> Subject: Promoting usability testing
> You ever approach a project ... and then you get all paranoid.
> Jeez, am *I* ever incompetent, you think.

Yes, oh yes.

Joe later wrote:

> Usability testing quantifies all our *other* work.

Quantifies, shapes, even drives. Yes, testing, in all of its guises and
opportunities has many potential contributions. It could be *the* driving
force for a design company ... or a client.

That (i.e., the unbounded potential) is what I intended to lay down when I
wrote (in

>> * Situating in your firm's professionals an unfaltering ability to present,
>> discuss, close, and execute a project implementation process that fully
>> includes usability testing
>> * Do this in a way that clients will embrace and even demand.

But note that I carefully did not say that is should be, must be, the raison
d'etre, the basho, (le for de fait, the field of play), or the altar mayor
for the firm and its work.

> I think it is not hard to imagine client meetings in which we merely
> say something like "Usability testing is part of industry best
> practices nowadays. It's expected of good Web designers, like testing
> pages on multiple browsers and platforms. We test our designs. Maybe
> other firms are so confident in their work that they assume theory
> equals practice, but we're trying to provide a bulletproof solution
> for you here."

Not hard to imagine at all. Bill Osborg sent me off to Bruce Tognazzini's
column for this month ( Thanks Bill. Judy, Alistair,
Gregory, and you, to name a few, also addressed these aspects. All advice
important to understand and to seek to apply.

But the question Bill wrote to us went beyond this.

His question was not just about an effort
>>> to incorporate usability testing into all our project[s]

but also relative to
>>> proposals

and the questions that come up there, in client meetings (he gave one, I'm
thinking of tens of others in the same line).

These two questions,

(a) the question of iterative design processes involving all constituencies
in highly-leveraged roles,

(b) the question of marketing and sales, in building and sustaining the life
blood of the firm,

although interrelated in motivations and execution, are, ultimately,

Do you have new arguments to demonstrate how they aren't? I'd love to hear

>> If so, the question might be better reversed: How important do you'all think
>> usability testing is to what you produce? Quantify it; qualify it.
> The thinking seems reversed.

Well, perhaps you're thinking of the other thrust of the message. The
thinking in this point is exactly those behind (a). If your firm believes
that usability testing is the most important thing to what is produced (and
I doubt that; I go further, I think it would be pathological -- I follow Don
Norman in the point he pursues, although perhaps not his argumentation :-),
as is obvious from these messages) then do it. Client questions be damned.
Let them walk out the door, hell, push them into the competition's arms ...
"We don't want their unenlightened kind in our midst. They'll kill off the
competition and we'll still be standing."

But my point which you quote as possibly an instance of reversing the
thinking does move us along. It begs the possibility that the firm's
projections, to stay within Bill's question for a moment, say that the
*mandatory* presence of "usability testing" will reduce the firm's win ratio
and its revenue vis-a-vis it's competition.

This possibility leads to the need for deep thinking in the firm about the
importance and relative emphasis of the components of (a). (One especially
doesn't want the firm to embrace some or all of these components as
universal, then repeatedly redline it from proposals. Right?)

What if, however, it increased profits? Not through some fancy new,
proprietary-no-doubt (the Experience-O-Magic Commerce-Tracker (tm) --
imagine the attachment armature), usability testing ... but because it
reduced the amount of advertising and marketing the firm must do: a higher
percentage of the firm's clients become segment leaders, become reference
accounts, talk about the firm in their Red Herring interviews. (Given the
high number of projects that involve the firing of at least one design firm,
this kind of increase is not at all far-fetched.)

>> You have other, perhaps as inappropriate for your firm, options. A
>> higher price without usability. A longer process without usability.
>> A guarantee of some sorts with it (perhaps scaled to various
>> levels); no guarantee without it.
> I do like this psychology.

Guarantees, which by their nature motivate the firm to do whatever it
thought best assured that it would not be called to cashier them, follow the
same line as I mention above. Usability testing *could* be an explicit part
of every proposal that contained a guarantee of some kind. Or not -- it
could be buried; it could be something the firm just does not do. Or,
perhaps, just does not do in a way that the client or competitors would
recognize as precisely usability testing. And none of these *require* that
the price be higher, or the schedule longer.

Further, what if a firm thought it sooooo very fundamental that it included
training for the client's staff to incorporate a similar way of dreaming up
and implementing projects. Even in the way the client interacted with its
users. Perhaps as a way to hedge their bets against a guarantee being
called. Okay ... I've gone too far.

>> I suggest, that while you are trying to justify usability's role
>> (and costs in money and time -- which I think in CHI-WEB we all
>> agree are actually lower *with* some usability work), that you
>> figure out what mission you've actually been given by your firm, and
>> how your firm might, as a whole, fulfill it. Otherwise you're left
>> with something akin to the role that usability plays when cast as
>> decoration: an answer that accomplishes nothing.
> I think that's a bit _trop_.

Perhaps. But this is the nub of it. All of the foregoing must be situated in
an economic framework that makes sense for the firm. By implication for the
client. Both within the marketplace. And such a requirement reveals that the
question of doing, or not doing, something called "usability testing", and
the question of talking about it in the pitch (or before or during -- staff
training!), or not talking about it, are entirely independent of each other.

If someone, or everyone, or anyone, in the presentation and close cycle with
a client feels compelled on their own recognizance to "fully address", or
simply to pitch, "usability testing," or even incremental development, then
that person's a danger to the firm. It must be done with an understanding of
what it takes to set the appropriate expectation in the client for the work
that might be won (now, or later after they firm the other firm), and for
making the sale. And that might mean being entirely silent on the matter,
regardless of whether you do, or don't do usability testing.

The problem of usability work being cast as decoration is well known to us
here, yes? Everything is done, and you're suppose to make it pretty. Such
work accomplishes (asymptotically, two-sided) nothing -- for the firm, for
the client, for the user.

A usability testing pitch unleashed in response to a simple probe from the
customer, or in response to a full-out objection to the presence to
usability (and the possibly extended time and inflated costs vis-a-vis
budgets or competitive commitments) is in precisely the same position (sans
metaphoric mapping).

I'd say the odds are that it is on the negative side of the equilibrium --
making things worse -- for if the usability pitch is in response to a
full-on objection, then there's something seriously weak in the firm's
marketing and the sales team's ability to read the customer. And at that
point what's needed isn't simply another run through the reasons for such an
approach to engineering the product ... but a review of why the client is in
the room.

All of this, but especially the last points, are why I think someone in a
situation such as Bill's, but perhaps not Bill in his current situation,
must push the question back up, into an organization-wide question. Such a
person must be aware of the economics, the sales and marketing, the
processes, the execution, ..., the theory and execution of the firm and its
clients, and the ultimate users. Simply having a "t" sheet for the role of
usability testing, and a WBS for it's place in the process, isn't gonna cut
it. Not in the long run.

And there's still much more to it than just this.

Nick Ragouzis         [log in to unmask]         1-415-922-3463

- Interfacility  - - - - - - - - - - - -
Helping organizations accelerate interactive design projects.

- iBuilders Project Managers Forum  -
Home of the Collexicon of Project Manager's Frustrations.