CHI-WEB Archives

ACM SIGCHI WWW Human Factors (Open Discussion)


Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Condense Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
"ACM SIGCHI WWW Human Factors (Open Discussion)" <[log in to unmask]>
Hal Shubin <[log in to unmask]>
Wed, 22 Mar 2006 16:06:39 -0500
text/plain; charset="us-ascii"; format=flowed
Hal Shubin <[log in to unmask]>
text/plain (273 lines)
I asked for opinions on how search behavior has changed over the 
years and whether or not people have become better searchers. 
Overall, it seems that there's still a long way to go.

Thanks for your responses.

[An aside: whenever I make up one of these summary messages, I notice 
that most people start their messages with something like "Hi Hal". I 
never do that. I wonder if it has to do with when one started using 
email, in what environment, etc. Or Maybe I'm not as friendly as the others...]

                                         -- hs

From: "McLaughlin, Gail G" <[log in to unmask]>

Hi Hal,

As a Web user since 1994 when all pages were gray with black text, 
and maybe a small square picture for visual relief, I have noticed my 
that my own search arguments are better formed to achieve optimal results

I tutor people at my local senior center on searching, among other 
things. They often have a hard time understanding what a search 
engine is, how it works, how to form a search argument, what results 
to expect, and how to choose the result that best provides the 
information they are looking for.

People have become better searchers through practice, trial and error 
and learning from others. In my experience, a new searchers tend to 
enter one word, form a question or string together a bunch of words 
(sometimes without spaces). They don't give much thought to the 
hierarchy of the words initially. I teach them to present their 
search argument as a hierarchy of words, to use Google, and click on 
"cache" in a promising result. So, if someone is looking for a good 
recipe for chicken tortilla soup, I have them enter "recipe soup 
chicken tortilla".
From: "Simon Forrest" <[log in to unmask]>


A couple of observations, with no claim for general applicability, from a
recent test session:

*The site being tested (a business information site) had a standard 
search box on each page with an option for advanced search. The 
advanced search had some checkboxes and select lists to filter the 
results. Fairly standard stuff. What surprised us is that every 
person who went to the advanced search page and picked one or more 
options from the filters expected it to increase the scope of the 
search, not restrict the scope of the search. Hence in each case the 
person was surprised that after they asked it to bring back more 
content (in their view of the way the advanced search worked), they 
actually got back less content than before. From their use of search 
throughout the test, these people appeared to be less sophisticated 
searchers - the more sophisticated searchers ignored the advanced 
search and simply rephrased their query if they didn't get the 
answers they expected.

* One person, when presented with the test script setting a task, 
typed the entire paragraph from the test script into the search box. 
When this returned no search results, this person also tried the 
advanced search filters to broaden rather than narrow the scope of the search.

The second example is no doubt rather extreme and unusual. However, 
we were genuinely surprised that a) everyone who used the advanced 
search misunderstood the way it worked, and b) those with the less 
successful search strategies were the ones who turned to advanced search.

Among other changes we made as a result of this session was to improve the
general search capability (eg adding a best bets feature) and to remove the
advanced search as it appeared to be consistently confusing and problematic.

Simon Forrest
[log in to unmask]


From: Hagan Rivers <[log in to unmask]>

Hal --

Jared specifically studied this in a series of usability tests he did 
a few years ago. I don't know if the data still holds, but I can't 
imagine why it would have changed. Might be worth a phone call or 
email to him or Christine Perfetti (on his staff) to find out. 
Anyway, here's the article:

- Hagan


From: Jeff Johnson <[log in to unmask]>


The base of Internet users is still expanding rapidly 
( The number of Americans who have 
Internet access at home is still rising, as is the number who are 
switching from dial-up to broadband.

Anecdotally, I can think of several relatives and friends who are 
just starting to use the Internet now.

Thus, at any point in time, a large number of people are just 
beginning to use search engines.

So assuming that Internet users are becoming more sophisticated 
searchers is like saying teenagers are becoming more sophisticated 
readers:  there aremillions of new ones every week.

Jeff Johnson
UI Wizards, Inc.
Product Usability Consulting


From: "Claudia Alden Case" <[log in to unmask]>


What the person might be talking about is Ask Jeeves, which I know 
was the first search engine my kids used because that was what the 
teachers (about 6 years ago) were teaching kids to use at 
school.  Ask Jeeves, which is still around as 
( allows users to type in an question and the 
results page will be sites that should have answers to the question. 
It was very useful for people who weren't familiar with the idea of 
search engines.

There are still some Help functions around that tell users to enter a 
question.  (Clippie, the hateful Office desktop assistant from 
Microsoft, allows that format.)  I would say that for the bulk of 
basic computer users, it might still be a viable interface 
solution.  My definition of basic computer user is someone who users 
the internet only to check mail and maybe visit a website if they 
know the URL (but they would likely not understand the term 
URL...instead they would call it a website address).  This class 
includes a large number of users who did not grow up with computers, 
especially PCs.  For that class of users, I would say "google" is not 
yet in their standard vocabulary and smart searching is not one of 
their standard skills.

That's my two cents.



From: "Caroline Jarrett" <[log in to unmask]>

Hi Hal

I've done quite a bit of search analysis for a major client over the 
last couple of years.

Consistently, one-word searches are far more common than 'proper 
sentences'. In around 20,000 search entries in a month, there were 
only about a dozen properly formed questions.

On the other hand, I've also done a lot of usability testing over the 
same time for the same client - so observed people searching as well 
as looking at the data. And what I've seen is that consistently, 
users try the shortest search they can think of, usually one or two 
words. But if that doesn't work, they often then try a longer search 
using the same words in the mixture.

What I have definitely observed is that people arrive at the 
usabiilty testing generally a lot more adept and confident on the 
web. And because search is such a key skill, I'd say that they're 
better at searching too.


Caroline Jarrett
[log in to unmask]
01525 370379


From: Avi Rappoport <[log in to unmask]>

My goodness, that's an interesting definition of "better".  In the 
site and intranet search logs I've examined people type 1 to 3 words 
in a query, and the length has been getting only fractionally longer 
over the last ten years.

In my experience, they never did type long natural-language queries, 
though librarians and IR people wished they would, to give more 
context to the query.  The examples they'd use would be "size of 
George Washington's false teeth"  "What George Washington's were made 
of", or "what happened to George Washington's teeth?"  All provide 
much more for an engine to work on.  Sadly, people simply don't do 
that, and never have.  AskJeeves made us think that people had, 
because they offered a list of possibly-related questions to choose, 
and then implied that those were the questions typed.

Leaving out articles and conjunctions, only makes search *better* if 
you are using a system that matches every query term (Boolean OR, 
recall-heavy) and doesn't do stopwords by default.  Google and most 
other large systems limit document matches to those which contain all 
the query terms (Boolean AND, more precise).  So the issue of 
articles is less important.

I have found users reformulating queries, fixing typing and spelling 
mistakes, and otherwise attempting to improve their searches, but I'm 
not sure I see specific improvements over time.

Hope that helps,

Avi Rappoport
Enterprise Search Analyst

From: "Claus Zimmermann" <[log in to unmask]>

Its hard to say,

Perhaps its a true statement, and a learned skill. Do a search, see 
the results and then start getting a feel how to ask questions. 
Socially we learn to with seeing how others search intelligently and 
productively and then we try to mimic that.

Good products also help, like google, that prompt and try to give 
good quality results. Its difficult when you ask a question and get 
the whole of the internet returned to you like some of the older 
search engines. Its probably a technological growth, as well as a 
testament to usability itself.


From: Skot Nelson <[log in to unmask]>

I'd argue that this doesn't mean people have become better searchers, 
but rather that search has failed to evolve.

Isn't the goal (ultimately) to ensure that people don't have to lower 
themselves to the intellectual capacity of computers?


From: "Mary Deaton" <[log in to unmask]>
To: "Hal Shubin" <[log in to unmask]>


The Pew Charitable Trust project on the Internet did a search study 
about two years ago that looked in great detail at how people search.

You can find the report at 

They have a variety of other reports that might also help you explore 
this issue.

    Tip of the Day: Quote only what you need from earlier postings
     CHI-WEB: POSTINGS: mailto:[log in to unmask]
              MODERATORS: mailto:[log in to unmask]