30 October 2007

Query the obscure

Be the first to share

Query the obscure
Gaining a better understanding of queries is a top priority of the search industry.

When it comes to common searches that repeat millions of times like “Britney Spears” or “Hybrid Cars,” returning the most appropriate results, or advertisements, is not difficult. But what about queries that are exceptionally rare and may never repeat more than a single time? Clearly, these queries are infinitely harder for the search engine to understand.

Andrei Broder, Yahoo! Research Fellow and Vice President of Search Technology and Computational Advertising, and a team of Yahoo! researchers set out to tackle this problem. Their work is outlined in a paper called Robust Classification of Rare Queries Using Web Knowledge, that appeared in SIGIR 2007.

To address the problem, the Yahoo! team proposed a methodology for using search results, as well as information available on the Web, as a source of external knowledge. To this end, they sent rare queries to a search engine and assumed that a majority of the highest-ranking search results were relevant to the query. Categorizing these results allowed the team to classify the original query with high accuracy.

The results definitively confirmed that using the Web as a repository of world knowledge contributes valuable information about the query, and aids in its correct classification. “We discovered the best source of information to understand what these rare queries are about is to look at the search results,” Broder explains. “If you look at each returned page as a vote on what the query is about, you find that the majority tends to be correct even though many individual pages are wrong.”

Read full story
Download paper

Be the first to share
17 May 2015
Design thinking without deep analysis is reckless. A more balanced approach is needed.
Larry Keeley, co-founder of innovation firm Doblin which is now a unit of Deloitte Consulting LLP, says that too often advocates of “design” regard it as an elixir that can somehow transform conservative companies into …
13 May 2015
We’re more than mere consumers, and business should remember that
Companies collect an ever-growing amount of information on customers, but forgetting the individuals behind big data is detrimental, not least to business, writes author and speaker John C. Havens. Today’s customers, empowered by mobile technologies and …
13 May 2015
Myth of globalization in consumer tech
In nearly every category Ben Bajarin (Principal Analyst, Creative Strategies) studies, he sees the regionalization of consumer tech, not its globalization. This view of the market is showing how regional technology players in many segments …
15 April 2015
The woes of a corporate anthropologist [Novel]
Satin Island: A novel by Tom McCarthy Knopf, 2015 From the author of Remainder and C (short-listed for the Man Booker Prize), and a winner of the Windham-Campbell Literature Prize, comes Satin Island, an unnerving novel that promises …
10 April 2015
We are citizens, not mere physical masses of data for harvesting
The deal we have struck with the information society over the extent to which our lives are shaped and our privacy invaded requires urgent renegotiation, argues law professor Julie E Cohen at the annual Law …
3 April 2015
Automotive sales could grow 24% if retail experience improved
New ethnographic and quantitative research from DrivingSales identifies how the growing gap between consumer expectations and the current automotive buying process is suppressing car sales volume. Automotive sales could grow up to 24% if the retail …
30 March 2015
MIT Technology Review special report on persuasive technology
The MIT Technology Review has just published a special business report on persuasive technology, i.e. how technologies from smartphones to social media are used to influence our tastes, behavior, and even habits. Free registration is …
27 March 2015
Sharp methodological critiques on current Big Data practices
Two methodological critiques on Big Data that caught our attention: In the Financial Times, economist and journalist Tim Harford points out that sampling bias and statistical errors are, if anything, magnified in Big Data research, …

We are an international experience design consultancy helping companies and organisations to innovate their products, services and processes by putting people and their experiences first.

5 May 2015
Experientia designer Dohun YuLuck Jang 유록 in Design 4 Disaster

Design 4 Disaster features an engaging illustrated safety manual for ship passengers, a personal project by Experientia designer Dohun YuLuck Jang 유록. After the Korean ferry accident last year, Yuluck (who is Korean) wanted to find a way to make safety manuals more interesting to read. He spent one year designing an interactive safety guide […]

16 March 2015
Better Health and Wellbeing: Giving the elderly in Singapore sparkling golden years

Invitation: sharing session, Singapore, 30 March 2015   What are the hopes and fears of the elderly in Singapore? How can designers offer solutions that support the elderly in managing their health and wellness? What can healthcare professionals do to help them keep active? What role can technology play in the elderly’s daily lives? Design consultants […]

1 January 2015
Happy Playful New Year
21 December 2014
Experientia’s Twitter feed live

Experientia has now its own Twitter feed. Four months of Putting People First posts and other links have already been uploaded. If you followed Experientia on Twitter through the feed of its CEO, Mark Vanderbeeken, make sure to now also follow the company (but don’t unfollow Mark, who will keep on tweeting away). And while […]

19 December 2014
Putting People First blog redesigned

Experientia’s Putting People First blog has been redesigned. It is now entirely responsive, allows for easier browsing, searching, and filtering, and features larger images on the posts. The entire history of posts remains accessible as before. We are still tweaking things and welcome any feedback.

27 November 2014
Why the world needs anthropologists – an update

Why the world needs anthropologists – Coming out of the ivory tower Location: Padua, Italy, Centro Culturale Altinate/San Gaetano Date and time: Friday, 5 December 2014, 13:00 – 18:00 Padua, Italy, 5 December 2014 – The second edition of the international symposium of applied anthropologists attempts to erase the boundary between ‘pure’ and ‘applied’ anthropology, […]

See all articles