30 October 2007

Query the obscure

Be the first to share

Query the obscure
Gaining a better understanding of queries is a top priority of the search industry.

When it comes to common searches that repeat millions of times like “Britney Spears” or “Hybrid Cars,” returning the most appropriate results, or advertisements, is not difficult. But what about queries that are exceptionally rare and may never repeat more than a single time? Clearly, these queries are infinitely harder for the search engine to understand.

Andrei Broder, Yahoo! Research Fellow and Vice President of Search Technology and Computational Advertising, and a team of Yahoo! researchers set out to tackle this problem. Their work is outlined in a paper called Robust Classification of Rare Queries Using Web Knowledge, that appeared in SIGIR 2007.

To address the problem, the Yahoo! team proposed a methodology for using search results, as well as information available on the Web, as a source of external knowledge. To this end, they sent rare queries to a search engine and assumed that a majority of the highest-ranking search results were relevant to the query. Categorizing these results allowed the team to classify the original query with high accuracy.

The results definitively confirmed that using the Web as a repository of world knowledge contributes valuable information about the query, and aids in its correct classification. “We discovered the best source of information to understand what these rare queries are about is to look at the search results,” Broder explains. “If you look at each returned page as a vote on what the query is about, you find that the majority tends to be correct even though many individual pages are wrong.”

- Read full story
Download paper

Be the first to share
30 March 2015
MIT Technology Review special report on persuasive technology
The MIT Technology Review has just published a special business report on persuasive technology, i.e. how technologies from smartphones to social media are used to influence our tastes, behavior, and even habits. Free registration is …
27 March 2015
Sharp methodological critiques on current Big Data practices
Two methodological critiques on Big Data that caught our attention: In the Financial Times, economist and journalist Tim Harford points out that sampling bias and statistical errors are, if anything, magnified in Big Data research, …
24 March 2015
Ethnographic research drives IKEA’s global success
In a long Fortune Magazine article on IKEA's successful global expansion, author Beth Kowitt devotes quite a few paragraphs to the importance of qualitative, ethnographic research: "Today research is at the heart of Ikea’s expansion. “The …
11 March 2015
From design thinking to ethnographic thinking
In the broader business world, writes social anthropologist Jay Hasbrouck, design thinking "is already a stale buzzword — a concept that spread widely as a panacea for dull and slow-moving corporate cultures." "Some say it’s now …
10 March 2015
Empathy makes you only more egocentric
In a battery of experiments, Imperial College’s Johannes Hattula and his coresearchers Walter Herzog, Darren Dahl, and Sven Reinecke interviewed marketing managers about their personal preferences for a selected product or service. The researchers then …
2 March 2015
Human-centered design should be a CMO’s best friend
The average tenure of a CMO is just 45 months, according to a recent study released by executive recruiting firm Spencer Stuart. Fortunately, for CMOs in need of help, there's an elegant and simplistic process called …
2 March 2015
Ethnographic research increasingly informing business strategy
Corporate ethnography isn’t just for innovation anymore. It’s central to gaining a full understanding of your customers and the business itself, writes Ken Anderson, anthropologist at Intel Research, in the Harvard Business Review. The ethnographic …
31 January 2015
The rise of the Chief Behavioral Officer
After the Chief Experience Officer, the new trendy job title from corporate America is the Chief Behavioral Officer: "Dan Egan of the automated investing service Betterment estimates that between 10 and 20 Fortune 500 companies have …

We are an international experience design consultancy helping companies and organisations to innovate their products, services and processes by putting people and their experiences first.

16 March 2015
Better Health and Wellbeing: Giving the elderly in Singapore sparkling golden years

Invitation: sharing session, Singapore, 30 March 2015   What are the hopes and fears of the elderly in Singapore? How can designers offer solutions that support the elderly in managing their health and wellness? What can healthcare professionals do to help them keep active? What role can technology play in the elderly’s daily lives? Design consultants […]

1 January 2015
Happy Playful New Year
21 December 2014
Experientia’s Twitter feed live

Experientia has now its own Twitter feed. Four months of Putting People First posts and other links have already been uploaded. If you followed Experientia on Twitter through the feed of its CEO, Mark Vanderbeeken, make sure to now also follow the company (but don’t unfollow Mark, who will keep on tweeting away). And while […]

19 December 2014
Putting People First blog redesigned

Experientia’s Putting People First blog has been redesigned. It is now entirely responsive, allows for easier browsing, searching, and filtering, and features larger images on the posts. The entire history of posts remains accessible as before. We are still tweaking things and welcome any feedback.

27 November 2014
Why the world needs anthropologists – an update

Why the world needs anthropologists – Coming out of the ivory tower Location: Padua, Italy, Centro Culturale Altinate/San Gaetano Date and time: Friday, 5 December 2014, 13:00 – 18:00 Padua, Italy, 5 December 2014 – The second edition of the international symposium of applied anthropologists attempts to erase the boundary between ‘pure’ and ‘applied’ anthropology, […]

30 October 2014
The BancoSmart ATM by Experientia for UniCredit selected for ADI Design Index

Last year Experientia designed the interface of an ATM of UniCredit, a major Italian bank. The interface is now rolled out across the bank’s ATMs in Italy, to great satisfaction of the bank and the customers alike, since interaction speed is much faster and error rates went down dramatically. Last year UniCredit and Experientia also […]

See all articles