FEATURE
Interview With Stefan Weitz
Putting the Bling Into Bing
by Barbara Quint
In setting up my interview with Stefan Weitz, a director on the Microsoft Search team developing Bing, I forwarded him a copy of the UpFront With Barbara Quint column ("You Say You Want A Decision Support Engine?") in the September 2009 IT that found Bing somewhat wanting.
Of the two searches covered in the column, one involved a food poisoning scare with Nestlé’s refrigerated Toll House cookie dough; the second involved finding a restaurant in Paris with a name ending "d’Argent" that had a famous pressed-duck dish on its menu. In the first case, no news items appeared in Bing’s search results to alert a person to the food poisoning risk, while a Google search linked to hundreds of news stories. In the second case, Bing found the "Tour d’Argent," but Google not only found it, it also provided an address, phone number, a map, directions, a menu, and restaurant reviews.
I knew Weitz would make a good interview when I received his email in response:
"A couple of things … on the Nestlé query, the length appears to be the culprit. If you enter ‘nestle toll house’ we’ll actually fire a news result (albeit at the bottom—I don’t like that treatment and I know we are going to change).
"On the Paris one, they just beat us."
"Great feedback, tho—these are the types of queries that can help us generalize parsing or index issues and fix [them] at a more systemic level. I’ve sent these across to engineering."
"Nice to e-meet you!"
Just before the scheduled interview took place, Microsoft announced that Bing would take over Yahoo!’s searching "Powered by Bing." However, Weitz could not provide any comments on that action. The interview has been edited for format and length.
Q: What are the differences between Bing and the previous Microsoft Live Search?
A: They’re pretty significant. With Bing, we looked at what customers are actually doing when they search. We’re No. 3 in a three-horse race. People didn’t know our brand, not nearly enough to try and adopt it. So we looked at what searchers were doing and saw the issues people were having. The first is that they were only successful a quarter of the time. That is staggering when you look at the high satisfaction, when you look at the data click algorithmic link, and they don’t come back. They may have gone off to Google. What happened to the query page is that a quarter of the time they were back in 30 seconds. That means they probably did the query, clicked on a result, realized it wasn’t what they wanted, and used the back button. The core issue is relevance. We didn’t have enough information on the results page, so it was easier for them to go after the query again.
The other two things we found out were that people do not necessarily have distinct queries when they navigate to a page or site. In the past few years, we’ve been getting longer and longer chains of queries, multiquery sessions. It’s fascinating to look at the time spent. Almost half the time spent online actually involves these long chains. The last conclusion is that more people are adding more tasks on search engines to make complex decisions that the current search engines were never designed for. With Bing, we want to first make clear which links for standard core searches are the good ones. We want better relevancy. We use the "hover" preview for results. Where we’re better is in organizing the results and then adapting a user interface that depends on the task the user is performing—different grammars.
Q: Does that mean you’re focusing on shopping areas?
A: I wouldn’t call us a commercial site. We looked at the top task areas and their low success. Finding local businesses, comparison shopping, and travel are just some of the focuses. Bing structures its results. Search on the left-hand side in search results to find all kinds of filters.
Q: I had some problems with a search that didn’t find the latest information. How do you handle currency issues and crawling rates?
A: Basically, the crawlers go over the web following links and attempting to index. Certain domains change rapidly, like news sites or real-time feeds, etc. For that, we would have superfresh faster crawlers, in particular. We don’t accept requests. We can tell how often sites change. It depends on how frequently we need to access the site. In the case of your looking for news on the cookie dough tainting, on the query parsing side, we happen to have a query spike mechanism. This is a way to watch what’s coming and the classes of suddenly higher concentration over a period of time. We can see if people are querying for cookie dough and systematically or manually re-jigger the results page for the query. We can fire up a different type of result, for example, news. All the search engines do it differently. Out of 10 or 20 queries, we’ll get some right and some wrong. We have to catch it.
Q: Is it different for different silos of content, such as news?
A: Yes, the news section of Bing is a different user experience. The news section is probably more dynamic with frequency spiking. If you’d clicked on indexing for news sites, you’d have had the story.
Q: If Bing is being tweaked continually, how does the news of changes reach the average user?
A: The major tweaks we promote let folks know that they’re there. Bing has ramped up its experimentation significantly. It enables us to iterate, to have an idea on Monday, to do a quick specification on Tuesday, cost it out, then to develop a question and answer, and to put it into Flight a week later. A Flight involves 1% of our traffic used for treatment and measure and control. It takes a lot of the emotions and "I think" out of the equation. We can put a notion into play and see how users use it and then maybe move into real production. We Flight everything. Every change first goes to Flight internally on the corporate network and then to external Flight-ing on a percentage of the traffic. When it comes to consumers and major changes, we do multiple Flights, named Super-Flights.
Q: In my September column, I suggested that Microsoft might want to focus on bringing together a portal of all the best vertical search engines. I have to ask you: Is it realistic?
A: Well, we can do it today—in a way we do vertical indexes with health.
Q: Well, if that idea didn’t work, how about a "Tough Times" section with information on loans, grants, childcare, jobs, welfare, refinancing mortgages, starting small businesses, etc.?
A: Hmm. That sounds interesting.
Q: By the way, along the same lines, could you mark your ads for any discounts/price cuts/special prices and so forth? A kind of deep-link advertising?
A: In the shopping vertical space, Microsoft offers a cash-back service. You do a search, for example, for a Nikon D690, and the results look like a traditional product search. But there’s a different column called cashback that offers a percentage of the purchase price as a credit for PayPal. The credit will apply to purchases from thousands of different vendors.
Q: There’s one thing I’ve been yelling about at the information industry for at least 2 decades. Every search service should provide a feedback form with every set of search results: "Was it good for you?" "Did you find what you wanted?" "What would have been better?" and, of course, "Do you want us to keep looking and send you whatever we find?"
A: The bottom line on a local search does have a "Help Us Improve" form.
[For more on Bing, turn to the Database Review on page 44. —Ed.]
|
