Search or Experience, But Not Both

Recently, I had the idea to perform a test that truly tested the relevance of the search results themselves; on Google, Yahoo! and MSN. My thought was this; when you look at the results themselves, they are not that different, so why does Google have nearly 64% of the search market share (according to Hitwise) where MSN sits in a distant third with less than 9%?

Is there something we’re not considering here?

Does Google just make it easier to do what you need to get done?

I think the answer to both of these questions is yes. There are some things that we’re not considering here, and having a background in graphic design made it easy for me to have an affinity for what’s being missed. As Marissa Mayer will admit, Google is very focused on the end user quite passionately and I’m sure that a part of that market share can be attributed directly to that fact, and maybe more than just a part.

The gist of our study involved a single page of results on paper from each of the big three engines using the same keyphrase, in this instance we chose “digital camera”. We passed this around a small sample of people and had them choose what they felt the top 3 most relevant results were out of the entire list, not on each page.

Not only were the results surprising, but they were completely backwards from what we and every other search marketer would have expected.

When we scored the results (3 points for 1^st choice, 2 points for 2^nd choice and 1 point for 3^rd), MSN came out as having far and away the most relevant results. The margin was not just a 10% or 15% margin, MSN was more than 3 times as successful as Google in terms of its relevance. And when measured against Yahoo, MSN outperformed those results as well, doubling Yahoo’s relevance score.

It seems that Google is doing many of the other things right in order for their relevance to be perceived as well as it has been. Their Progressive Disclosure model has allowed them to keep the interface clean and their users well taken care of, as the Google tools they need only show up when they need them.

Basic design principles are adhered to on the results page and, as much as people mock the simplicity of the homepage, there are more benefits to it than looking clean and tidy. This “clean and tidy” homepage also keeps the user’s mindframe “clean and tidy” or more focused if you prefer. There are less distractions to a user’s focus when compared to the portal page for either Yahoo or MSN.

This is proven through our Eye Tracking 2 whitepaper. When you compare the condensed pattern on Google results pages against the spread out activity on the busy portal pages, it displays obvious remnants of distraction. And when you consider that short-term memory has a forward memory span of approximately seven items plus or minus two (Miller, 1956), those distractions are doing more to detrimentally affect the pure search experience by filling our little short-term memories with news, pictures, ads and information… All of which have nothing to do with what we’re searching for.

So, MSN may have the most relevant results (according to our little panel), but it turns out that it’s more important to do the rest of the search experience up right. Which begs another bit of insight; it seems that in the phrase, search experience, the word that we would expect to be the most important often isn’t.

So MSN, or Windows Live Search as you are trying to be referred to as, if you want to put a cork in the leaky dam of market share, put more experience into the search experience.

SES Chicago - Gord Hotchkiss Discusses the Eye-Tracking Study

Enquiro's CEO and President, Gord Hotchkiss, is interviewed by TopRank Blog at SES Chicago today about the new MSN, Yahoo and Google Eye-Tracking Report. In the interview Gord touches on some of the key points in the study including how the MSN and Yahoo users experience on Google appeared to be considerably more favorable than on their engine of choice. Also discusses the issue of perceived relevancy on the engines. For more information, visit this review of the report.

We're all Verklempt, Discuss Amongst Yourselves, We'll Give you a Topic... Google vs. MSN vs. Yahoo!

If you've read the original Google Eye-Tracking White Paper released by Enquiro, Eyetools and Did-It, you'll know the impact and discussion it has caused in the industry with the unveiling of the Golden Triangle and accompanying insights giving the search marketer an edge with their Google specific strategies. Quotes from this article have been picked up all over the industry, from sources such as Danny Sullivan and Anne Holland, and is used as training collateral for many organizations with an interest in online marketing.

Well, prepare yourselves because the next wave of insight is at the industry's doorstep! A few of us here at Enquiro have been working dilligently on the last few pieces of information for the new Eye-Tracking White Paper with Gord Hotchkiss at the helm, a White Paper that broadens the scope of the original by 2 engines, MSN and Yahoo! This new paper definitely offers insights for the search marketers out there by digging into topics like the the effect of sponsored listings on each engine by area, banner blindness and the growth of navigational search. But this paper also has plenty of new information that those usability and the onsite experience junkies will find very interesting. For instance, have you ever thought about the impact that the MSN or Yahoo! portal page might have on the rest of the search experience? How positive or negative news stories or images might affect the success of the searchers experience? No? How about perceived relevancy on the engines? Which engine is really the most relevant? Is Google always number one? We all have our theories, and this paper will help to quantify some of them, and generate discussion around others..

Do we have your attention? I haven't seen the final version yet, but I'm really curious! Be sure to check it out!

Download a sample of the impending report here.