Search or Experience, But Not Both

Recently, I had the idea to perform a test that truly tested the relevance of the search results themselves; on Google, Yahoo! and MSN. My thought was this; when you look at the results themselves, they are not that different, so why does Google have nearly 64% of the search market share (according to Hitwise) where MSN sits in a distant third with less than 9%?

Is there something we’re not considering here?

Does Google just make it easier to do what you need to get done?

I think the answer to both of these questions is yes. There are some things that we’re not considering here, and having a background in graphic design made it easy for me to have an affinity for what’s being missed. As Marissa Mayer will admit, Google is very focused on the end user quite passionately and I’m sure that a part of that market share can be attributed directly to that fact, and maybe more than just a part.

The gist of our study involved a single page of results on paper from each of the big three engines using the same keyphrase, in this instance we chose “digital camera”. We passed this around a small sample of people and had them choose what they felt the top 3 most relevant results were out of the entire list, not on each page.

Not only were the results surprising, but they were completely backwards from what we and every other search marketer would have expected.

When we scored the results (3 points for 1^st choice, 2 points for 2^nd choice and 1 point for 3^rd), MSN came out as having far and away the most relevant results. The margin was not just a 10% or 15% margin, MSN was more than 3 times as successful as Google in terms of its relevance. And when measured against Yahoo, MSN outperformed those results as well, doubling Yahoo’s relevance score.

It seems that Google is doing many of the other things right in order for their relevance to be perceived as well as it has been. Their Progressive Disclosure model has allowed them to keep the interface clean and their users well taken care of, as the Google tools they need only show up when they need them.

Basic design principles are adhered to on the results page and, as much as people mock the simplicity of the homepage, there are more benefits to it than looking clean and tidy. This “clean and tidy” homepage also keeps the user’s mindframe “clean and tidy” or more focused if you prefer. There are less distractions to a user’s focus when compared to the portal page for either Yahoo or MSN.

This is proven through our Eye Tracking 2 whitepaper. When you compare the condensed pattern on Google results pages against the spread out activity on the busy portal pages, it displays obvious remnants of distraction. And when you consider that short-term memory has a forward memory span of approximately seven items plus or minus two (Miller, 1956), those distractions are doing more to detrimentally affect the pure search experience by filling our little short-term memories with news, pictures, ads and information… All of which have nothing to do with what we’re searching for.

So, MSN may have the most relevant results (according to our little panel), but it turns out that it’s more important to do the rest of the search experience up right. Which begs another bit of insight; it seems that in the phrase, search experience, the word that we would expect to be the most important often isn’t.

So MSN, or Windows Live Search as you are trying to be referred to as, if you want to put a cork in the leaky dam of market share, put more experience into the search experience.