Bing vs Google: A Usability Face-Off

By / Sep 30, 2010 / Tips
shares

We all know Google and Bing, and probably use one or both of them on more or less a daily basis. Search results are generally the defining factor that makes us have a good experience, and I suspect most of us don’t give too much consideration to the usability of the services.

However, with hundreds of millions of people using both of the services each and every day, even subtle changes in the usability of the two could add or shave off huge amounts of time for the two, and each small change can lead to a more enjoyable, more usable experience.

Today we will take a look at these two leviathans and see how they compare in terms of usability, and explain just what usability should mean to you. Want to learn how to improve conversions, get more subscribers, happier clients and better designs? Read on to find out how:

The testing process:

I have set up a quick test based on screenshots of both Google and Bing. There are just 7 questions in the test, and when you take it you will be directed to either the ‘Google’ channel, or the ‘Bing’ channel. This is a random process, and roughly 50% of people will be directed to each channel – this is to ensure a fair spread of users, habits and behaviours across the test.


To complete the test, you simply click on the location you think you would click to complete the assigned task. For example, if you were asked where do you enter your search term you would probably click in the search box; the testing application will generate a heatmap of where everyone has clicked, and you will have a quick visual guide of where people have been clicking. At the same time it will also record your average time to complete the task – obviously there is a big difference in completing a test in 5 seconds versus 10 seconds, and the results will reflect that.

You can find the test at http://video.intuitionhq.com/bing-vs-google – as I said, there are just 7 questions, and it should take you 1-2 minutes to complete.

So, what does all this mean?

Having taken the test, we then need consider the questions, and the answers that we get from them. The questions were:

1) Where would you click to enter a search query?

- This is mainly a warm up question, and I suspect most people will be familiar enough with both of the interfaces that there should be very little difference in this task.

2) Where would you search for maps?

- The aim of this question is to see if people find the navigation menus are logically laid out. Some people have commented to me that they found the Bing navigation hard to read, while others have said they didn’t notice the Google navigation. This question should show up these issues.

3) How would you go to the New Zealand site?

- Local search is a huge earner in search, and getting to these local search sites is relatively important for both Google and Bing. I know which one is more obvious to me, but I’m curious to see what the results show.

4) Supposing you had email with this provider, where would you go to check it?

- Integrating services is another important point for both Google and Bing – they would like us to be tied into their who ecosystem. Email as a tool we use from day to day is a big part of this.

5) What part of the interface most attracts your attention?

- For both Bing and Google, search is the key service, and as such the search box should be the primary focus for both. These results should show us if it holds true for these providers.

6) How would you advertise with this provider?

- Advertising revenue is key to both of these businesses, so its key for advertisers to be able to find further information about advertising. Which one do you think will be better?

7) Where would you sign in?

- You can register for an account with both services, and logging in can customize your experience. Users who sign in are generally more committed to a service as there is a higher switching cost when they leave. The easier it is to sign in, the more likely people will do it.

Hopefully you can see why we asked these questions, and have some idea of the value of them. In our time testing we have seen people set up a wide range of different tests with different goals, and different information that they are trying to capture – just to give you some ideas:

  • Where would you click to…
  • Which layout is the most visually appealing
  • Asking questions and giving a range of answers to choose from
  • Comparing wireframes and current designs
  • Testing layout sketches
  • A/B testing on navigation layouts and wording

There are of course many more things you can do with usability testing than this, but hopefully this gives you a few ideas to get started.

The Results:

Now to have a quick look at the results. You can view them at http://video.intuitionhq.com/pub/405.

As more people take the test the results will be updated, and you might notice some changes, but the general results will be more or less the same.

1) Where would you click to enter a search query?


Google is quicker, but the clicks aren’t always in the right place. A win to Bing?

Bing 1 – Google 0

2) Where would you search for maps?


More or less a dead heat here. Both are very good.

Bing 2 – Google 1

3) How would you go to the New Zealand site?


Bing is ahead here – 7.29 seconds vs 9.2 for Google. 2 seconds is a big difference.

Bing 3 – Google 1

4) Supposing you had email with this provider, where would you go to check it?


Google with a clear lead – 4.22 seconds difference between the two.

Bing 3 – Google 2

5) What part of the interface most attracts your attention?


Seems Bings crop circles are very popular. Google is more focused on this one.

Bing 3 – Google 3

6) How would you advertise with this provider?


Google is faster by 1 second, similar success rates.

Bing 3 – Google 4

7) Where would you sign in?


Stunning results for both (less than 4 seconds), Bing is marginally ahead but too close to call.

Bing 3 – Google 4

So based on the current results it’s really a very near run thing. I’m quite surprised to see this actually, evidently Bing’s UI is easier to navigate than I might have expected.

I think a lot of us have learnt behaviors for Google based on years of experience, so Bing is really doing well to stay up there with Google, especially considering the two UIs are quite far apart.

Both services could obviously make a few small tweaks to their UIs, but as you’d expect with two such popular websites, they are already very streamlined.

As more people take the test, we can expect to see results to evolve somewhat. Be sure to check back at http://video.intuitionhq.com/pub/405 to see how they change.

What Next?

Of course, usability testing doesn’t need to be solely the domain of large companies like Microsoft and Google, we could all do with making some tweaks to our own sites. Each improvement we make leads to people having a better, more enjoyable experience, and in turn means they are far more likely to come back to our site in the future, to subscribe or make purchase from our site or recommend it to others in the future.

As website designers, programmers and owners, these are the outcomes we are aiming for, and usability testing and usable design will help us get there.

So why not start out by testing your own site today, and working with your clients to improve their sites in the future. When we all focus on usability we can make the whole web a friendler, more enjoyable place to be.

Any questions about usability? What to know how to get started? Or comments on the article? Other sites you’d like us to compare? Please let us know in the comments! We’d love to share our views on usability.

About the Author

Jacob Creech is a usability geek at IntuitionHQ.com, a quick, easy usability service by designers for designers. He can be found tweeting about usability, UX and design @IntuitionHQ. When not entertaining himself on the internet, he's probably studying Chinese, having spent three years living in China. He is also partial to all kinds of tea.

23 Comments

  1. Ferdy
    September 30, 2010

    I disagree that their UIs are far apart. Both are almost empty pages with a single search text box in it. Both have the same sign in UI, and both have domain-specific searches solved the same way. Bing shows a background image but Google Search has this too, as an option. I actually think that Bing’s UI is so similar to Google because it just works.

    Reply
  2. inpixelitrust
    September 30, 2010

    Very interesting results.
    I took the test with the google search, as an “only google” user I don’t know if my results are therefore relevant : I know the UI, so I can go quicker with it. However, I don’t think I would have performed the task so quickly if I would have had the bing questions.
    Maybe it would be interresting to know, in the end, if the person who took the test is a total beginner (as I would be on bing), or a more advanced one ? Just an idea ^^

    Reply
  3. Free
    September 30, 2010

    Its kinda funny that people enter everything they’re looking for in the search input. I personnaly know few people, who enter some websites through typing it’s whole address into google, and then clicking on the first result :p

    Reply
  4. Kyle
    September 30, 2010

    I wonder how relevant these results are based on the users involved in the study? There is no mention of the participants exposure to each of the search engines. Also I question your methods of switching between time take, and clustering. Its a common problem with people using an eye tracker to evaluate systems selecting a view or result that matches there claims. I concerns me that this company offer their services as usability experts, and yet appear to be making a number of rookie errors. I for one wont be recommending them to anyone.

    disappointing article

    Reply
    • sapphire
      October 3, 2010

      I agree with your point of view, the results would be more meaningful if more background info of participants is provided such as the search engine experience, even the browser users usually use. Take task1 for example, I’m the one who would type search keywords in the URL column, but this doesn’t mean that Google UI isn’t intuitive, and I’m wrong, but because I’m a Chrome user, and the user experience changes my behavior to search; however, I think it’s a convenient design.

      Reply
  5. peter
    September 30, 2010

    Does it really matter…… if you use a site regularly you remember where the buttons are so maybe it take a few seconds more to to get started, but over the lifetime use of the project it is irrelevant. I don’t use this site (webdesignledger) very often so it will take me longer to navigate if I look for something, that doesn’t mean I shouldn’t use it, it just means it’s individual.

    Reply
  6. Web Designer Leicester
    September 30, 2010

    A nice comprehensive article, one interesting thing to me is that for particular phrases, both search engines bring up totally different results on page 1. So i guess we could judge them on their sole purpose and that’s the results they give us fr a search.

    Reply
  7. Craig Pennings
    September 30, 2010

    Interesting post… kind of humorous that someone would click a white background as the part that attracts their attention most.

    Reply
  8. Craig S
    September 30, 2010

    I think a couple of these questions are less significant. We’re evaluating search engine vs search engine… for me I never care how to reach email from a search task. What if we were evaluating Ask.com or Cuil? So for me, the finding email question does not play that much into search usability.

    Next: On the question of what attracts attention, Bing draws some attention to the background… which is by design. The background has the 4 clickable spots that also bring you to a search (even if it’s not the one the user originally approached the computer for). If you argue that the design is a detriment and is hindering the user’s ability to search, remember the first most important question was ‘where do you click to search’? You said Bing won that one, remember? So even though there are other attractive elements on the page while Google only has one, this design doesn’t prevent people from searching on Bing.

    Reply
  9. DesignMango
    September 30, 2010

    I personally prefer Google, much cleaner

    Reply
    • Jay
      October 3, 2010

      Except that Google keeps f’ing up their site by introducing new features that are more bing-ish.

      Reply
  10. Louise KR
    October 1, 2010

    You mentioned how for a new service, Bing is doing pretty well against something old like Google. I think the interface of both are somewhat similar. And because we’ve been accustomed to Google for years already, we find Bing’s familiar and easy to navigate as well. Google has helped Bing become an easier experience.

    Also, with the first question’s result, I think it’s funny how most people mis-clicked on the very obvious search box at the very enter of an otherwise clear-spaced page of Google. There might be other reasons behind this except the notion that ‘they don’t know where to search’ because that is pretty stupid on their part and I believe for people knowing Google for ages, they are not ‘stupid’.

    Reply
  11. mathmos
    October 1, 2010

    To be honest, I think this is a poor article on flawed science. Poor offering.

    Reply
  12. Chris
    October 2, 2010

    I think the heatmap results have more to do with the users mouse habits than anything.

    For instance, when I’m reading a page, I constantly click and highlight text. It’s just what I do.

    While taking the test, I was aware of my mouse habits because I knew that clicking would record. But I felt like clicking.

    Overall, I don’t think this test has much merit because everyone is different.

    Reply
  13. Maksim Shaihalov
    October 3, 2010

    Good example. Thx!

    Reply
  14. Craig`
    October 3, 2010

    Interesting article, and agree with some of the comments about being familiar with the search providers UI, that you can find the informatiuon quicker and as mentioned, the most important factor is relevance of the SERP’s.

    Reply
  15. Jay
    October 3, 2010

    On another note, why would someone go to a search engine to check their email? That makes no sense. If you already have an account with one of these providers then you already know where to go to check your email. If you don’t have an account, then you probably don’t even know they offer that service.

    Reply
  16. Jermaine Young
    October 4, 2010

    Very interesting test. I whole heartedly support Google and was surprised to see how closely the two scored. I think Google’s advantage is exactly what you stated for on and also the fact that Google is nearly synonymous with search in the first place. Although Bing is quite a bit more aesthetically pleasing Google’s integrated services are the main reason that it is still my primary browser of choice.

    Reply
  17. jsa
    October 4, 2010

    based on the results I saw, Google was the more usable site (quicker response times), so… I’m not sure how you came to those conclusions.

    Reply
  18. Juegos
    October 5, 2010

    Interesting but I prefer Google as well, It’s better than Bing from all points of view.

    Reply
  19. Chris
    October 7, 2010

    There is one giant confound in this study: Google’s familiarity.

    Even though people are randomly assigned, it’s likely that overall the majority of people have more experience using Google than Bing. Therefore we should expect to see faster results when people are tested with Google.

    This is by virtue of the fact that Google has been around for much longer than Bing and has more people using it.

    Reply
  20. Ilie Ciorba
    October 9, 2010

    You want to say that bing is almost as good as google? Google is the God of search, nothing can’t even get close to it! and Bing is a piece of sh*t!

    Reply
  21. GIK Web Design
    August 31, 2011

    I always found google to be fast and clean not bombarded with articles and latest news Im not interested in. Ive also found that I feel you get better results search results on google rather than bing, I suppose its like everything its all about personal preference though.

    Reply

Leave a Reply