Mobile Menu AB Tested: Hamburger Not the Best Choice?

Recently I’ve been reading books about quantum physics (clearly I’ve got too much time on my hands).

I don’t profess to understand even a tenth of what I’ve read, but it’s fascinating (even if my friends shake their heads at the new levels of geekdom I’ve aspired to).

The subatomic world seems to exist in a mystical state of uncertain probabilities – UNTIL it’s measured and observed.

Exactly like user experience and usability.

We really have no idea at all how users interact with our websites.

Until we measure and observe, our assumptions are based on uncertain probabilities.

Previous Hamburger Icon Test

I did a previous test that seemed to show that a bordered “hamburger” (aka sandwich) icon was used more than other options.

hamburgertest1

The menu icon on the right was clicked more than the previous two.

I then decided to test the hamburger icon against the word “MENU”.

A/B Test Conditions

The test was run against all mobile browsers across all pages.

The duration was about 5 days, and served to around 50,000 mobile visitors.

Demographics of the site show a skewing towards the 25-34 age group.

Demographics

Demographics

Mobile visitors are split into the following:

  • iOS 64%
  • Android 34%
  • Windows Phone and Blackberry make up most of the remaining 2%.

Original (baseline)

Based on the results of my previous test, the site now has a bordered ‘hamburger’.

baseline

Variation 1 (MENU + Border)

2-variation1

Variation 2 (MENU + Hamburger + Border)

2-variation2

Variation 3 (MENU without Border)

I would never consider this implementation, but I wanted to test, and check my assumptions.

2-variation3

Results

hamburgertest2

As predicted the word “MENU” alone performed poorly (but not as badly as I presumed).

Of interest is the bordered menu was clicked on significantly more than our hamburger icon.

iOS vs Android

Another tracking metric I’ve setup is event tracking in Analytics. I record an event every time the mobile menu is clicked.

What the preliminary data is showing:

iOS users are 2-3 times more likely to tap a menu icon than Android users.

Interpretations

Based on this and my previous AB test, a flat hamburger icon may not be ideal on a responsive website (remember this is a website not an app). Using the word MENU (and making it look like a button) could be more helpful for visitors.

This does not mean that users do not understand the hamburger/sandwich – it could be that the word MENU draws more attention.

UPDATE: New research from Nielsen Norman Group shows: Users are very familiar with the magnifying glass icon for search, but “Users are still unfamiliar with newer icons, including the three-line menu icon and the map-marker icon”.

No comments yet.

Leave a Reply