More conversions, more sales, guaranteed!

Split Testing, also called A/B testing, is an extraordinarily powerful way to find out what works and what doesn’t, or what works better in website marketing.  Surprisingly, it’s very much under-utilized by marketers.

I’ve written about this before at Split Testing/Sex (how can you possibly resist clicking on that?!) but it’s an important topic, so I thought I would take a little deeper look at it . . .

In the olden days before the Internet, a split test could take weeks, or months.

For example, you mailed a solicitation with one headline to 10% of your mailing list, and the same solicitation, just with a different headline, to another 10%. You see which one works better, and use it for the remaining 80% of your list.

This worked just fine, but you had to wait for the printing and mailing and responses, all of which could take a long time.

Because of the time lag, it was hard to react and make timely changes.

Today, we can know the results of a change in headline or other component of an offer in just hours or days.

So why is split testing underutilized in website marketing, when it can lead to such dramatic increases in whatever you’re measuring, whether it’s clicks, conversions, or profits?

Two reasons, I think.

First, although it’s simple in concept, it still isn’t well-understood by a lot of marketers.

Second, people are afraid that it’s too complex to implement.

Taking the second point first, it’s really not that difficult. There are plenty of tools for doing it, including some free ones, and the free “Google Analytics” tools work just fine for most people and most tests.

Becoming a master of Google Analytics isn’t easy, but almost anyone can plow his or her way through enough of it to do some split testing, and if you can’t, there are plenty of people on Elance or Vworker who can do it for you, very inexpensively.

Also, there’s a WordPress Plug-in called “Premise” that makes running split tests easy.  This solution costs a little more, whereas Google Analytics is free.  But we use Premise and the cost is nothing compared to the increased profits you can get if you just do it.  If you’re like me, the easier and quicker you can make this kind of stuff, the more likely it is that you’ll do it.

So How Does Split Testing Work, Anyway?

How does a split test lead to a 300% increase in conversions? I’ll talk more about this particular experiment later, but the quick version is that changing the position of the opt-in box made the difference.

I guess I should point out that you don’t often get huge performance improvements like that with a single change, as wonderful as that would be.

But you don’t need that kind of huge improvement to make a real difference.

Most people don’t realize that every time you get an increase in clicks or conversions, that increase compounds on the increases you got earlier on.

If you get a 17% increase with a better-performing headline, and then you find that adding a graphic gives you a 12% bump, and then you try a different “buy now” button which improves conversions 11%, and then you simplify your order form for a 9% increase, all of a sudden, your conversions are 58 ½% better than they were. (100 x 1.17 x 1.12 x 1.11 x 1.09)

Pretty soon, you’re talking real money, as they say—and these are the kinds of improvements anyone can get by split testing.

What can you split test?

Anything. Here are just a few things you might test:

The wording of your Headline

The color of your headline

The font of your headline. And the size.

The size, color, design, placement and wording of your opt-in.

The size, position, color, shape and wording of your order button.

Your price.

Your images

Your trust seals.

All the elements of your AdWords ads that drive traffic to your site

The Subject Lines of your emails

The truth is that you should never stop split testing. You should always be testing some element against a different version, trying to find a version that improves results. That becomes your new standard, or “Control,” and then you test a new version against that one. And so on.

So Where Do You Start?

Before you start testing single elements, you work on the page as a whole. Try several completely different, or at least moderately different, layouts of your page and find one that works well. Then start on the pieces.

Of course you start on your headline first, since that’s the place where viewers will bounce away, because you didn’t capture their attention, or they’ll read further because you did.

One thing that’s important to remember is that you should only change one thing at a time, or you won’t know which change made the difference.  (Yes, you Taguchi fans, I know…but really?  I do this for a living and I don’t even have time to implement it!)

Another thing to remember is that you will be fooled. Often, the version that you just know is superior will get trounced by the one you run against it, sometimes against all logic. Happens to us all the time.

When Do You Know You Have a Winner?

This may be another area that stops people from doing split testing—knowing when you have a winner delves a little bit into the mysterious area of statistics, which is pretty scary for lots of people.

I’ll explain a little, then give you some shortcuts that make it all much easier.

For example:

Your control page gets 974 visitors and 5 conversions. Your “challenge” page gets 961 visits and 7 conversions. The challenge page is clearly better, right? Seven is 40% better than five.

Nope. Statistically, according to one writer, there is only a 45% chance that the challenge page will actually perform better over time. Its apparent superiority at this point is 55% likely to be just due to chance.

Without getting deep into the statistics of it all, the point is that you don’t have enough data yet to make a decision.

One “rule” is that you need at least 30 actions before you can be reasonably confident that the long-term results will be about the same as what you see after just the first 30 actions.

That’s a little loosey-goosey, but useful as a first approximation if you want to keep it simple.

In the example above, if you had 30 conversions and 42 conversions, you could be 89% certain that the challenge page would be the long term winner. Most experts think you shouldn’t name a winner until you’re 95% confident, so we’re still not quite there.

So when do you know—without cracking open your high school statistics textbook or calling for help?

Here’s the easy way:

There’s a free A/B Test Calculator at www.splittestcalculator.com. It uses the two-sided Chi-Square statistic with Yates Correction.

I have no idea what that means , but the calculator is easy to understand and use— in our example above, you would just enter 974 and 5, and 961 and 7. You’re told that your confidence level is 62.77%– a long ways from 95%. [I'm not a statistics geek, and I don't know why we get 62.77% confidence with this tool, and the writer above says the results are 55% likely to be due to chance alone. Whatever, both these numbers are indicating that with only this much information, it's about a crapshoot which version is better.]

In this case, you wouldn’t be 95% confident until your challenger had 62 conversions and your control had 44 conversions. Sixty-two is the same 40% better than forty-four, but now you have enough data that the results are statistically significant.

There’s a lot more to know about split testing, but now you know how to measure your results, and how to assess whether the difference is meaningful.

Back to our 300% headline.

This one was a little counter-intuitive.

One of the “rules” of web design is that you have your opt-in box “above the fold” so that it’s right there, and your viewer doesn’t have to scroll down to see it.

However, in this particular case, with a rather complex landing page, moving the opt-in well down the page resulted in a 309% improvement in conversions. The supposition was that moving it down gave people more of a chance to get the information they needed before they were asked to make a decision.

Interesting.  And that’s why we test, right?

What can you test – today?

Scott

 

 

 

 

{ 0 comments }