2 by 2 by 2 or A vrs B

2 by 2 by 2 or A vrs B was Originally Posted on March 3, 2013 by lavarock

A recent article in Wired Magazine explains the A/B test and how it is used. Google is used as an example.

Sometimes we present ourselves with a myriad of decisions and then not take any action. Those decisions are just too overwhelming to get a grip on. The article starts out explaining how you can create a perfect website. Rather than rely upon a web designer to figure it all out, let your visitors design it. HUH? Well Google has a lab function that allows you to evaluate multiple versions of ads (or webpages).. You build a version of a page, then make a small change, such as move the graphic above the text on the new page. Then send some of your visitors to one page and the rest to the other. Track which page gets a better readership (more viewing minutes or more sales) and use that page and drop the other. Then copy that page and make a small change (perhaps two column instead of one) and if the two column does better, keep that and drop the other.

Silly you think, that by changing these items can make a difference? Ask President Obama. It seems that an ex-Google guy came to the Obama campaign and made some small changes to the website using A/B testing. Because the website was getting tremendous traffic, it didn’t tale long to se if changing a photo or the size of text would increase donations.

So the page had a video of Obama. He tried a turquoise photo and then a black and white photo an the black and white photo increased donations by 13.1%! When he swapped “Learn More” as a call for action button, the signups increased by 18.6% over the “Sign Up” text on the same button.

All in all, the changes made on this site, directed by the public, increased donations by 75 million dollars! The public was involved in a lab experiment and never even knew it. By making small changes and directing some traffic that way, a test can be quietly performed. Half of the traffic does not have to be redirected, just enough to see if a trend is happening.

In the case of the Obama website, the changes were quietly made and the determining factors were number of signups and amount of donations.

You see this type of test all the time on the internet. You might see a puppy with a sad face and a request for money to the ASPCA. Someone else may see the same ad, but with a sad-faced cat. If the cat brings in significantly more money than the dog photo, the dog ad may be discontinued. If you have an ad budget and the tools to track “sales” why would you continue to run an ad that does not work as well as another.

Sometimes I do the same thing on my website, not that I have enough sales to be able to tell if a change is working. Still, sometimes I offer a percentage sale off bags of coffee and sometimes I offer free shipping. In either case, some people love to see a percentage sale while others think free shipping saves them. On a regular order, the savings may be the same, but people seem to have a favorite way to save so who am I to decide for them.

According to the same article, it is not useful to try to out-think the public on these tests. This year the public may prefer a photo over text and next year prefer text under a photo. If your website can change to their preference, why not do it.

There appears to be software that can automatically rearrange elements on a webpage based upon evaluation of some outcome such as sales. Behind the scenes small changes are implemented and the result tracked. If the outcome favors the change, it is made, if not, it is discarded and another change is made and tested.

So besides websites, is this A/B testing done other places? Yes. Years ago I noticed TV Guide had two covers on the newsstands at the same time for the same week. For example, one might have been of the actor in a series and the other cover the actress in the same series. Although a short test, they could be determining which person caused more magazines to be sold and thus will use the same actor or actress on a future cover.

By making small changes, observing the outcome and keeping or dropping the change, can make a big difference in the final product.

I had a similar test designed for coffee shops. I was going to suggest coffee shops hold a taste test of freshly brewed Kona Coffees from different farms and have the visitors vote on their favorite. The favorite would be offered for sale at the coffee shop. Rather than have the shop owner just choose his favorite, let the buying public decide in a quick test.

If you want to read the article, here is a link:

http://www.wired.com/business/2012/04/ff_abtesting/

I am seeing targeted ads online using certain keywords that seem to work for them. Those words are “weird” and “shocking”. This particular ad say that a “weird food kills blood pressure” and that a “shocking video” will tell all. YAWN!

By the way, my view of the page has the header showing “A” on the left and “B” on the right