Don’t Rely on Gut Feelings, Use A/B Testing

What is A/B Testing and what is the purpose?

A/B testing is having created two versions of a test, test A and test B, to determine which is more successful in achieving your goal. One test would be the Control, typically the version that already exists, and the other test would be the Treatment, the altered version. A/B testing is useful in marketing for determining how to best optimize your website.


The results of A/B testing can be quite surprising. You might assume one version will be better than the other, but you are most likely wrong. This is why A/B testing is very important in choosing how to go about changing your website. It is difficult to predict the success of a new website design. You want to be listening to your customers and not the Highest Paid Person’s Opinion (HiPPO). The customer is the one actually using the website, not some higher level person in the company. Therefore, do not make any decisions based on opinion or intuitions; test it out!

The goal for A/B testing is to increase your website’s conversion rate by gaining insight on visitor behavior. Essentially what is being measured is which version receives better feedback from customers; whether more people sign-up for your website or whether it increases sales. Even the slightest of changes can be beneficial. Whether it is changing the wording of one sentence, or changing the colors of the website.

According to the ‘The Ultimate Guide to A/B Testing‘ by Paras Chopra in Smashing Magazine, the following elements are usually always included in the testing:

  • The “call to action’s” or button’s wording, size, color, and placement
  • Headline
  • Product description
  • Length of from’s and types of fields
  • The website’s design
  • Product pricing and promotional offers
  • Images on the pages
  • Short vs. long text on the pages

It is important to note that when a version fails, do not be discouraged and give up. This is what A/B testing is about; seeing what works and what doesn’t work. Take those failed versions and adjust them to make them better. Having multiple A/B tests is helpful!

Basic Guidelines for Testing

When it comes to A/B testing, there are some guidelines to follow. These guidelines are important in making an accurate test, and are not exclusive.

  • Focus on the specific aspects that are important for achieving your goal. This is where you will be making adjustments to your website.
  • Test carefully. Making on rookie mistake can result in invalidate results.
  • Always test the two versions simultaneously. If you were to test these two versions at different times, other outside factors can affect the results.
  • Know how long to run a test; you do not want to conclude too early.
  • Make sure your A/B testing is consistent throughout the entire website.
  • Avoid using regular visitors in your test; try to use the testing of the Treatment version on new visitors.
  • Repeat visitors should receive the same version as they did before.
  • Determine if the results are significantly significant.

There are many other guidelines and factors to be considered when A/B testing. Make sure you do your research to figure out the best methods of conducting your test.

Examples of A/B Testing Resutls

The site Unbounce used A/B testing to see how they could get more people to sign up for their website. In doing so, they found that using the word your vs. my in a sentence can make all the difference.

Negative A/B Test Results

In their control they said “Start your free 30 day trial” and in the treatment they said “Start my free 30 day trial.” By using “my” they had a 90% increase in click-through-rates. This shows how changing one single word can increase conversion rate dramatically.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s