Tomicide Solutions October 2006: How To Test Your Marketing

By Tom "Bald Dog" Varjan

First let's start with something interesting that can save you a truckload of dough. You must...

...Understand the Difference Between Research And Testing

This is crucial. If I wan to swim in the swimming pool, all I have to do is to check...

  1. ...whether or not there is water in the pool
  2. ...and if there is whether or not it's the right temperature

If both of these criteria are fulfilled, I can start I'm ready to swim at my heart's content until I've had enough.

But I don't need to hire a team of consulting engineers to launch a full-blown research project on swimming pool buildings, the science of concrete pouring and fluid dynamics. That's retarded but that what most companies do under the aegis of "market research."

Yes, I'm all for some research (Hey, I come from a consulting engineering background), but there is a limit to everything. All right, maybe the only exception is the famous German Black Forest Chocolate Cake.

Don't waste your time and money researching the market. You will never get true answers to your research. Instead, you will always get what you're looking for. This phenomenon is called the Participative Universe Principle, and John Wheeler Archibald has done extensive research on it.

This is why one group of doctors have lots of "research evidence" to prove that salty food increases blood pressure, while another group of doctors have preponderance of research evidence to prove the opposite. So, that's all about research.

Many moons ago, in 1943, Thomas Watson, the chairman of IBM spent a small fortune on hiring a prestigious consulting team to research the personal computer market. The team concluded, "Maybe only for a few computers." That was a very expensive research to find the wrong answer.

"Louis Pasteur's theory of germs is ridiculous fiction." - commented the highly esteemed Pierre Pachet, Professor of Physiology in 1872

Or one of my favourite research mistakes, "Guitar music is on the way out." - To Decca Records' executive to the Beatles in 1962 and declined to record for them.

What you need is not a neat group of highly-paid pinstripes to conduct "research." Instead of research, you can just test the market for instant results.

You see, Edison didn't research the light bulb. He tested his ideas over and over again, and then it eventually worked out. So did the Wright brothers with the airplane. Luckily, the other people who were fiddling with airplanes at that time, Alexander Graham Bell and the President of the Smithsonian Institution were too busy "researching." The Wright brothers just kept testing. They were not hindered by being engineers.

Split-Testing Online

The great thing about testing online that it's very easy to modify everything and you get pretty quick feedback as to whether or not the new version is better than your control. Remember, a headline alone can make as much as 2,100% difference to your response rate. When there is so much at stake, you can't afford to rely on random results. You want to optimise your results. I deliberately don't say "maximise" because what the use of flooding yourself with tyre-kickers. By optimising I mean the highest number of properly qualified people.

Also, on the web, you can track everything pretty easily. If you use the same web host I do, you will have a broad range of pretty neat tools to fine-tune your pages.

Once upon a time it required lots of programming to set up split-testing, but nowadays you can either use your mass email distribution system, the one you use for sending newsletters (My favourite is Aweber Communications) or your shopping car (My favourite is 1Shopingcart).

And then comes the testing itself.

Let's say you want to test two website landing pages (index.html). First create two identical landing pages, index11.html and index22.html. I always make sure there is more than a one-character difference between the pages. Thus I would never use index1 and index2. When the computer converts them into binary code, the difference is so tiny, that it can be misunderstood. Can you see how handy that I come from a microprocessor designer background? You can't even imagine how much headache this problem caused me at the beginning of my designer career. But I cracked it.

Using index11 and index22 I have a difference in two characters not only one.

Now you have two identical pages. Leave index11.html alone. That's what we call "control." This is the measuring stick relative to which you measure response rate.

On index22.html change one single variable. This is important because if you change more than one variable you won't know what caused the change in response.

Start by changing the headline si8nce it has the highest impact on response rates.

So, lets say on your index11 the headline is: "Attention and Warning for Construction Companies: Are You Making These 7 Hiring Mistakes?"

On your index22 the headline is: "Attention, Warning and FREE Special Report for Construction Companies: Are You Making These 7 Hiring Mistakes?"

You've just inserted "FREE Special Report". That's the only change. Run these two pages for a week and see which one gets more responses. Email half of your list with a link to index11 and the other half to index22.

Whichever page wins, it becomes the control and the other one is ditched. Let's say index22 wins. Ditch index11 and create index33 with a new headline. Then run them. Let's say index22 wins again. Ditch index33 and create index44 with a new headline.

By the time you reach index99, you have a kick-arse headline. Now you can start fiddling with other elements, such as...

Yes, it takes a bit of time and fiddling, but you're creating a peddle-free automated client acquisition system, so your investment will pay of ion spades.

Now it's time to analyse your results. If you use Aweber) or 1Shopingcart), then your job is fairly easy. They both provide great analytical tools.

What you learn from this split testing is more valuable than any silly survey. When surveyed, people don't say what they mean. They say what makes them look good. But when you ask them to respond to something, they will show the white of their teeth. That is you get real reliable results.

Remember, based on surveys, New Coke was a blazing success. Reality it was miserable flop.

So, every time you're about to release something new, make sure you test it for response. And forget about that research rubbish. If you go to the water with a broad selection of baits, you have a better chance to catch fish than the next dude who goes to the library to study and research the eating habits of fish. Or remember the swimming pool example. Stick you finder in the water and if the temperature is right, then jump.

Look back on history. The greatest things have been invented by testers not researchers. Alternative healers have been successfully healing cancer patients based on tried and tested methods. In the meantime the medical industry sinks billions of dollars into cancer research.

Split-Testing Testing Off-Line

Tracking is vital to figure out where the best responses are coming from. This way after a few runs you can re-evaluate your ads and get rid of the duds.

Her is one more consideration. Although we start the advertising process off-line, it will end online. Why? Because it's irresponsible to run a business without a website and the web can be nicely automated.

And here is one thing you must understand. We're living in a sceptical world. Most people live their lives based on fear and they are scared shitless. Here in Canada alone 19% of the population is disturbed and 1.2 million are clinical nutcases. If you ask people to call you, they get worried that there is a hard-nosed commission-hungry salesperson at the other end who tries to sell them something. So, they won't call.

Emphasise that they call a voice mail system that collects their data that's needed to send out the requested information. So, I can imagine the following response vehicles...

Make sure the response is NOT a personal interaction. Personal interaction is a lose-lose. People hate it and for you it clogs up your time. And don't even dream about "closing people" when they call you for the free offer. Many companies have tried trick and lost their credibility.

Tracking Off-Line Responses

With off-line advertising you must make sure both parts of your split ad run under very similar circumstances. They run on the same days, on the same pages, run for the same period of time. You have to specify your request to the newspaper's creative people who organise the ads. You know what? Even the weather can effect the fate of your ad. On rainy days more people stay at home and read the ad section.

The next question is the format of the ad. Start with a small classified ad. People who read the classified section are actually reading it with the intention of buying if they find what they're looking for. Yes, the space is limited but a good two-step ad can do magic. The ad "sells" the free report and the free report sells the main service. A sure-fire advertising approach.

Voice mail

Set up your phone system with several extensions and in your ad ask people to call specific extensions. "To request your copy of our Special Report, please call voice mail 123-456-7890 extension 123, and leave your details and we'll rush you the report in a jiffy."

If you can't have extensions, ask people to call a general voice mail and specify the name of the person they're leaving the mess sage. "To request your copy of our Special Report, please call Cathy's voice mail 123-456-7890, leave your details and she'll rush you the report in a jiffy."

So, here Cathy is the imaginary contact for the ad in the Vancouver Sun, Jeff is the contact for the ad in the Dogbreeder's Times and Andrea is the contact for the ad in the Gravedigger Monthly. So, you can track clearly how many people call you from which magazine.

Fax

On fax you can create departments like so: "To request your copy of our Special Report, please fax your request to 123-456-7890 Dept. 1D, and we'll rush [fax, mail or email] you the report in a jiffy." Here "Dept. 1D" identifies the paper in which the ad was placed.

Email Autoresponders

Here you can create lots of emails for the same company or even the same person. Fred Tigernuts at Nutcrunchers Inc. can be replicated as...

And then you can assign a specific email to a specific ad. You will know that f_tigernuts666@nutcrunchers.com is the email belonging to the ad in the 6 June 2006 issue of Embalmer's Guild Monthly. Then you can slap all the responses into a spreadsheet and you can analyse your results.

Something to mail to you

Similar to the fax. You attach a department identification to every paper you run the ad in. And then just ask. Send your pre-paid response card to Courageous Coffin Carvers Inc. Dept D2. And D2 identifies the magazine where the ad ran.

Alternative Testing-Tracking Methods

You can speed up the test process by running ad A against ad B, and ad C against ad D. Then select the two winners and run them against each other. Now you have one winner out of for ads. You can repast this for another set of four ads and have another winner. Then you can run the two winners, and you have a winners out of eight ads.

Another method is t run all your ads against a pre-set control ad.

Ad A against the Control

Ad B against the Control

Ad C against the Control

Ad D against the Control

...

Ad J against the Control

Then tabulate your results...

Ad A is 25% better than the Control

Ad B is 17% poorer than the Control

Ad C is 35% better than the Control

Ad D is 12% poorer than the Control

Ad J is 3% better than the Control

By this point it becomes as obvious as a ham sandwich that ad C is the best, and ad B is the poorest.

Let's Summarise...

I do agree that we need some market research, but the real data lies in testing. During my engineering years I often worked with scientists, that is, people who learnt the same electronics I did, but from a theoretical perspective. Instead of testing, they researched and calculated things. When an new challenge came up, we, the engineers would build a prototype and hook it up to the oscilloscope to see the real results and the real behaviour of the circuit. The scientists, on the other hand, grabbed their calculators and started calculating fancy models from one of the text books. And I can report to you that the two group often had pretty different results. Putting it differently, there is a world of difference between jumping out of a plane with a parachute and calculating gravity in a quiet corner of the library.

And, yes, we want to calculate a bit, but then it's time for testing. So, do your best to write your ad or any other promo piece and then take it to the market. Let the market decide which one is the best.

Attribution: "This article was written by Tom "Bald Dog" Varjan who helps privately held information technology companies to develop high leverage client acquisition systems and business development teams in order to sell their products and services to premium clients at premium fees and prices. Visit Tom's website at http://www.varjan.com.