Seven tips for interpreting your email marketing reports

The open rate this week was 31%. Last week it was 28%. [Long pause] Um...OK. [Silence]

Campaign reports are like exhibits in a Museum of Modern Art. We're pretty certain they're valuable, but we're not quite sure what they're trying to say.

Is there some deep meaning in there about the nature of our subject lines? Or is it just random numbers?

Here some tips to help you get your head around those report metrics...

Look at the right numbers

Open rates and click rates are discussed everywhere. The danger is we focus on those two to the exclusion of other numbers: numbers that truly reflect how our emails are performing in the context of what we want to achieve.

Each of the numbers you get in a report tells you something. But the final measure of an email's performance is not usually the open rate...it's how the email contributed to your stated goals: sales, downloads, pageviews, customer interactions, whatever.

Understand what the number really represents

Interpreting a number starts with understanding exactly what it represents. Be sure you understand how a number is measured and calculated.

To the innocent, for example, the term "open rate" implies conscious effort on behalf of the recipient to view your email. We know that's not true.

And does an open rate of 30% mean 3000 people opened one of the 10,000 emails you sent. Or 300 people opened an email ten times? Or 1500 people opened one of the 5,000 emails that didn't bounce back as undeliverable? Megan Ouellet has some more examples along these lines.

Compare the right numbers

Numbers in isolation have little meaning. They are only relevant when compared to other numbers. Is an open rate of 12% good? Good compared to what?

Previously, much weight was given to comparisons with published benchmarks. How does my open rate compare with industry averages?

While such statistics have some value, they don't help much with practical decision making.

If you're looking to sell your Ferrari, it helps little to know the average price for all used car sales in the USA.

The useful comparisons are with your previous results and your goals. So you can relate changes in results to changes in your emails and list. So you can repeat the things that bring improvements and avoid those that don't. Spencer Kollas and DJ Waldow have both written recently on this topic.

Consider the full range of explanations

When interpreting numbers, the tendency is to jump on the obvious or traditional explanation for a change.

My sales dropped, clearly the offer wasn't good enough. My open rate increased, so my subject line was better.

As Anna Billstrom describes, it may not be so simple. There can be several explanations for a change in performance, and it pays to give each due consideration. In particular...

Look at numbers together

...avoid looking at numbers in isolation. The email you send provokes a chain of events at the subscriber's end:

[receive] --> [open] --> [click] --> [take further action]

The different numbers within a report each represent a position on that chain. And each number impacts on the subsequent numbers further along the chain. Low sales do not indicate a poor offer if you know that hardly anybody saw the email in the first place.

Be aware of things that don't show up

It sounds obvious, but campaign reports feature those numbers that are easily measured. They don't provide numbers on intangibles or any effects that are long-term in nature.

This can lead to poor decision making.

If it cost $1000 to send out a campaign and it brought in $5000 in immediate sales, you'd judge it a success. But what if those emails were mind numbingly uninteresting to the vast majority of your subscribers.

The reports don't tell you about the long-term brand damage and subsequent lost sales through other channels. So you need to remain aware of what's missing from the spreadsheet.

Do something with them

Finally, understanding and interpreting reports only has purpose if you then use the results to make improvements to your emails or tests.

This isn't just about using insights to change elements of your email program, for example by knowing which offers get the most response. It's also about passing on those insights and numbers to other parts of the business.

If you know which subject lines resonate with your subscribers, can you use that info when writing ads for your PPC search engine listings?

In another example, Michael DesRochers highlights the value to sales or service staff of knowing who clicked on an email and where.

Now go out and enjoy the art...

More on email marketing statistics | Tags: email marketing, email marketing metrics, email reports

 feed | email | Follow me on Twitter

The main website for everything email marketing
Email marketing search engine | Introduction
Email advertising basics | Service listings | Sources of statistics
Advice on email design, delivery issues and list building

Top email marketing info sources for 2009
Outlook 2007 and HTML design
42 HTML email design resources

follow me on Twitter