Using Postal Techniques to Find a Better Way to Test Email Lists
When my firm first began renting email lists for clients about two years, my initial experiences were pretty good.
We began working with established, mail-responsive postal list sources with email selects, such as “House & Garden” magazine or United Way donors.
But many of my local clients needed larger sources, so we started testing some larger email lists from consumer mass marketers. And that’s when our troubles began.
These lists were much lower in cost, and the counts were high—a hundred million names or more, mostly built from online surveys. The list owner claimed the open and clickthrough rates for these names were good, but we just couldn’t generate sufficient e-mail signups, store visits or online purchases from any of these lists, even though we had a history of good results for these clients with direct mail.
Here’s some lessons we’ve learned in the process of our tests.
1. When the e-mail list supplier provides the open or click through rate, you’re never really sure what is being measured. If the same person opens the e-mail twice, will that counts as two opens? If the e-mail program automatically opens every e-mail into the preview pane, is that counted as an open or not? Do the list sources sometimes artificially boost the count? I hope not.
2. Email campaigns using rented files are usually deployed by the list owner or the list owner’s agent. Why is this important? Think about it. When you get a postal list, there are techniques you can employ to evaluate the list. But if you don’t see the list before the mailing….well, you get the picture.
If it’s supposed to be females, does it mostly have women’s names? Say you order a postal file of accountants, but the list you receive is entirely comprised of libraries. With a postal list you can see on inspection that the list house sent you the wrong list, but with e-mail lists you are 100% at the mercy of the e-mail list provider.
3. Validating email list quality is difficult for multiple reasons. You can’t see if any of the names on the e-list are neighbors or people you know from the community to help validate list quality. Also, you can’t validate the zipcode/geographic selection, or see if you got a true nth sample across your whole geographic select.
Another problem is that you can’t select a small quantity of names, say 1,000 to 2,000 and do a small pre-test mailing to see if you get any results at all. While this is not statistically valid, we’ve found that small pretests are a great way to save money on list tests. While you still have to pay for the 5,000 test names, you save the costs of 3,000 to 4,000 for printing, assembly and postage.
Finally, when we had a postal list we weren’t sure of, we’d actually lookup phone numbers and call a handful of names across the list just to see if the list was as represented. If the list was supposed to be RV owners, were at least seven or eight of the 10 calls RV owners? If not, we’d call the list source, complain, and send the list back.
A New Idea
So, after 20 no/low-results local e-mail experiences, we put a hold on recommending these large e-mail databases for local prospecting or list building.
But we have a new idea, unfolding right now.
We’re looking at multi-media, getting samples of postal addresses delivered to us, along with ordering the e-blast for the same names. For example, we’d order a 25,000 e-mail deployment (done by the list owner), but order an nth sample of 3,000 postal names from the same database.
Now we can use all of our inspection methods for postal files, and verify the list before the eblast goes out.
And, we can also do 3-way split tests:
A. One batch sent to e-mail addresses alone.
B. One batch sent to postal address alone.
C. One batch sent to both.
Hopefully these tests will help us determine the impact and relative strength of the e-mail versus direct mail to the same database.
We’re excited about these new possibilities and we’ll share the results in a future article.
John Klein, is president of Klein Direct LLC.
