Insight Center ARCHIVES

Industry knowledge to help you grow your business

2006 2nd Quarter:

Don’t Count “Fairways Missed”

Measuring Your Worth As A Marketer

Grover Pagano, VP, Modeling & Analysis

A highly effective marketing engine is the most efficient way to grow market share. In the absence of one, growth options for financial institutions may be limited to purchasing other financial institutions or building new branch locations. But how do you measure marketing success?

The following article explains just how good a job you may be doing when managing marketing programs directed at existing customers and members of a financial institution. The key is knowing where to look to prove your worth and then looking there over time. It certainly was a nice surprise for a large retail bank with assets over $50 billion and more than 650 branches.

Measuring your worth as a marketer takes time–often more than you think. Moreover, you may be looking in the wrong places as you attempt to measure your marketing prowess. Just as missing every fairway in golf doesn’t guarantee you will shoot your worst round, deploying a series of marketing campaigns that individually fail to outperform a control group does not necessarily mean that you are missing the mark when managing retention and cross-sell programs.

Current Situation

To establish some definitions, “performance measurement,” “results analysis” and “response analysis” are all members of the same family and focus on measuring the return on your marketing dollars. A “control group” is a group that does not receive a marketing communication, while a “treatment group” does receive your marketing communication. Success occurs whenever the treatment group has a better return on investment than the control group.

However, a more common scenario finds the January credit card promotion fails to outperform the control group; or the March CD and April home equity campaigns show no significant differences than their respective control groups. Unfortunately, it seems success is a rare occurrence, or it is not readily apparent. Moreover, if and when success does occur, it may not seem significant enough to warrant the cost and effort.

But, you may not be looking deep enough. You may be counting the “fairways missed,” but not measuring the final score. You may actually be shooting under par!

Common Measurement Problems When Marketing to Existing Accountholders

Typically, marketers measure to ensure that the targeted population outperforms the control population. For example, did the CD target group outperform the control group that was not selected for the promotion, but would have performed had it not been in the control group? The problem is that we often neglect to take our analysis a step farther. For instance, did we measure whether the CD group selected a money market to a greater extent than the control group; or whether the group that was targeted for a credit card ended up selecting a home equity loan instead? Another problem is timing. We usually measure deposit programs after 60 days and loan programs after 90 days. What if the treatment group responds to a home equity loan in 95 days versus the control group that never selects a home equity loan? More complicated still is calculating the customers that were targeted with a credit card offer, but selected a home equity loan in 95 days. Even the most detailed process may not account for these indirect successes. Finally, measuring retention efforts is another difficult challenge as we look to see if our treatment group has increased or decreased its deposit and loan activity in totality over an extended period of time. Too often, our analysis paints a picture of our marketing efforts producing minor gains.

Looking Deeper

If the true goal is to shoot par or better, rather than falsely claim success or failure after the first few holes, we must take a long-term approach when measuring treatment versus control.

One progressive bank did just that and found some very compelling results. At the start of an 18-month test to capture the effectiveness of the bank’s collective targeting efforts for cross-sell and retention, database marketing experts at the bank marked a significant number of households as the control population and excluded them from every type of marketing communication feasible. For households in the treatment group, the bank deployed a number of cross-sell and retention messages using branch courtesy calls, direct mail, e-mail and anti-attrition campaigns. Of course the control group still received the standard “noise,”such as media and in-branch promotions that add to the complexity of measuring existing customers. But for the most part, over the duration of the 18-month test, the treatment group was the target of a variety of marketing communications and strategies, while the control group was exposed to dramatically fewer marketing messages.


To determine the results, the bank measured the difference in deposit balances and revolving loan balances between the treatment and control groups after the 18-month period. The results were impressive and highlight the impact of successfully targeting and “talking” to the right groups of accountholders:

  • Deposits–Net interest margin impact (NIM) of better retention efforts approached $5 million (assuming a 3.5% NIM).
  • Revolving Loans–NIM impact of better retention of these loans exceeded $3.25 million (assuming a 3.5% NIM).

Over $8 million of bank NIM far exceeded the expense the bank incurred for the team of professionals responsible for driving the targeting and analytic strategy.


“Measuring retention over 12 to 24 months is a unique way to monitor and illustrate our collective value as direct and database marketers.”

Measuring retention over 12 to 24 months is a unique way to monitor and illustrate our collective value as direct and database marketers. Halo effects of our campaign efforts, campaign timing, and care calls to “at risk” consumers are all captured in retention metrics. As customers and members move from single to multi-service households, retention strengthens and this is what we truly seek – long-term, engaged consumers that use your establishment as their primary financial institution.

Our efforts over time show the real impact of these types of programs. By their nature, direct and database marketing techniques incrementally optimize events time and time again. Although our individual campaigns may not always appear overwhelming, our results over time are certainly very impressive.
In Summary:

  • Define a random control group and exclude this group from every promotion possible
  • Implement best-in-class targeting processes
  • Measure the difference in treatment versus control over 12 to 24 months, in addition to individual campaign metrics you may already have in place

Most importantly, don’t hesitate to do some self or group promotion when you find successes, or even better, “a hole in one!”

To learn more about Harland’s marketing and analytical services, contact your Harland sales representative or call 1-866-281-5788.