Defining Customer-Centric Metrics

On September 24, 1929, Lieutenant James H. Doolittle of the U.S. Army Air Corps made history by flying blind to demonstrate that with the aid of newly invented instruments such as the artificial horizon, the directional gyroscope, and the barometric altimeter, it was possible to fly a precise course even with the cockpit shrouded by a canvas hood. Before the invention of the artificial horizon, pilots flying into a cloud or fog bank would often end up flying upside down. Now, thanks to all those gauges in the cockpit, we calmly munch pretzels, sip coffee, and revise spreadsheets in weather that would have grounded even Lieutenant Doolittle. Good business metrics are just as crucial to keeping a large business flying on the proper course.

Business metrics are the signals that tell management which levers to move and in what direction. Selecting the right metrics is crucial because a business tends to become what it is measured by. A business that measures itself by the number of customers it has will tend to sign up new customers without regard to their expected tenure or prospects for future profitability. A business that measures itself by market share will tend to increase market share at the expense of other goals such as profitability. The challenge for companies that want to be customer-centric is to come up with realistic customer-centric measures. It sounds great to say that the companys goal is to increase customer loyalty; it is harder to come up with a good way to measure that quality in customers. Is merely having lasted a long time a sign of loyalty? Or should loyalty be defined as being resistant to offers from competitors? If the latter, how can it be measured?

Even seemingly simple metrics such as churn or profitability can be surprisingly hard to pin down. When does churn actually occur:

On the day phone service is actually deactivated?

On the day the customer first expressed an intention to deactivate?

At the end of the first billing cycle after deactivation?

On the date when the telephone number is released for new customers?

Each of these definitions plays a role in different parts of a telephone business. For wireless subscribers on a contract, these events may be far apart. And, which churn events should be considered voluntary? Consider a subscriber who refuses to pay in order to protest bad service and is eventually cut off; is that voluntary or involuntary churn? What about a subscriber who stops voluntarily and then doesnt pay the final amount owed? These questions do not have a right answer; they do suggest the subtleties of defining the customer relationship.

As for profitability, which customers are considered profitable depends a great deal on how costs are allocated.

Collecting the Right Data

Once metrics such as loyalty, profitability, and churn have been properly defined, the next step is to determine the data needed to calculate them correctly. This is different from simply approximating the definition using whatever data happens to be available. Remember, in the ideal data mining environment, the data mining group has the power to determine what data is made available!

Information required for managing the business should drive the addition of new tables and fields to the data warehouse. For example, a customer-centric company ought to be able to tell which of its customers are profitable. In many companies this is not possible because there is not enough information available to sensibly allocate costs at the customer level. One of our clients, a wireless phone company, approached this problem by compiling a list of questions that would have to be answered in order to decide what it costs to provide service to a particular customer. They then determined what data would be required to answer those questions and set up a project to collect it.

The list of questions was long, and included the following:

How many times per year does the customer call customer care?

Does the customer pay bills online, by check, or by credit card?

What proportion of the customers airtime is spent roaming?

On which outside networks does the customer roam?

What is the contractual cost for these networks?

Are the customers calls to customer care handled by the IVR or by human operators?

Answering these cost-related questions required data from the call-center system, the billing system , and a financial system. Similar exercises around other important metrics revealed a need for call detail data, demographic data, credit data, and Web usage data.

From Customer Interactions to Learning Opportunities

A customer-centric organization maintains a learning relationship with its customers. Every interaction with a customer is an opportunity for learning, an opportunity that can be siezed when there is good communication between data miners and the various customer-facing groups within the company.

Almost any action the company takes that affects customers-a price change, a new product introduction, a marketing campaign-can be designed so that it is also an experiment to learn more about customers. The results of these experiments should find their way into the data warehouse, where they

will be available for analysis. Often the actions themselves are suggested by data mining.

As an example, data mining at one wireless company showed that having had service suspended for late payment was a predictor of both voluntary and involuntary churn. That late payment is a predictor of later nonpayment is hardly a surprise, but the fact that late payment (or the companys treatment of late payers) was a predictor of voluntary churn seemed to warrant further investigation.

The observation led to the hypothesis that having had their service suspended lowers a customers loyalty to the company and makes it more likely that they will take their business elsewhere when presented with an opportunity to do so. It was also clear from credit bureau data that some of the late payers were financially able to pay their phone bills. This suggested an experiment: Treat low-risk customers differently from high-risk customers by being more patient with their delinquency and employing gentler methods of persuading them to pay before suspending them. A controlled experiment tested whether this approach would improve customer loyalty without unacceptably driving up bad debt. Two similar cohorts of low-risk, high-value customers received different treatments. One was subjected to the business as usual treatment, while the other got the kinder, gentler treatment. At the end of the trial period, the two groups were compared on the basis of retention and bad debt in order to determine the financial impact of switching to the new treatment. Sure enough, the kinder, gentler treatment turned out to be worthwhile for the lower risk customers-increasing payment rates and slightly increasing long term tenure.

Mining Customer Data

When every customer interaction is generating data, there are endless opportunities for data mining. Purchasing patterns and usage patterns can be mined to create customer segments. Response data can be mined to improve the targeting of future campaigns. Multiple response models can be combined into best next offer models. Survival analysis can be employed to forecast future customer attrition. Churn models can spot customers at risk for attrition. Customer value models can identify the customers worth keeping.

Of course, all this requires a data mining group and the infrastructure to support it.

The Data Mining Group

The data mining group is specifically responsible for building models and using data to learn about customers-as opposed to leading marketing efforts,