of maleness. It seems safe to assume that the link is from men to beer and not vice versa.

With behavioral data, the direction of causality is not always so clear. Consider a couple of actual examples from real data mining projects:

People who have purchased certificates of deposit (CDs) have little or no money in their savings accounts.

Customers who use voice mail make a lot of short calls to their own number.

Not keeping money in a savings account is a common behavior of CD holders, just as being male is a common feature of beer drinkers. Beer companies seek out males to market their product, so should banks seek out people with no money in savings in order to sell them certificates of deposit? Probably not! Presumably, the CD holders have no money in their savings accounts because they used that money to buy CDs. A more common reason for not having money in a savings account is not having any money, and people with no money are not likely to purchase certificates of deposit. Similarly, the voice mail users call their own number so much because in this particular system that is one way to check voice mail. The pattern is useless for finding prospective users.

Prediction

Profiling uses data from the past to describe what happened in the past. Prediction goes one step further. Prediction uses data from the past to predict what is likely to happen in the future. This is a more powerful use of data. While the correlation between low savings balances and CD ownership may not be useful in a profile of CD holders, it is likely that having a high savings balance is (in combination with other indicators) a predictor of future CD purchases.

Building a predictive model requires separation in time between the model inputs or predictors and the model output, the thing to be predicted. If this separation is not maintained, the model will not work. This is one example of why it is important to follow a sound data mining methodology.

The Methodology

The data mining methodology has 11 steps.

1. Translate the business problem into a data mining problem.

2. Select appropriate data.

3. Get to know the data.

4. Create a model set.

5. Fix problems with the data.

6. 7. 8. 9. 10.

Transform data to bring information to the surface. Build models. Asses models. Deploy models. Assess results.

11. Begin again.

As shown in Figure 3.5, the data mining process is best thought of as a set of nested loops rather than a straight line. The steps do have a natural order, but it is not necessary or even desirable to completely finish with one before moving on to the next. And things learned in later steps will cause earlier ones to be revisited.

Figure 3.5 Data mining is not a linear process.

Step One: Translate the Business Problem into a Data Mining Problem

A favorite scene from Alice in Wonderland is the passage where Alice asks the Cheshire cat for directions:

Would you tell me, please, which way I ought to go from here?

That depends a good deal on where you want to get to, said the Cat.

I dont much care where- said Alice.

Then it doesnt matter which way you go, said the Cat.

-so long as I get somewhere, Alice added as an explanation.

Oh, youre sure to do that, said the Cat, if you only walk long enough.

The Cheshire cat might have added that without some way of recognizing the destination, you can never tell whether you have walked long enough! The proper destination for a data mining project is the solution of a well-defined business problem. Data mining goals for a particular project should not be stated in broad, general terms, such as:

Gaining insight into customer behavior

Discovering meaningful patterns in data

Learning something interesting

These are all worthy goals, but even when they have been achieved, they are hard to measure. Projects that are hard to measure are hard to put a value on. Wherever possible, the broad, general goals should be broken down into more specific ones to make it easier to monitor progress in achieving them. Gaining insight into customer behavior might turn into concrete goals:

Identify customers who are unlikely to renew their subscriptions.

Design a calling plan that will reduce churn for home-based business customers.

Rank order all customers based on propensity to ski.

List products whose sales are at risk if we discontinue wine and beer sales.

Not only are these concrete goals easier to monitor, they are easier to translate into data mining problems as well.

What Does a Data Mining Problem Look Like?

To translate a business problem into a data mining problem, it should be reformulated as one of the six data mining tasks introduced in Chapter One: