How Data Mining Is Being Used Today 15

A Supermarket Becomes an Information Broker 15

A Recommendation-Based Business 16

Cross-Selling 17

Holding on to Good Customers 17

Weeding out Bad Customers 18

Revolutionizing an Industry 18

And Just about Anything Else 19

Lessons Learned 19

Chapter 2 The Virtuous Cycle of Data Mining 21

A Case Study in Business Data Mining 22

Identifying the Business Challenge 23

Applying Data Mining 24

Acting on the Results 25

Measuring the Effects 25

What Is the Virtuous Cycle? 26

Identify the Business Opportunity 27

Mining Data 28

Take Action 30

Measuring Results 30

Data Mining in the Context of the Virtuous Cycle 32

A Wireless Communications Company Makes

the Right Connections 34

The Opportunity 34

How Data Mining Was Applied 35

Defining the Inputs 37

Derived Inputs 37

The Actions 38

Completing the Cycle 39

Neural Networks and Decision Trees Drive SUV Sales 39

The Initial Challenge 39

How Data Mining Was Applied 40

The Data 40

Down the Mine Shaft 40

The Resulting Actions 41

Completing the Cycle 42

Lessons Learned 42

Chapter 3 Data Mining Methodology and Best Practices 43

Why Have a Methodology? 44

Learning Things That Arent True 44

Patterns May Not Represent Any Underlying Rule 45

The Model Set May Not Reflect the Relevant Population 46

Data May Be at the Wrong Level of Detail 47

Learning Things That Are True, but Not Useful 48

Learning Things That Are Already Known 49

Learning Things That Cant Be Used 49

Hypothesis Testing 50

Generating Hypotheses 51

Testing Hypotheses 51

Models, Profiling, and Prediction 51

Profiling 53

Prediction 54

The Methodology 54

Step One: Translate the Business Problem

into a Data Mining Problem 56

What Does a Data Mining Problem Look Like? 56

How Will the Results Be Used? 57

How Will the Results Be Delivered? 58

The Role of Business Users and Information Technology 58

Step Two: Select Appropriate Data 60

What Is Available? 61

How Much Data Is Enough? 62

How Much History Is Required? 63

How Many Variables? 63

What Must the Data Contain? 64

Step Three: Get to Know the Data 64

Examine Distributions 65

Compare Values with Descriptions 66

Validate Assumptions 67

Ask Lots of Questions 67

Step Four: Create a Model Set 68

Assembling Customer Signatures 68

Creating a Balanced Sample 68

Including Multiple Timeframes 70

Creating a Model Set for Prediction 70

Partitioning the Model Set 71

Step Five: Fix Problems with the Data 72

Categorical Variables with Too Many Values 73

Numeric Variables with Skewed Distributions and Outliers 73

Missing Values 73

Values with Meanings That Change over Time 74

Inconsistent Data Encoding 74

Step Six: Transform Data to Bring Information to the Surface 74

Capture Trends 75

Create Ratios and Other Combinations of Variables 75

Convert Counts to Proportions 75

Step Seven: Build Models 77

Step Eight: Assess Models 78

Assessing Descriptive Models 78

Assessing Directed Models 78

Assessing Classifiers and Predictors 79

Assessing Estimators 79

Comparing Models Using Lift 81

Problems with Lift 83

Step Nine: Deploy Models 84

Step Ten: Assess Results 85

Step Eleven: Begin Again 85

Lessons Learned 86

Chapter 4 Data Mining Applications in Marketing and

Customer Relationship Management 87

Prospecting 87

Identifying Good Prospects 88

Choosing a Communication Channel 89

Picking Appropriate Messages 89

Data Mining to Choose the Right Place to Advertise 90

Who Fits the Profile? 90

Measuring Fitness for Groups of Readers 93

Data Mining to Improve Direct Marketing Campaigns 95

Response Modeling 96

Optimizing Response for a Fixed Budget 97

Optimizing Campaign Profitability 100

How the Model Affects Profitability 103

Reaching the People Most Influenced by the Message 106

Differential Response Analysis 107

Using Current Customers to Learn About Prospects 108

Start Tracking Customers before They Become Customers 109

Gather Information from New Customers 109

Acquisition-Time Variables Can Predict Future Outcomes 110

Data Mining for Customer Relationship Management 110

Matching Campaigns to Customers 110

Segmenting the Customer Base 111

Finding Behavioral Segments 111

Tying Market Research Segments to Behavioral Data 113

Reducing Exposure to Credit Risk 113

Predicting Who Will Default 113

Improving Collections 114

Determining Customer Value 114

Cross-selling, Up-selling, and Making Recommendations 115

Finding the Right Time for an Offer 115

Making Recommendations 116

Retention and Churn 116

Recognizing Churn 116

Why Churn Matters 117

Different Kinds of Churn 118