Collaborative Filtering: A Nearest Neighbor Approach to

Making Recommendations 282

Building Profiles 283

Comparing Profiles 284

Making Predictions 284

Lessons Learned 285

Chapter 9 Market Basket Analysis and Association Rules 287

Defining Market Basket Analysis 289

Three Levels of Market Basket Data 289

Order Characteristics 292

Item Popularity 293

Tracking Marketing Interventions 293

Clustering Products by Usage 294

Association Rules 296

Actionable Rules 296

Trivial Rules 297

Inexplicable Rules 297

How Good Is an Association Rule? 299

Building Association Rules 302

Choosing the Right Set of Items 303

Product Hierarchies Help to Generalize Items 305

Virtual Items Go beyond the Product Hierarchy 307

Data Quality 308

Anonymous versus Identified 308

Generating Rules from All This Data 308

Calculating Confidence 309

Calculating Lift 310

The Negative Rule 311

Overcoming Practical Limits 311

The Problem of Big Data 313

Extending the Ideas 315

Using Association Rules to Compare Stores 315

Dissociation Rules 317

Sequential Analysis Using Association Rules 318

Lessons Learned 319

Chapter 10 Link Analysis 321

Basic Graph Theory 322

Seven Bridges of Konigsberg 325

Traveling Salesman Problem 327

Directed Graphs 330

Detecting Cycles in a Graph 330

A Familiar Application of Link Analysis 331

The Kleinberg Algorithm 332

The Details: Finding Hubs and Authorities 333

Creating the Root Set 333

Identifying the Candidates 334

Ranking Hubs and Authorities 334

Hubs and Authorities in Practice 336

Case Study: Who Is Using Fax Machines from Home? 336

Why Finding Fax Machines Is Useful 336

The Data as a Graph 337

The Approach 338

Some Results 340

Case Study: Segmenting Cellular Telephone Customers 343

The Data 343

Analyses without Graph Theory 343

A Comparison of Two Customers 344

The Power of Link Analysis 345

Lessons Learned 346

Chapter 11 Automatic Cluster Detection 349

Searching for Islands of Simplicity 350

Star Light, Star Bright 351

Fitting the Troops 352

K-Means Clustering 354

Three Steps of the K-Means Algorithm 354

What K Means 356

Similarity and Distance 358

Similarity Measures and Variable Type 359

Formal Measures of Similarity 360

Geometric Distance between Two Points 360

Angle between Two Vectors 361

Manhattan Distance 363

Number of Features in Common 363

Data Preparation for Clustering 363

Scaling for Consistency 363

Use Weights to Encode Outside Information 365

Other Approaches to Cluster Detection 365

Gaussian Mixture Models 365

Agglomerative Clustering 368

An Agglomerative Clustering Algorithm 368

Distance between Clusters 368

Clusters and Trees 370 Clustering People by Age: An Example of

Agglomerative Clustering 370

Divisive Clustering 371

Self-Organizing Maps 372

Evaluating Clusters 372

Inside the Cluster 373

Outside the Cluster 373

Case Study: Clustering Towns 374

Creating Town Signatures 374

The Data 375

Creating Clusters 377

Determining the Right Number of Clusters 377

Using Thematic Clusters to Adjust Zone Boundaries 380

Lessons Learned 381

Chapter 12 Knowing When to Worry: Hazard Functions and

Survival Analysis in Marketing 383

Customer Retention 385

Calculating Retention 385

What a Retention Curve Reveals 386

Finding the Average Tenure from a Retention Curve 387

Looking at Retention as Decay 389

Hazards 394

The Basic Idea 394

Examples of Hazard Functions 397

Constant Hazard 397

Bathtub Hazard 397

A Real-World Example 398

Censoring 399

Other Types of Censoring 402

From Hazards to Survival 404

Retention 404

Survival 405

Proportional Hazards 408

Examples of Proportional Hazards 409

Stratification: Measuring Initial Effects on Survival 410

Cox Proportional Hazards 410

Limitations of Proportional Hazards 411

Survival Analysis in Practice 412

Handling Different Types of Attrition 412

When Will a Customer Come Back? 413

Forecasting 415

Hazards Changing over Time 416

Lessons Learned 418

Chapter 13 Genetic Algorithms 421

How They Work 423

Genetics on Computers 424

Selection 429

Crossover 430

Mutation 431

Representing Data 432 Case Study: Using Genetic Algorithms for

Resource Optimization 433

Schemata: Why Genetic Algorithms Work 435

More Applications of Genetic Algorithms 438

Application to Neural Networks 439

Case Study: Evolving a Solution for Response Modeling 440

Business Context 440

Data 441

The Data Mining Task: Evolving a Solution 442

Beyond the Simple Algorithm 444

Lessons Learned 446