important that the product documentation fully describes the algorithms used, not just the operation of the tool. Your organization should not be basing decisions on techniques that are not understood. A data mining tool that relies on any sort of proprietary and undisclosed secret sauce is a poor choice.

Availability of Training for Both Novice and Advanced Users, Consulting, and Support

It is not easy to introduce unfamiliar data mining techniques into an organization. Before committing to a tool, find out the availability of user training and applications consulting from the tool vendor or third parties.

If the vendor is small and geographically remote from your data mining locations, customer support may be problematic. The Internet has shrunk the planet so that every supplier is just a few keystrokes away, but it has not altered the human tendency to sleep at night and work in the day; time zones still matter.

Vendor Credibility

Unless you are already familiar with the vendor, it is a good idea to learn something about its track record and future prospects. Ask to speak to references who have used the vendors software and can substantiate the claims made in product brochures.

We are not saying that you should not buy software from a company just because it is new, small, or far away. Data mining is still at the leading edge of commercial decision-support technology. It is often small, start-up companies that first understand the importance of new techniques and successfully bring them to market. And paradoxically, smaller companies often provide better, more enthusiastic support since the people answering questions are likely to be some people who designed and built the product.

Lessons Learned

The ideal data mining environment consists of a customer-centric corporate culture and all the resources to support it. Those resources include data, data miners, data mining infrastructure, and data mining software. In this ideal data mining environment, the need for good information is ingrained in the corporate culture, operational procedures are designed with the need to gather good data in mind, and the requirements for data mining shape the design of the corporate data warehouse.

Building the ideal environment is not easy. The hardest part of building a customer-centric organization is changing the culture and how to accomplish that is beyond the scope of this book. From a purely data perspective, the first

step is to create a single customer view that encompasses all the relationships the company has with a customer across all channels. The next step is to create customer-centric metrics that can be tracked, modeled, and reported.

Customer interactions should be turned into learning opportunities whenever possible. In particular, marketing communications should be set up as controlled experiments. The results of these experiments are input for data mining models used for targeting, cross-selling, and retention.

There are several approaches to incorporating data mining into a companys marketing and customer relationship management activities. Outsourcing is a possibility for companies with only occasional modeling needs. When there is an ongoing need for data mining, it is best done internally so that insights produced during mining remain within the company rather than with an outside vendor.

A data mining group can be successful in any of several locations within the company organization chart. Locating the group in IT puts it close to data and technical resources. Locating it within a business unit puts it close to the business problems. In either case, it is important to have good communication between IT and the business units.

Choosing software for the data mining environment is important. However, the success of the data mining group depends more on having good processes and good people than on the particular software found on their desktops.

Preparing Data for Mining

As a translucent amber fluid, gasoline-the power behind the transportation industry-barely resembles the gooey black ooze pumped up through oil wells. The difference between the two liquids is the result of multiple steps of refinement that distill useful products from the raw material.

Data preparation is a very similar process. The raw material comes from operational systems that have often accumulated crud, in the form of eccentric business rules and layers of system enhancements and fixes, over the course of time. Fields in the data are used for multiple purposes. Values become obsolete. Errors are fixed on an ongoing basis, so interpretations change over time. The process of preparing data is like the process of refining oil. Valuable stuff lurks inside the goo of operational data. Half the battle is refinement. The other half is converting its energy to a useful form-the equivalent of running an engine on gasoline.

The proliferation of data is a feature of modern business. Our challenge is to make sense of the data, to refine the data so that the engines of data mining can extract value. One of the challenges is the sheer volume of data. A customer may call the call center several times a year, pay a bill once a month, turn the phone on once a day, make and receive phone calls several times a day. Over the course of time, hundreds of thousands or millions of customers are generating hundreds of millions of records of their behavior. Even on todays computers, this is a lot of data processing. Fortunately, computer systems have become powerful enough that the problem is really one of having an adequate