Artificial Neural Networks

Artificial neural networks are popular because they have a proven track record in many data mining and decision-support applications. Neural networks- the artificial is usually dropped-are a class of powerful, general-purpose tools readily applied to prediction, classification, and clustering. They have been applied across a broad range of industries, from predicting time series in the financial world to diagnosing medical conditions, from identifying clusters of valuable customers to identifying fraudulent credit card transactions, from recognizing numbers written on checks to predicting the failure rates of engines.

The most powerful neural networks are, of course, the biological kind. The human brain makes it possible for people to generalize from experience; computers, on the other hand, usually excel at following explicit instructions over and over. The appeal of neural networks is that they bridge this gap by modeling, on a digital computer, the neural connections in human brains. When used in well-defined domains, their ability to generalize and learn from data mimics, in some sense, our own ability to learn from experience. This ability is useful for data mining, and it also makes neural networks an exciting area for research, promising new and better results in the future.

There is a drawback, though. The results of training a neural network are internal weights distributed throughout the network. These weights provide no more insight into why the solution is valid than dissecting a human brain explains our thought processes. Perhaps one day, sophisticated techniques for

probing neural networks may help provide some explanation. In the meantime, neural networks are best approached as black boxes with internal workings as mysterious as the workings of our brains. Like the responses of the Oracle at Delphi worshipped by the ancient Greeks, the answers produced by neural networks are often correct. They have business value-in many cases a more important feature than providing an explanation.

This chapter starts with a bit of history; the origins of neural networks grew out of actual attempts to model the human brain on computers. It then discusses an early case history of using this technique for real estate appraisal, before diving into technical details. Most of the chapter presents neural networks as predictive modeling tools. At the end, we see how they can be used for undirected data mining as well. A good place to begin is, as always, at the beginning, with a bit of history.

Neural networks have an interesting history in the annals of computer science. The original work on the functioning of neurons-biological neurons-took place in the 1930s and 1940s, before digital computers really even existed. In 1943, Warren McCulloch, a neurophysiologist at Yale University, and Walter Pitts, a logician, postulated a simple model to explain how biological neurons work and published it in a paper called A Logical Calculus Immanent in Nervous Activity. While their focus was on understanding the anatomy of the brain, it turned out that this model provided inspiration for the field of artificial intelligence and would eventually provide a new approach to solving certain problems outside the realm of neurobiology.

In the 1950s, when digital computers first became available, computer scientists implemented models called perceptrons based on the work of McCulloch and Pitts. An example of a problem solved by these early networks was how to balance a broom standing upright on a moving cart by controlling the motions of the cart back and forth. As the broom starts falling to the left, the cart learns to move to the left to keep it upright. Although there were some limited successes with perceptrons in the laboratory, the results were disappointing as a general method for solving problems.

One reason for the limited usefulness of early neural networks is that most powerful computers of that era were less powerful than inexpensive desktop computers today. Another reason was that these simple networks had theoretical deficiencies, as shown by Seymour Papert and Marvin Minsky (two professors at the Massachusetts Institute of Technology) in 1968. Because of these deficiencies, the study of neural network implementations on computers slowed down drastically during the 1970s. Then, in 1982, John Hopfield of the California Institute of Technology invented back propagation, a way of training neural networks that sidestepped the theoretical pitfalls of earlier approaches.

A Bit of History

Team-Fly®