Table 7.1 Common Features Describing a House

Training the network builds a model which can then be used to estimate the target value for unknown examples. Training presents known examples (data from previous sales) to the network so that it can learn how to calculate the sales price. The training examples need two more additional features: the sales price of the home and the sales date. The sales price is needed as the target variable. The date is used to separate the examples into a training, validation, and test set. Table 7.2 shows an example from the training set.

The process of training the network is actually the process of adjusting weights inside it to arrive at the best combination of weights for making the desired predictions. The network starts with a random set of weights, so it initially performs very poorly. However, by reprocessing the training set over and over and adjusting the internal weights each time to reduce the overall error, the network gradually does a better and better job of approximating the target values in the training set. When the appoximations no longer improve, the network stops training.

Table 7.2 Sample Record from Training Set with Values Scaled to Range -1 to 1

This process of adjusting weights is sensitive to the representation of the data going in. For instance, consider a field in the data that measures lot size. If lot size is measured in acres, then the values might reasonably go from about Vs to 1 acre. If measured in square feet, the same values would be 5,445 square feet to 43,560 square feet. However, for technical reasons, neural networks restrict their inputs to small numbers, say between -1 and 1. For instance, when an input variable takes on very large values relative to other inputs, then this variable dominates the calculation of the target. The neural network wastes valuable iterations by reducing the weights on this input to lessen its effect on the output. That is, the first pattern that the network will find is that the lot size variable has much larger values than other variables. Since this is not particularly interesting, it would be better to use the lot size as measured in acres rather than square feet.

This idea generalizes. Usually, the inputs in the neural network should be smallish numbers. It is a good idea to limit them to some small range, such as -1 to 1, which requires mapping all the values, both continuous and categorical prior to training the network.

One way to map continuous values is to turn them into fractions by subtracting the middle value of the range from the value, dividing the result by the size of the range, and multiplying by 2. For instance, to get a mapped value for

Year Built (1923), subtract (1850 + 1986)/2 = 1918 (the middle value) from 1923 (the year the oldest house was built) and get 7. Dividing by the number of years in the range (1986 - 1850 + 1 = 137) yields a scaled value and multiplying by 2 yields a value of 0.0730. This basic procedure can be applied to any continuous feature to get a value between -1 and 1. One way to map categorical features is to assign fractions between -1 and 1 to each of the categories. The only categorical variable in this data is Heating Type, so we can arbitrarily map B 1 and A to -1. If we had three values, we could assign one to -1, another to 0, and the third to 1, although this approach does have the drawback that the three heating types will seem to have an order. Type -1 will appear closer to type 0 than to type 1. Chapter 17 contains further discussion of ways to convert categorical variables to numeric variables without adding spurious information.

With these simple techniques, it is possible to map all the fields for the sample house record shown earlier (see Table 7.2) and train the network. Training is a process of iterating through the training set to adjust the weights. Each iteration is sometimes called a generation.

Once the network has been trained, the performance of each generation must be measured on the validation set. Typically, earlier generations of the network perform better on the validation set than the final network (which was optimized for the training set). This is due to overfitting, (which was discussed in Chapter 3) and is a consequence of neural networks being so powerful. In fact, neural networks are an example of a universal approximator. That is, any function can be approximated by an appropriately complex neural network. Neural networks and decision trees have this property; linear and logistic regression do not, since they assume particular shapes for the underlying function.

As with other modeling approaches, neural networks can learn patterns that exist only in the training set, resulting in overfitting. To find the best network for unseen data, the training process remembers each set of weights calculated during each generation. The final network comes from the generation that works best on the validation set, rather than the one that works best on the training set.

When the models performance on the validation set is satisfactory, the neural network model is ready for use. It has learned from the training examples and figured out how to calculate the sales price from all the inputs. The model takes descriptive information about a house, suitably mapped, and produces an output. There is one caveat. The output is itself a number between 0 and 1 (for a logistic activation function) or -1 and 1 (for the hyperbolic tangent), which needs to be remapped to the range of sale prices. For example, the value 0.75 could be multiplied by the size of the range ($147,000) and then added to the base number in the range ($103,000) to get an appraisal value of $213,250.