Applies to: NeuralTools 5.x and Up

How does NeuralTools address the problem of overfitting?

Overfitting occurs when neural net "memorizes" the training data instead of finding general patterns.

• With an overfitted net, the testing results on the training set are very good, but they are poor when the net is tested on data that was set aside for testing.

This potential problem is addressed in NeuralTools as follows:

• PNN nets are not prone to overfitting.
• For MLF nets, we recommend Best Net Search to avoid overfitting.
  • It starts with a small net with two nodes, which will probably not overfit.
  • It then tries bigger nets, to get to a point where the net is big enough to train properly, but not so big as to be capable of overfitting.

Sensitivity Analysis was introduced in NeuralTools version 6.0 which should help in diagnosing overfitting.

If we get overfitting with a given architecture, that will probably show in unstable results from Sensitivity Analysis.