5. Feature discovery by competitive learning

Rumelhart, D. E., & Zipser, D. (1985). Feature discovery by competitive learning. Cognitive Science, 9, 75-112. Printable PDF

Abstract

This article describes a novel algorithm for training neural networks without requiring any external teacher or feedback. Instead, the neural network becomes trained on the basis of the statistics latent in the stimuli presented to the network. The algorithm works by starting with homogenous, undifferentiated detectors. When an input pattern is presented, the detector that is most similar to the pattern adapts itself so that it becomes even more tuned to the pattern. The remaining detectors are prevented from adapting to the pattern, leaving them available to become specialized to other patterns. In this manner, the algorithm achieves one of the primary goals of cognitive science - the creation of systems that organize themselves with training so that they exhibit richer structure than their original starting configuration. Sample simulations apply the algorithm to pattern formation and recognition, the automatic generation of categories and features, and the integration of data-driven and top-down influences on category formation.