| • Science | • People | • Locations | • Timeline |
All modern connectionist models adhere to two major principles regarding the mind.
Connectionists generally stress the importance of learning in their models. As a result, many sophisticated learning procedures for neural networks have been developed by connectionists. Learning always involves modifying the connection weights. These generally involve mathematical formula to determine the change in weights when given sets of data consisting of activation vectors for some subset of the neural units.
By formalizing learning in such a way connectionists have many tools at their hands. A very common tactic in connectionist learning methods is to incorporate gradient descent over an error surface in a space defined by the weight matrix. All gradient descent learning in connectionist models involves changing each weight by the partial derivative of the error surface with respect to the weight. Backpropagation, first made popular in the 1980s, is probably the most commonly known connectionist gradient descent algorithm today.
The prevailing form of connectionist models today is known as Parallel Distributed Processing (PDP). PDP form became popular in the 1980s with the release of Parallel Distributed Processing: Explorations in the Microstructure of Cognition - Volume 1 (foundations) & Volume 2 (Psychological and Biological Models), by James L. McClelland , David E. Rumelhart , and the PDP Research Group. PDP's roots are the perceptron theories from the 1950s. In spite of the fact that as early as 1952 Friedrich Hayek posited the idea of spontaneous order in the brain arising out of decentralized networks of simple units ( neurons), Hayek's work was never cited in the literature of connectionism.
Another form of connectionist models is the Relational Network framework developed by the linguist Sydney Lamb in the 1960s. Relational Networks have only ever been used by linguists.
An earlier and rather different connectionistic view was held by Edward Thorndike, a turn of the century psychologist who studied learning, with his most famous contributions being work on how cats escaped from puzzle boxes, and his formulation of the Law of Effect. His analysis (and its descendants) are peppered with references to associations between stimuli and responses. Though the S-R aspect has today been abandoned by radical behaviorists and cognitive psychologists (including connectionists), it is easy to impose the notion of association and modification of association strength on connectionist models.
Connectionists are generally in agreement that recurrent networks (networks wherein connections of the network can form a directed cycle) are more like the human brain than feedforward networks (networks with no directed cycles). A lot of recurrent connectionist models incorporate dynamical systems theory as well. Many researchers, such as the connectionist Paul Smolensky (one of the authors of the original PDP books), have argued that the direction connectionist models will take is towards fully continuous, high-dimensional, non-linear dynamic systems approaches.