Home > Self-organizing map

1 A mathematical approach to the algorithm

1.1 Preliminary definitions

The actual training of the net is called vector quantization. "Unsupervised learning" is, technically, supervised in that we do have a desired output.

To explain the algorithm in-depth, let us create a 10x10 array of nodes. Each node will contain a weight vector, and will be omniscient of its "physical location", i.e. its location in the array. The weight vector each node contains will be of the same dimension as the following input vectors. (N.B. Weights in the nodes in the map are initially set to random values.)

Now we need input to feed the map. (Note: the generated map and the given input exist in separate subspaces!) Sticking with the norm, we will create three vectors to represent colors. In the world of computing, colors have the following three components: red, green, and blue. Consequently our input vectors will have three components, each one corresponding to a color space. Our input vectors will now be thus...

1.2 Some variables

Vectors are in bold

1.3 Stepping through the algorithm

1. Randomize the map's node's weight vectors
2. Grab an input vector
3. Traverse each node in the map
   1. Use Euclidean distance formula to find similarity
   2. Track the node that produces the smallest distance (this node will be called the Best Matching Unit (BMU)
4. Update the nodes in the neighbourhood of BMU by pulling them closer to the input vector
   1. Wv(t + 1) = Wv(t) + Θ(t)α(t)(D(t) - Wv(t))

2 An analytic approach to the algorithm

The SOM algorithm is fed with feature vectors , which can be of any dimension. In most applications, however, the number of dimensions will be high. Output maps can also be made in different dimensions: 1-dimensional, 2-dimensional, etc., but most popular are 2D and 3D maps, for SOMs are mainly used for dimensionality reduction rather than expansion.

The algorithm is explained most easily in terms of a set of artificial neurons, each having its own physical location on the output map, which take part in a winner-take-all process (a competitive network) where a node with its weight vector closest to the vector of inputs is declared the winner and its weights are adjusted making them closer to the input vector. Each node has a set of neighbours. When this node wins a competition, the neighbours' weights are also changed. They are not changed as much though. The further the neighbour is from the winner, the smaller its weight change. This process is then repeated for each input vector, over and over, for a number (usually large) of cycles. Different inputs produce different winners.

The network winds up associating output nodes with groups or patterns in the input data set. If these patterns can be named, the names can be attached to the associated nodes in the trained net.

Like most artificial neural networks, the SOM has two modes of operation:

1. During the training process a map is built, the neural network organises itself, using a competitive process. The network must be given a large number of input vectors, as much as possible representing the kind of vectors that are expected during the second phase (if any). Otherwise, all input vectors must be administered several times...
2. During the mapping process a new input vector may quickly be given a location on the map, it is automatically classified or categorised. There will be one single winning neuron: the neuron whose weight vector lies closest to the input vector. (This can be simply determined by calculating the Euclidean distance between input vector an weight vector.)