|
 |
[an error occurred while processing this directive]
In essence, Griffiths objection to symmetry in the synaptic weight space has stemmed from his nonconcurrence with the theory proposed by Cragg and Temperley to consider the neural networks as aggregates of interacting spins (as in ferromagnetic materials). In enunciating a correspondence between the neural networks and magnetic spin systems, as done by Cragg and Temperley, it was Griffiths opinion that the Hamiltonian of the neural assembly is totally unlike a ferromagnetic Hamiltonian ... the (neural) Hamiltonian has the undesirable features of being intractably complicated and also non-hermitian. ... [hence] the original analogy [between neural network and magnetic spin system] is invalid. ... This appears to reduce considerably the practical value of any such analogy.
Notwithstanding the fact that the spin-glass analogy extended to neuronal activity was regarded by Griffith as having no practical value, a number of studies have emerged in the last two decades either to justify the analogy or to use the relevant parallelism between the spin-glass theory and neural dynamics in artificial neural networks. Such contributions have stemmed from cohesive considerations related to statistical physics, neurobiology, cognitive and computer sciences, and relevant topics which cover the general aspects of time-dependent problems, coding and retrieval considerations, hierarchical organization of neural systems, biological motivations in modeling neural networks analogous to spin glasses, and other related problems have been developed. The analogy of the neural complex with spin systems had become an important topic of interest due to the advances made in understanding the thermodynamic properties of disordered systems of spins, the so-called spin glasses over the last scores of years. When the pertinent results are applied to neural networks, the deteministic evolution law of updating the network output is replaced by a stochastic law where the state variable of the cell (at a new instant of time) is assigned according to a probabilistic function depending on the intensity of the synaptic input. This probabilistic function is dictated by the pseudo-temperature concepts outlined in Chapter 4. The stochastical evolution law pertains to the features of real neurons wherein spontaneous firing without external excitation may be encountered leading to a persistent noise level in the network.
Among the existing studies, more basic considerations into the one-to-one analogy of spin-glass theory and neuronal activity were considered exclusively in detail by Little [33] and by a number of others, a chronological summary of which is presented in the following sections.
5.4 Littles Model
Subsequent to Griffiths verdict on the spin-glass model of the neural complex, Little in 1974 [33] demonstrated the existence of persistent states of firing patterns in a neural network when a certain transfer matrix has approximately degenerate maximum eigenvalues.* He demonstrated a direct analogy of the persistence in neuronal firing patterns (considered in discrete time-steps) to the long-range spatial order in an Ising spin crystal system; and the order-disorder situations in the crystal lattice are dictated by the thermodynamic considerations specified by the system temperature T. The ordered phase of the spin system occurs below a critical temperature (TC), well known as the Curie point. Analogously, a factor (β) representing the temperature of the neural network is assumed in Littles model for the transfer matrix that depicts the persistent states. The approach envisaged by Little can be summarized as follows.
*A matrix which has no two eigenvalues equal and which has, therefore, just as many distinct eigenvalues as its dimension is said to be nondegenerate. That is, if more than one eigenvector has the same eigenvalue, the matrix is degenerate.
|