![]() |
|
|||
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
|
![]() |
[an error occurred while processing this directive]
which explicitly specifies no spatial correlation between the states α1 and α2. However, if the maximum eigenvalue λmax is degenerate, the above factorization of Γ(α1, α2) is not possible and there will be a spatial correlation in the synaptic transmission behavior. Such a degeneracy (in spatial order) can be attributed to any possible transition from isotropic to anisotropic nematic phase in the neuronal configuration. That is, in the path of synaptic transmission, should there be a persistent or orientational linkage/interaction of neurons, the degeneracy may automatically set in. In the spin system, a similar degeneracy refers to the transition from a paramagnetic to ferromagnetic phase. In a neural system, considering the persistency in the time-domain, Little [33] observes that long-range time-ordering is related to the short-term memory considerations as dictated by intracellular biochemical process(es). 5.15 Langevin MachineThe integrated effect of all the excitatory and inhibitory postsynaptic axon potentials in a neuronal network, which decides the state transition (or firing), is modeled conventionally by a network with multi-input state-vectors Si (i = 1, 2, , M) with corresponding outputs σj (j = 1, 2, , N) linked via N-component weight-states Wij and decided by a nonlinear activation function. The corresponding input-output relation is specified by: where θi is an external (constant) bias parameter that may exist at the input and ξn is the error (per unit time) due to the inevitable presence of noise in the cellular regions. The input signal is further processed by a nonlinear activation function FS to produce the neurons output signal, σ. That is, each neuron randomly and asynchronously evaluates its inputs and readjusts σi accordingly. The justification for the above modeling is based on Hopfields [31,36] contention that real neurons have continuous, monotonic input-output relations and integrative time-delays. That is, neurons have sigmoid (meaning S-shaped) input-output curves of finite steepness rather than the steplike, two-state response curve or the logical neuron model suggested by McCulloch and Pitts [7]. The commonly used activation function to depict the neuronal response as mentioned earlier is the hyperbolic tangent given by FS(Λσi) = tanh(Λσi) where Λ is a gain/scaling parameter. It may be noted that as Λ tends to infinity, FS(Λσi) becomes the signum function indicating the all-or-none response of the McCulloch-Pitts model. Stornetta and Hubernian [84] have noted about the training characteristics of back-propagation networks that the conventional 0-to-1 dynamic range of inputs and hidden neuron outputs is not optimum. The reason for this surmise is that the magnitude of weight adjustment is proportional to the output level of the neuron. Therefore, a level of 0 results in no weight modification. With binary input vectors, half the inputs, on the average, will, however, be zero and the weights they connect to will not train. This problem is solved by changing the input range to ±1/2 and adding a bias to the squashing function to modify the neuron output range to ±1/2. The corresponding squashing function is as follows: which is again akin to the hyperbolic tangent and/or exponential function forms discussed earlier. These aforementioned sigmoids are symmetrical about the origin and have bipolar limiting values. They were chosen on an empirical basis, purely on the considerations of being S-shaped. That is, by observation, they match Hopfields model in that the output variable for the ith neuron is a continuous and monotone-increasing function of the instantaneous input to ith neuron having bipolar limits. In Section 5.12, however, the Langevin function has been shown to be the justifiable sigmoid on the basis of stochastical attributions of neuronal activity and the implications of using Langevin function in lieu of the conventional sigmoid in the machine description of neuronal activity are discussed in the following section. Such a machine is designated as the Langevin machine.
Copyright © CRC Press LLC
![]() |
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |