EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


which explicitly specifies no spatial correlation between the states α1 and α2. However, if the maximum eigenvalue λmax is degenerate, the above factorization of Γ(α1, α2) is not possible and there will be a spatial correlation in the synaptic transmission behavior. Such a degeneracy (in spatial order) can be attributed to any possible transition from isotropic to anisotropic nematic phase in the neuronal configuration. That is, in the path of synaptic transmission, should there be a persistent or orientational linkage/interaction of neurons, the degeneracy may automatically set in. In the spin system, a similar degeneracy refers to the transition from a paramagnetic to ferromagnetic phase. In a neural system, considering the persistency in the time-domain, Little [33] observes that long-range time-ordering is related to the short-term memory considerations as dictated by intracellular biochemical process(es).

5.15 Langevin Machine

The integrated effect of all the excitatory and inhibitory postsynaptic axon potentials in a neuronal network, which decides the state transition (or “firing”), is modeled conventionally by a network with multi-input state-vectors Si (i = 1, 2, …, M) with corresponding outputs σj (j = 1, 2, …, N) linked via N-component weight-states Wij and decided by a nonlinear activation function. The corresponding input-output relation is specified by:

where θi is an external (constant) bias parameter that may exist at the input and ξn is the error (per unit time) due to the inevitable presence of noise in the cellular regions.

The input signal is further processed by a nonlinear activation function FS to produce the neuron’s output signal, σ. That is, each neuron randomly and asynchronously evaluates its inputs and readjusts σi accordingly.

The justification for the above modeling is based on Hopfield’s [31,36] contention that real neurons have continuous, monotonic input-output relations and integrative time-delays. That is, neurons have sigmoid (meaning S-shaped) input-output curves of finite steepness rather than the steplike, two-state response curve or the logical neuron model suggested by McCulloch and Pitts [7].

The commonly used activation function to depict the neuronal response as mentioned earlier is the hyperbolic tangent given by FS(Λσi) = tanh(Λσi) where Λ is a gain/scaling parameter. It may be noted that as Λ tends to infinity, FS(Λσi) becomes the signum function indicating the “all-or-none” response of the McCulloch-Pitts model.

Stornetta and Hubernian [84] have noted about the training characteristics of back-propagation networks that the conventional 0-to-1 dynamic range of inputs and hidden neuron outputs is not optimum. The reason for this surmise is that the magnitude of weight adjustment is proportional to the output level of the neuron. Therefore, a level of 0 results in no weight modification. With binary input vectors, half the inputs, on the average, will, however, be zero and the weights they connect to will not train. This problem is solved by changing the input range to ±1/2 and adding a bias to the squashing function to modify the neuron output range to ±1/2. The corresponding squashing function is as follows:

which is again akin to the hyperbolic tangent and/or exponential function forms discussed earlier.

These aforementioned sigmoids are symmetrical about the origin and have bipolar limiting values. They were chosen on an empirical basis, purely on the considerations of being S-shaped. That is, by observation, they match Hopfield’s model in that the output variable for the ith neuron is a continuous and monotone-increasing function of the instantaneous input to ith neuron having bipolar limits.

In Section 5.12, however, the Langevin function has been shown to be the justifiable sigmoid on the basis of stochastical attributions of neuronal activity and the implications of using Langevin function in lieu of the conventional sigmoid in the machine description of neuronal activity are discussed in the following section. Such a machine is designated as the Langevin machine.


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.