EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


5.16 Langevin Machine versus Boltzmann Machine

In Boltzmann machines, the neurons change state in a statistical rather than a deterministic fashion. That is, these machines function by adjusting the states of the units (neurons) asynchronously and stochastically so as to minimize the global energy. The presence of noise is used to escape from the local minima. That is, as discussed in Chapter 4, occasional (random) jumps to configurations of higher energy are allowed so that the problem of stabilizing to a local rather than a global minimum (as suffered by Hopfield nets) is largely avoided. The Boltzmann machine rule of activation is decided probabilistically so that the output value σi is set to one with the probability p(σi = 1), where p is given in Equation (4.8), regardless of the current state. As discussed in Chapter 4 Akiyama et al. [54] point out that the Boltzmann machine corresponds to the Gaussian machine in that the sigmoidal characteristics of p fit very well to the conventional gaussian cumulative distribution with identical slope at the input σi = 0 with an appropriate choice of the scaling parameter.

The Boltzmann function, namely, {1/[1 + exp(-x)]} and the generalized Langevin function [1 + Lq(x)]1/2 represent identical curves with a slope of +1/4 at x = 0, if q is taken as +4. Therefore, inasmuch as the Boltzmann machine can be matched to the gaussian machine, the Langevin function can also be matched likewise; and, in which case, it is termed here as the Langevin machine.

Considering neural network optimization problems, sharpening schedules which refer to the changes in the reference level with time adaptively are employed in order to achieve better search methods. Such a scheduling scheme using Langevin machine strategies is also possible and can be expressed as:

where a0 is the reference activation level which is required to decrease over time. A0 is the initial value of a0, and τa0 is the time constant of the sharpening schedule.

Using the Langevin machine, the annealing can also be implemented by the following scheme:

where T0 is the initial temperature and τTn is the time constant of the annealing schedule which may differ from τa0. By proper choice of q, the speed of annealing can be controlled.

5.17 Concluding Remarks

The formal theory of stochastic neural network is based heavily on statistical mechanics considerations. However, when a one-to-one matching between real neuronal configuration and stochastical neural network (evolved from the principles of stochastical mechanics) is done, it is evident that there are as many contradictions and inconsistencies as the analogies that prevail in such a comparison. The analogies are built on the common notion, namely, the interactive collective behavior of the ensemble of units — the cells in the neural complex and the magnetic spins in the material lattice. The inconsistencies blossom from the asymmetric synaptic coupling of the real neurons as against the inherently symmetric attributes of magnetic spin connection strengths.

Hopfield’s ingenious, “step backward” strategry of incorporating a symmetry in his neural models and the pros and cons discussions and deliberations by Little and Perelto still, however, dwell on the wealth of theoretical considerations pertinent to statistical mechanics consistent with the fact that a “reasearch under a paradigm must be a particularly effective way of inducing a paradigm change”.


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.