![]() |
|
|||
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
|
![]() |
[an error occurred while processing this directive]
In terms of the consensus function, the stationary distribution of a Boltzmann machine can be written as: where Csmax is the maximum consensus over all states. Annealing refers to T → 0 in the stationary distribution πs. This reduces πs to a uniform distribution over all the maximum consensus states which is the reason for the Boltzmann machine being capable of performing global optimization. If the solution of binary output is described probabilistically, the output value oi is set to one with the probability where ΔEi is the change in energy identifiable as NETi and T is the pseudo-temperature. Given a current slate i with energy Ei, then a subsequent state j is generated by applying a small disturbance to transform the current state into a next state with energy Ej. If the difference (Ej - Ei) = ΔEi is less than or equal to zero, the state j is accepted as the current state. If ΔEj > 0, the state j is accepted with a probability as given above. This rate of probability acceptance is also known as the Metropolis criterion. Explicitly, this acceptance criterion determines whether j is accepted from i with a probability: where f(i), f(j) are cost-functions (equivalent of energy of a state) in respect to solutions i and j; and Cp denotes a control parameter (equivalent to the role played by temperature). The Metropolis algorithm indicated above differs from the Boltzmann machine in that the transition matrix is defined in terms of some (positive) energy function Es over S. Assuming Es ≥ Es′: It should be noted that pss′ ≠ ps′s. The difference is determined by the intrinsic ordering on {s, s′} induced by the energy function. Following the approach due to Akiyama et al. [54], a machine can in general be specified by three system parameters, namely, a reference activation level, ao; pseudo-temperature, T; and discrete time-step, Δt. Thus, the system parameter space for a Boltzmann machine is S(ao = 0, T, Δt = 1), with the output being a unit step-function. The distribution of the output oi is specified by the following moments: where Φ(x) is the standard cumulative gaussian distribution defined by: It may be noted that oi being binary, <oi> refers to the probability of oi equal to 1; and the cumulative gaussian distribution is a sigmoid which matches the probability function of the Boltzmann machine, defined by Equation (4.3). As indicated before, the Boltzmann machine is a basic model that enables a solution to a combinational problem of finding the optimal (the best under constrained circumstances) solution among a countably infinite number of alternative solutions. (Example: The traveling salesman problem*.)
In the Metropolis algorithm (as the Boltzmanns acceptance rule), if the lowering of the temperature is done adequately slowly, the network reaches thermal equilibrium at each pseudo-temperature as a result of a large number of transitions being generated at a given temperature value. This thermal equilibrium condition is decided by Boltzmann distribution, which as indicated earlier refers to the probability of state i with energy Ei at a pseudo-temperature T. It is given by the following conjecture: where Z(T) is the partition function defined as Z(T) = Σexp[Ej/(kBT)] with the summation over all possible states. It serves as the normalization constant in Equation (4.13). 4.3.2 McCulloch-Pitts MachineSince the McCulloch-Pitts model has a binary output with a deterministic decision and instantaneous activation in time, its machine parameter space can be defined by: Sm (ao = 0, T = 0, Δt = 1). The corresponding output is a unit step function, assuming a totally deterministic decision. 4.3.3 Hopfield MachineContrary to the McCulloch-Pitts model, the Hopfield machine has a graded output with a deterministic decision but with continuous (monotonic) activation in time. Therefore, its machine parameter is Sm (ao, 0, Δt). If the system gain approaches infinity, then the machine parameter becomes Sm(0, 0, 0). The neuron model employed in the generalized delta rule* is described by Sm(ao, 0, 1) , since it is a discrete time version of the Hopfield machine.
4.3.4 Gaussian MachineAkiyama et al. [54] proposed a machine representation of a neuron and termed it as a gaussian machine which has a graded response like the Hopfield machine, and behaves stochastically as the Boltzmann machine. Its output is influenced by a random noise added to each input and as a result forms a probabilistic distribution. The relevant machine parameters allow the system to escape from local minima. The properties of the gaussian machine are derived from the normal distribution of random noise added to the neural input. The machine parameters are specified by Sm(ao, T, Δt.). The other three machines discussed earlier are special cases of the Gaussian machine as depicted by the system parameter space shown in Figure 4.2.
Copyright © CRC Press LLC
![]() |
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |