![]() |
|
|||
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
|
![]() |
[an error occurred while processing this directive]
4.11 Learning from Examples Generated by a PerceptronThe perceptron, in general, refers to a system with an input-output relation dictated by a nonlinear squashing function. The learning rule of a test perceptron corresponds to a target to be learned as in a reference perceptron with {x′j} inputs, {W′ij} coupling weights resulting in the output o′j. The corresponding sets for the test perceptron are taken as {xj} and {Wij}. Because xj and x′j are not identical, a correlation coefficient can be specified in terms of a joint gaussian distribution of the variates as indicated below: where σx2 is the variance of the inputs. Since the output is binary, the associated error measure eo(x) is also binary. That is, eo(x) = 0 or 1, if x < 0, or x > 0, respectively. The corresponding total training error is therefore: which permits an explicit determination of the partition function (ZM) defined by Equation (4.37). The ensemble average of the partition function can also be represented in terms of an average of a logarithm, converted to that of a power as follows: In the case of completely random examples, <ZnM>En bifurcates into two parts, one being a power of the connectivity N and the other that of the number M of the training examples. Further, in reference to the correlation coefficients (Equation 4.43), the averaging process leads to two quantities which characterize the ensemble of interconnected cells. They are:
where δab is the Kronecker delta. In the dynamics of neural network training, the basic problem is to find the weighting parameters Wij for which a set of configuration (or patterns) {ξiμ} (i = 1, 2, 3, ..., N; μ = 1, 2, 3, ..., p) are stationary (fixed) points of the dynamics. There are two lines of approach to this problem. In the first approach, the Wij are given a specific storage (or memory) prescription. The so-called Hebbs rule which is the basis of Hopfields model essentially follows this approach. Another example of this strategy is the pseudo-inverse rule due to Kohonen which has been applied to Hopfields net by Personnaz et al. [62] and studied analytically by Kanter and Sompolinsky [63]. Essentially, Wij are assumed as symmetric (that is, Wij = Wji) in these cases. In the event of a mixed population of symmetric and asymmetric weights, asymmetry parameters ηs can be defined as follows: or equivalently: where Wsy,asy = 1/2(Wij ± Wji) are the symmetric and asymmetric components of Wij, respectively. When ηs = 1, the matrix is fully symmetric and when ηs = -1, it is fully asymmetric. When ηs = 0, Wij and Wji are fully correlated on the energy, implying that the symmetric and asymmetric components have equal weights.*
As mentioned earlier, the network training involves finding a set of stationary points which affirm the convergence towards the target configuration or pattern. The two entities whose stationary values are relevant to the above purpose are the overlap parameters, namely, q and R. The stationary values of q and R can be obtained via replica symmetry ansatz solution (see Appendix C).
Copyright © CRC Press LLC
![]() |
![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |