Analysis Element

The actions taken on the problem environment (input to the problem environment) and the subsequent changes to the problem environment (output of the problem environment) are evaluated using a computer model. If the model-predicted response of the problem environment matches the actual response of the problem environment (when the actual and model-simulated environments have been subjected to the same forcing function), then the system parameters are assumed to be unchanged. However, if the two environment responses do not match, then the parameters are considered to have changed.

The analysis element uses the computer model to evaluate the real-world system. It applies the same forces to a simulated cart-pole system that are being sent from the control element to the actual problem environment. Thus, the response of the computer model should be the same as the response of the real-world system. When it is not, then something in the real-world system has changed. Figure 9.4 shows a schematic of this concept. Note in the figure that at time t_c the response of the actual system shown on the left no longer matches the predicted response of the model as shown on the right.

Figure 9.4  When the response of the real-world system differs substantially from the response predicted by the analysis element’s model simulation, it is assumed that the system parameters have changed. In the schematic above, this occurs at time t_c.

When the above approach is adopted, the problem of computing new system parameters becomes a curve fitting problem (Karr, 1991). The parameters associated with the computer model produce a unique response to a force, or action variables. Parameters must be selected by the analysis element so that the response of the model matches the response of the actual problem environment.

The GA used in the analysis element employs the familiar concatenated, mapped, unsigned binary coding scheme (Goldberg, 1989). The bit-strings produced by this coding strategy were of length 60: fifteen bits for each of the four system parameters that had to be represented, m, l, μ_c, and μ_p. The fitness function can be developed using the idea of curve fitting. Data exists describing the response of the real-world system. Thus, the GA must select the combination of system parameters that elicit this response for the given forcing function. Thus, the GA is looking for parameter values that will produce a series of states (x, , θ, and, ) that match those in the real-world system.

This concept is by the following fitness function:

where w₁ = 1.0 and w₂ = 10.0 are weighting constants that adjust for the fact that the values of x are generally an order of magnitude larger than the values of θ. With this definition of the fitness function, the problem becomes a minimization problem: the GA must minimize f, which, as it has been defined, represents the difference between the response predicted by the model and the response of the physical cart-pole system. When f (which is basically an error term) is driven to zero, then the response of the model exactly matches the response of the real-world system. Thus, it is assumed that the model is now working with an acceptable set of system parameters, a set that accurately reflects the real world.

Figure 9.5 compares the response of the cart-pole system being manipulated by the control element to the response of the cart-pole as simulated using parameters determined by the analysis element’s GA. This figure shows that the responses of the two systems are virtually identical, thereby demonstrating the effectiveness of a GA in this application. The GA was able to locate the correct parameters after only 500 function evaluations, where a function evaluation consisted of simulating the cart-pole system for 10 seconds. Locating the correct parameters (the mass in the cart system was 2.0 kg; the GA computed a value of 1.992 kg) took approximately 10 seconds on a pentium-based personal computer.

It is important to note here that in the current situation, the “real-world” cart-pole system is itself a simulation. Therefore, we are able to stop time in the real-world system while we implement the analysis element. If we were working with a real cart-pole system, then the 10 seconds needed by the analysis element would be too long if the change in the system parameter is rapid; we could not figure out the magnitude of the changes, much less update our control strategies, before the pole would come crashing down. Industrial systems may mandate that a new control action be learned in less than 10 seconds. In such cases, the time the GA is allotted to update the model parameters can be restricted. However, there are numerous industrial systems in which the system parameters do not change rapidly relative to the requisite 10 seconds in the cart-pole system. It is these systems that are best suited to the adaptive control system described.

Once new parameters (and thus the new response characteristics of the problem environment) have been determined, the learning element must alter the control element. An updated control strategy must be developed that is effective at manipulating the new problem environment.

Figure 9.5  The analysis element employs a GA to determine the correct values of system parameters. The fact that the response of the real-world system and the response of the simulated system are virtually identical indicates that the analysis element successfully accomplishes its assigned task.