Learning Element

The learning element alters the control element in response to changes in the problem environment. It does so by altering the membership functions (and in later chapters the rules) employed by the fuzzy controller. In this way, the control element is able to implement a new strategy that has been optimized for the new problem environment.

This task is equivalent to the fuzzy controller design problems presented in earlier chapters in which rules were discovered and membership functions were tuned to achieve a particular objective most effectively. The only difference is that the design is performed “on-line” (remember that for the cart-pole system, time is stopped while the analysis and learning elements accomplish their tasks) as opposed to the off-line designs performed earlier. The learning element has access to a computer model of the problem environment, it knows the current values of the system parameters in the real world, and it is also provided with the current state of the cart-pole system.

Altering the membership functions (the definition of the fuzzy terms in the rule set) is consistent with the way humans control complex systems. Quite often, the rules-of-thumb humans use to manipulate a problem environment remain the same despite even dramatic changes to that environment; only the meaning of the rules is altered.

The current adaptive control system uses a GA to alter the membership functions associated with a fuzzy controller, and this technique has been well documented. There is, however, one point that should be considered here. In the design of the cart-pole controller described in Chapter 7, the fitness function was developed to ensure that the controller could effectively drive the system to the desired setpoint from a variety of initial conditions. Here, the goal is to define membership functions that allow the controller to maintain the current state of the system. Thus, there is no need to look for membership functions that are effective from every possible initial condition case so the search problem is reduced dramatically. The fitness function employed allows for an increase in speed by a factor of four since it contains only one summation. The fitness function employed is:

Although the additional summation did not matter in the preceding off-line design problems, the associated increase in speed is important in the adaptive control systems we are now considering.

In systems for which time is of the essence, the search problem can be reduced further. For instance, if the real-world cart-pole system is in one region of state space, the search can be constrained to those membership functions that have non-zero values in that particular region of the state space. In this way, the GA’s task is much easier. Other than these two alternative considerations, the mechanics of the learning element are exactly as described in Chapter 7 and will not be repeated here.

9.4 Results

An adaptive controller for the time-varying cart-pole system has been developed. To demonstrate its effectiveness, we use a particular situation in which the mass of the cart changes according to Figure 9.6. Notice here the mass is allowed to change dramatically, but not rapidly. Thus, this version of the cart-pole system presents a nice test bed for the adaptive controller.

Figure 9.6  The cart mass changes as a function of time. The magnitude of the mass changes dramatically, but the mass is not changing rapidly. Thus, this problem provides a nice test bed for the adaptive controller.

Figure 9.7 shows the effectiveness of the adaptive fuzzy control system that accounted for changes in the cart mass. This adaptive system was able to avoid the catastrophic failures of the pole falling over or the cart striking a wall despite dramatic changes in cart mass by altering its membership functions “on-line.” Every time a change in cart mass was made, the analysis element recognized and quantified the changes. The mass changes were each identified to within 2 percent. The learning element employed a GA to locate new membership functions that were effective for the current state of the system. As can be seen in Figure 9.7, the adaptive GA-FC outperformed a non-adaptive FC that could not prevent the pole from falling over when the mass is altered.

Figure 9.7  The adaptive fuzzy controller is able to manage the changes in cart mass where the non-adaptive system holds firm to its rules and membership functions despite the fact that they are no longer valid for the altered system.