Abstract:

In this paper, we develop a novel solution to the output-feedback adaptive optimal control problem of general nonlinear nonaffine systems based on reinforcement learning (RL). This bridges a gap between RL and nonlinear output- feedback control theory by designing high-fidelity data-driven controllers to achieve disturbance rejection in an optimal sense. Based on the condition of uniform observability, the state is reconstructed by the retrospective input and output information, which can be seen as equivalent to a deadbeat observer. Both policy and value iteration algorithms are proposed to learn the optimal output-feedback control policy and value function. The convergence of the proposed algorithms and the practical stability of the closed-loop system with learned control policy are rigorously ensured even when the optimal value function is not positive definite.