Learning to Control Partial Differential Equations: Regularized Fitted Q-Iteration Approach

This paper formulates a class of partial differential equation (PDE) control problems as a reinforcement learning (RL) problem. We design an RL-based algorithm that directly works with the state of PDE, an infinite dimensional vector, thus allowing us to avoid the model order reduction, commonly used in the conventional PDE controller design approaches. We apply the method to the problem of flow control for time-varying 2D convection-diffusion PDE, as a simplified model for heating, ventilating, air conditioning (HVAC) control design in a room.