Reinforcement Learning-based Estimation for Partial Differential Equations

    •  Mowlavi, S., Benosman, M., Nabi, S., "Reinforcement Learning-based Estimation for Partial Differential Equations", SIAM Conference on Applications of Dynamical Systems, May 2023.
      BibTeX TR2023-066 PDF
      • @inproceedings{Mowlavi2023may,
      • author = {Mowlavi, Saviz and Benosman, Mouhacine and Nabi, Saleh},
      • title = {Reinforcement Learning-based Estimation for Partial Differential Equations},
      • booktitle = {SIAM Conference on Applications of Dynamical Systems},
      • year = 2023,
      • month = may,
      • url = {}
      • }
  • MERL Contacts:
  • Research Areas:

    Dynamical Systems, Machine Learning, Optimization


In systems governed by nonlinear partial differential equations such as fluid flows, the design of state estimators such as Kalman filters relies on a reduced-order model (ROM) that projects the original high-dimensional dynamics onto a computationally tractable low-dimensional space. However, ROMs are prone to large errors, which negatively affects the performance of the estimator. Here, we in- troduce the reinforcement learning reduced-order estimator (RL-ROE), a ROM- based estimator in which the correction term that takes in the measurements is given by a nonlinear policy trained through reinforcement learning. The nonlinearity of the policy enables the RL-ROE to compensate efficiently for errors of the ROM, while still taking advantage of the imperfect knowledge of the dynamics. Using examples involving the Burgers and Navier-Stokes equations, we show that in the limit of very few sensors, the trained RL-ROE outperforms a Kalman filter designed using the same ROM. Moreover, it yields accurate high-dimensional state estimates for reference trajectories corresponding to various physical parameter values, without direct knowledge of the latter.


  • Related News & Events

    •  NEWS    Keynote address given by Philip Orlik at 9th annual IEEE Smartcomp conference
      Date: June 26, 2023
      Where: International Conference on Smart Computing (SMARTCOMP), Vanderbilt University, Nashville, Tennessee
      MERL Contact: Philip V. Orlik
      Research Areas: Communications, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Signal Processing
      • VP & Research Director, Philip Orlik, gave a keynote titled, "Smart Technologies for Smarter Buildings" at the 9th edition of the IEEE International Conference on Smart Computing (SMARTCOMP) focusing on some of the research challenges and opportunities that arise as we seek to achieve net-zero emissions in Smart building environments.

        SMARTCOMP is the premier conference on smart computing. Smart computing is a multidisciplinary domain based on the synergistic influence of advances in sensor-based technologies, Internet of Things, cyber-physical systems, edge computing, big data analytics, machine learning, cognitive computing, and artificial intelligence.