Machine Learning Based State-Space Approximate Dynamic Programming Approach for Energy and Reserve Management of Power Plants

This paper proposes a machine learning based state-space approximate dynamic programming (MSADP) approach to solve the self-scheduling problem faced by power plants under an integrated energy and reserve market. By extending the concept of residual demand curves (RDCs) from energy to reserve, the residual reserve curves (RRCs) is proposed to model the regulation price as a function of the power plant's reserve power. Both RRCs and RDCs are obtained using a clustering based neural network approach, which resulted in better estimates than using only a non-parametric approach. The machine learning is used to make approximations to the state space, and the dynamic programming only loops over the required states. As such, the computation effort is reduced but the solution quality does not be impacted. The value functions generated during the day-ahead optimization are used to generate optimal supply offer curves for the day-ahead market, and make real-time decisions and real-time bids to stay optimal by solving Bellman optimality condition. The effectiveness of the MSADP approach is demonstrated using empirical data obtained from the New England ISO.