TR2012-083

Markov Decision Processes for Train Run Curve Optimization


    •  Nikovski, D.; Lidicky, B.; Zhang, W.; Kataoka, K.; Yoshimoto, K., "Markov Decision Processes for Train Run Curve Optimization", Electrical Systems for Aircraft, Railway and Ship Propulsion (ESARS), DOI: 10.1109/ESARS.2012.6387473, ISSN: 2165-9400, ISBN: 978-1-4673-1370-4, October 2012, pp. 1-6.
      BibTeX Download PDF
      • @inproceedings{Nikovski2012oct,
      • author = {Nikovski, D. and Lidicky, B. and Zhang, W. and Kataoka, K. and Yoshimoto, K.},
      • title = {Markov Decision Processes for Train Run Curve Optimization},
      • booktitle = {Electrical Systems for Aircraft, Railway and Ship Propulsion (ESARS)},
      • year = 2012,
      • pages = {1--6},
      • month = oct,
      • doi = {10.1109/ESARS.2012.6387473},
      • issn = {2165-9400},
      • isbn = {978-1-4673-1370-4},
      • url = {http://www.merl.com/publications/TR2012-083}
      • }
  • MERL Contact:
  • Research Areas:

    Data Analytics, Predictive Modeling


We propose three computationally efficient methods for finding optimal run curves of electrical trains, all based on the idea of approximating the continuous dynamics of a moving train by a Markov Decision Process (MDP) model. Deterministic continuous train dynamics are converted to stochastic transitions on a discrete model by observing the similarity between the properties of convex combinations and those of probability mass functions. The resulting MDP uses barycentric coordinates to effectively represent the cost-to-go of the approximated optimal control problem. One of the three solution methods uses equal distance steps, as opposed to the usual equal- time steps, to avoidself transitions of the MDP, which allows very fast computation of the cost-to-go in one pass only.