Simulation Failure Robust Bayesian Optimization for Data-Driven Parameter Estimation


Advances in modeling and computation have resulted in high-fidelity digital twins capable of simulating the dynamics of a wide range of industrial systems. These simulation models often require calibration, or the estimation of an optimal set of parameters in some goodness-of-fit sense, to reflect a system’s observed behavior. While searching over the parameter space is an inevitable part of the calibration process, simulation models are rarely designed to be valid for arbitrarily large parameter spaces. Application of existing calibration methods, therefore, often results in repeated model evaluations using parameters that can cause the simulations to be impractically slow or even result in catastrophic failure. In general, the shape of subregions in the parameter space that could result in simulation failure is unknown. In this paper, we propose a novel failure- robust Bayesian optimization (FR-BO) algorithm that learns these failure regions from online simulations and informs a Bayesian optimization algorithm to avoid failure regions while optimizing model parameters. This results in acceleration of the optimizer’s convergence and prevents wastage of time trying to simulate parameters with high failure probabilities. The effectiveness of the proposed failure-robust Bayesian optimization algorithm is demonstrated via a well-known benchmark example where we compare against state-of-the-art gradient matching techniques, and a practical example related to parameter es- timation for digital twins of buildings.


  • Related Publication

  •  Chakrabarty, A., Bortoff, S.A., Laughman, C.R., "Simulation Failure Robust Bayesian Optimization for Estimating Black-Box Model Parameters", IEEE International Conference on Systems, Man, and Cybernetics (SMC), DOI: 10.1109/​SMC52423.2021.9658893, October 2021.
    BibTeX TR2021-128 PDF Video
    • @inproceedings{Chakrabarty2021oct2,
    • author = {Chakrabarty, Ankush and Bortoff, Scott A. and Laughman, Christopher R.},
    • title = {Simulation Failure Robust Bayesian Optimization for Estimating Black-Box Model Parameters},
    • booktitle = {IEEE International Conference on Systems, Man, and Cybernetics (SMC)},
    • year = 2021,
    • month = oct,
    • doi = {10.1109/SMC52423.2021.9658893},
    • url = {}
    • }