TR2020-172

The Missing Input Problem


    •  Laftchiev, E., Yan, Q., Nikovski, D.N., "The Missing Input Problem", IEEE Big Data, December 2020.
      BibTeX TR2020-172 PDF
      • @inproceedings{Laftchiev2020dec,
      • author = {Laftchiev, Emil and Yan, Qing and Nikovski, Daniel N.},
      • title = {The Missing Input Problem},
      • booktitle = {IEEE Big Data},
      • year = 2020,
      • month = dec,
      • url = {https://www.merl.com/publications/TR2020-172}
      • }
  • MERL Contact:
  • Research Areas:

    Data Analytics, Machine Learning

Rapid advances in information and communications technologies (ICT) have made it possible to deploy large collections of sensors to be used in traditional Supervisory Control and Data Acquisition (SCADA) systems and modern Internet of Things (IoT) installations. These sensors are intended for use in analytical formulas, AI algorithms, and traditional rulebased monitoring that determine optimal operation parameters, maintain smooth operation, or detect operation anomalies. Yet, advances in ICT do not always improve the reliability of data collection. Instead, the frequent use of consumer-grade sensors in IoT deployments, and an increasing array of customer choices often lead to inaccessibility of sensors or systematic absence of sensor readings. The lack of reliability in data collection leads to failures in the algorithms responsible for monitoring the system operation. We term this ”the missing input problem”, and discuss several state-of-the-art solutions. We specifically focus on the straightforward approach using standard imputation methods, as well as recent deep learning imputation methods. We show that none of the existing algorithms today perform very well in the face of missing sensors, and we outline several research directions that can lead to improvements.