TR2016-088

Driver Confusion Status Detection Using Recurrent Neural Networks


    •  Hori, C., Watanabe, S., Hori, T., Harsham, B.A., Hershey, J.R., Koji, Y., Fujii, Y., Furumoto, Y., "Driver Confusion Status Detection Using Recurrent Neural Networks", IEEE International Conference on Multimedia and Expo (ICME), DOI: 10.1109/ICME.2016.7552966, July 2016.
      BibTeX TR2016-088 PDF
      • @inproceedings{Hori2016jul,
      • author = {Hori, Chiori and Watanabe, Shinji and Hori, Takaaki and Harsham, Bret A. and Hershey, John R. and Koji, Yusuke and Fujii, Youichi and Furumoto, Yuki},
      • title = {Driver Confusion Status Detection Using Recurrent Neural Networks},
      • booktitle = {IEEE International Conference on Multimedia and Expo (ICME)},
      • year = 2016,
      • month = jul,
      • doi = {10.1109/ICME.2016.7552966},
      • url = {https://www.merl.com/publications/TR2016-088}
      • }
  • MERL Contacts:
  • Research Areas:

    Artificial Intelligence, Speech & Audio

In this paper, we present a method for estimating the confusion level of a driver using a classifier trained on multimodal sensor data. Using the driver confusion status detector, a car navigation system can proactively support the driver when he/she is confused. A corpus of data was collected during onroad driving in traffic using a navigation system and a car instrumented with a variety of sensors. The data was manually annotated with the driver's confusion status and with multiple features representing driver's behavior and the traffic conditions. We compared different types of classifiers trained from the data: logistic regression, a feed-forward neural network, a recurrent neural networks, and a long short-term memory (LSTM)-based recurrent neural network. The accuracy was evaluated using F-max as well as precision/recall. We found that the LSTM outperformed the other models.