MERL - Mitsubishi Electric Research Laboratories

Share:

Tim Marks

Tim Marks
Tim's Quick Links

  • Biography

    Prior to joining MERL's Imaging Group in 2008, Tim did postdoctoral research in robotic Simultaneous Localization and Mapping in collaboration with NASA's Jet Propulsion Laboratory. His research at MERL spans a variety of areas in computer vision and machine learning, including face recognition under variations in pose and lighting, and robotic vision and touch-based registration for industrial automation.

  • Recent News & Events

    •  NEWS   MERL's Scene-Aware Interaction Technology Featured in Mitsubishi Electric Corporation Press Release
      Date: July 22, 2020
      Where: Tokyo, Japan
      MERL Contacts: Anoop Cherian; Bret Harsham; Chiori Hori; Takaaki Hori; Jonathan Le Roux; Tim Marks; Alan Sullivan; Anthony Vetro
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
      Brief
      • Mitsubishi Electric Corporation announced that the company has developed what it believes to be the world’s first technology capable of highly natural and intuitive interaction with humans based on a scene-aware capability to translate multimodal sensing information into natural language.

        The novel technology, Scene-Aware Interaction, incorporates Mitsubishi Electric’s proprietary Maisart® compact AI technology to analyze multimodal sensing information for highly natural and intuitive interaction with humans through context-dependent generation of natural language. The technology recognizes contextual objects and events based on multimodal sensing information, such as images and video captured with cameras, audio information recorded with microphones, and localization information measured with LiDAR.

        Scene-Aware Interaction for car navigation, one target application, will provide drivers with intuitive route guidance. The technology is also expected to have applicability to human-machine interfaces for in-vehicle infotainment, interaction with service robots in building and factory automation systems, systems that monitor the health and well-being of people, surveillance systems that interpret complex scenes for humans and encourage social distancing, support for touchless operation of equipment in public areas, and much more. The technology is based on recent research by MERL's Speech & Audio and Computer Vision groups.


        Demonstration Video:



        Link:

        Mitsubishi Electric Corporation Press Release
    •  
    •  NEWS   MERL researchers presenting four papers and organizing two workshops at CVPR 2020 conference
      Date: June 14, 2020 - June 19, 2020
      MERL Contacts: Anoop Cherian; Michael Jones; Toshiaki Koike-Akino; Tim Marks; Kuan-Chuan Peng; Ye Wang
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Brief
      • MERL researchers are presenting four papers (two oral papers and two posters) and organizing two workshops at the IEEE/CVF Computer Vision and Pattern Recognition (CVPR 2020) conference.

        CVPR 2020 Orals with MERL authors:
        1. "Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction," by Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, Qi Tian
        2. "Collaborative Motion Prediction via Neural Motion Message Passing," by Yue Hu, Siheng Chen, Ya Zhang, Xiao Gu

        CVPR 2020 Posters with MERL authors:
        3. "LUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood," by Abhinav Kumar, Tim K. Marks, Wenxuan Mou, Ye Wang, Michael Jones, Anoop Cherian, Toshiaki Koike-Akino, Xiaoming Liu, Chen Feng
        4. "MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird’s Eye View Maps," by Pengxiang Wu, Siheng Chen, Dimitris N. Metaxas

        CVPR 2020 Workshops co-organized by MERL researchers:
        1. Fair, Data-Efficient and Trusted Computer Vision
        2. Deep Declarative Networks.
    •  

    See All News & Events for Tim

  • Awards

    •  AWARD   MERL Researchers win Best Paper Award at ICCV 2019 Workshop on Statistical Deep Learning in Computer Vision
      Date: October 27, 2019
      Awarded to: Abhinav Kumar, Tim K. Marks, Wenxuan Mou, Chen Feng, Xiaoming Liu
      MERL Contact: Tim Marks
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Brief
      • MERL researcher Tim Marks, former MERL interns Abhinav Kumar and Wenxuan Mou, and MERL consultants Professor Chen Feng (NYU) and Professor Xiaoming Liu (MSU) received the Best Oral Paper Award at the IEEE/CVF International Conference on Computer Vision (ICCV) 2019 Workshop on Statistical Deep Learning in Computer Vision (SDL-CV) held in Seoul, Korea. Their paper, entitled "UGLLI Face Alignment: Estimating Uncertainty with Gaussian Log-Likelihood Loss," describes a method which, given an image of a face, estimates not only the locations of facial landmarks but also the uncertainty of each landmark location estimate.
    •  
    See All Awards for MERL

  • Research Highlights

  • Internships with Tim

    • CV1567: Generative Adversarial Networks (GANs) for 3D face generation

      MERL is seeking a highly motivated intern to conduct original research in the area of generative adversarial networks for realistic 3D face generation. The successful candidate will collaborate with MERL researchers to design and implement new models, conduct experiments, and prepare results for publication. The candidate should be a PhD student in computer vision with experience in GANs and related deep learning methods, as well as good general knowledge in machine learning and a strong publication record. Previous experience with 3D face models and video generation is preferred. Strong programming skills in Python and flexibility working across various deep learning platforms (e.g., PyTorch and TensorFlow) are expected.

    • CV1568: Uncertainty Estimation in 3D Face Landmark Tracking

      We are seeking a highly motivated intern to conduct original research extending MERL's work on uncertainty estimation in face landmark localization (the LUVLi model) to the domains of 3D faces and video sequences. The successful candidate will collaborate with MERL researchers to design and implement new models, conduct experiments, and prepare results for publication. The candidate should be a PhD student in computer vision and machine learning with a strong publication record. Experience in deep learning-based face landmark estimation, video tracking, and 3D face modeling is preferred. Strong programming skills, experience developing and implementing new models in deep learning platforms such as PyTorch, and broad knowledge of machine learning and deep learning methods are expected.

    • CV1570: Health monitoring from video

      MERL is seeking a highly motivated intern to conduct original research in the area of monitoring vital signs, such as heart rate and heart rate variability, from video of a person. The successful candidate will collaborate with MERL researchers to derive and implement new models, collect data, conduct experiments, and prepare results for publication. The candidate should be a PhD student in computer vision with a strong publication record and experience in computer vision, signal processing, machine learning, and health monitoring. Strong programming skills (Python, Matlab, C/C++, etc.) are expected.

    See All Internships at MERL

  • MERL Publications

    •  Nowara, E., Marks, T., Mansour, H., Veeraraghavan, A., "Near-Infrared Imaging Photoplethysmography During Driving", IEEE Transactions on Intelligent Transportation Systems, December 2020.
      BibTeX TR2020-161 PDF
      • @article{Nowara2020dec,
      • author = {Nowara, Ewa and Marks, Tim and Mansour, Hassan and Veeraraghavan, Ashok},
      • title = {Near-Infrared Imaging Photoplethysmography During Driving},
      • journal = {IEEE Transactions on Intelligent Transportation Systems},
      • year = 2020,
      • month = dec,
      • url = {https://www.merl.com/publications/TR2020-161}
      • }
    •  Nowara, E., Marks, T., Mansour, H., Pai, A., Nagamatsu, G., Kawasaki, H., Veeraraghavan, A., "HeartCam: Camera-Based Physiology Monitoring in the Wild", IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 2020.
      BibTeX
      • @inproceedings{Nowara2020jun,
      • author = {Nowara, Ewa and Marks, Tim and Mansour, Hassan and Pai, Amruta and Nagamatsu, Genki and Kawasaki, Hiroshi and Veeraraghavan, Ashok},
      • title = {HeartCam: Camera-Based Physiology Monitoring in the Wild},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
      • year = 2020,
      • month = jun
      • }
    •  Kumar, A., Marks, T., Mou, W., Wang, Y., Cherian, A., Jones, M.J., Liu, X., Koike-Akino, T., Feng, C., "LUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: 10.1109/CVPR42600.2020.00826, June 2020.
      BibTeX TR2020-067 PDF Video Data Software
      • @inproceedings{Kumar2020jun,
      • author = {Kumar, Abhinav and Marks, Tim and Mou, Wenxuan and Wang, Ye and Cherian, Anoop and Jones, Michael J. and Liu, Xiaoming and Koike-Akino, Toshiaki and Feng, Chen},
      • title = {LUVLi Face Alignment: Estimating Landmarks’ Location, Uncertainty, and Visibility Likelihood},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2020,
      • month = jun,
      • publisher = {IEEE},
      • doi = {10.1109/CVPR42600.2020.00826},
      • issn = {2575-7075},
      • isbn = {978-1-7281-7168-5},
      • url = {https://www.merl.com/publications/TR2020-067}
      • }
    •  D’Haro, L.F., Yoshino, K., Hori, C., Marks, T., Polymenakos, L., Kummerfeld, J.K., Galley, M., Gao, X., "Overview of the seventh Dialog System Technology Challenge: DSTC7", Computer Speech and Language, DOI: 10.1016/j.csl.2020.101068, Vol. 62, March 2020.
      BibTeX TR2020-029 PDF
      • @article{D’Haro2020mar,
      • author = {D’Haro, Luis, Fernando and Yoshino, Koichiro and Hori, Chiori and Marks, Tim and Polymenakos, Lazaros and Kummerfeld, Jonathan, k. and Galley, Michel and Gao, Xiang},
      • title = {Overview of the seventh Dialog System Technology Challenge: DSTC7},
      • journal = {Computer Speech and Language},
      • year = 2020,
      • volume = 62,
      • month = mar,
      • doi = {10.1016/j.csl.2020.101068},
      • url = {https://www.merl.com/publications/TR2020-029}
      • }
    •  Cherian, A., Wang, J., Hori, C., Marks, T., "Spatio-Temporal Ranked-Attention Networks for Video Captioning", IEEE Winter Conference on Applications of Computer Vision (WACV), DOI: 10.1109/WACV45572.2020.9093291, February 2020, pp. 1606-1615.
      BibTeX TR2020-016 PDF
      • @inproceedings{Cherian2020feb,
      • author = {Cherian, Anoop and Wang, Jue and Hori, Chiori and Marks, Tim},
      • title = {Spatio-Temporal Ranked-Attention Networks for Video Captioning},
      • booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
      • year = 2020,
      • pages = {1606--1615},
      • month = feb,
      • publisher = {IEEE},
      • doi = {10.1109/WACV45572.2020.9093291},
      • url = {https://www.merl.com/publications/TR2020-016}
      • }
    See All Publications for Tim

  • Other Publications

    •  Tim K Marks, Andrew Howard, Max Bajracharya, Garrison W Cottrell and Larry H Matthies, "Gamma-SLAM: Visual SLAM in unstructured environments using variance grid maps", Journal of Field Robotics, Vol. 26, No. 1, pp. 26-51, 2009.
      BibTeX
      • @Article{marks2009gamma,
      • author = {Marks, Tim K and Howard, Andrew and Bajracharya, Max and Cottrell, Garrison W and Matthies, Larry H},
      • title = {Gamma-SLAM: Visual SLAM in unstructured environments using variance grid maps},
      • journal = {Journal of Field Robotics},
      • year = 2009,
      • volume = 26,
      • number = 1,
      • pages = {26--51},
      • publisher = {Wiley Online Library}
      • }
    •  Luke Barrington, Tim K Marks, Janet Hui-wen Hsiao and Garrison W Cottrell, "NIMBLE: A kernel density model of saccade-based visual memory", Journal of Vision, Vol. 8, No. 14, 2008.
      BibTeX
      • @Article{barrington2008nimble,
      • author = {Barrington, Luke and Marks, Tim K and Hsiao, Janet Hui-wen and Cottrell, Garrison W},
      • title = {NIMBLE: A kernel density model of saccade-based visual memory},
      • journal = {Journal of Vision},
      • year = 2008,
      • volume = 8,
      • number = 14,
      • publisher = {Association for Research in Vision and Ophthalmology}
      • }
    •  Tim K Marks, Andrew Howard, Max Bajracharya, Garrison W Cottrell and Larry Matthies, "Gamma-SLAM: Using stereo vision and variance grid maps for SLAM in unstructured environments", Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, 2008, pp. 3717-3724.
      BibTeX
      • @Inproceedings{marks2008gamma,
      • author = {Marks, Tim K and Howard, Andrew and Bajracharya, Max and Cottrell, Garrison W and Matthies, Larry},
      • title = {Gamma-SLAM: Using stereo vision and variance grid maps for SLAM in unstructured environments},
      • booktitle = {Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on},
      • year = 2008,
      • pages = {3717--3724},
      • organization = {IEEE}
      • }
    •  Lingyun Zhang, Matthew H Tong, Tim K Marks, Honghao Shan and Garrison W Cottrell, "SUN: A Bayesian framework for saliency using natural statistics", Journal of Vision, Vol. 8, No. 7, 2008.
      BibTeX
      • @Article{zhang2008sun,
      • author = {Zhang, Lingyun and Tong, Matthew H and Marks, Tim K and Shan, Honghao and Cottrell, Garrison W},
      • title = {SUN: A Bayesian framework for saliency using natural statistics},
      • journal = {Journal of Vision},
      • year = 2008,
      • volume = 8,
      • number = 7,
      • publisher = {Association for Research in Vision and Ophthalmology}
      • }
    •  Tim K Marks, Andrew Howard, Max Bajracharya, Garrison W Cottrell and Larry Matthies, "Gamma-SLAM: Stereo visual SLAM in unstructured environments using variance grid maps", IROS visual SLAM workshop, 2007.
      BibTeX
      • @Article{marks2007gamma,
      • author = {Marks, Tim K and Howard, Andrew and Bajracharya, Max and Cottrell, Garrison W and Matthies, Larry},
      • title = {Gamma-SLAM: Stereo visual SLAM in unstructured environments using variance grid maps},
      • journal = {IROS visual SLAM workshop},
      • year = 2007,
      • publisher = {Citeseer}
      • }
    •  Tim K Marks, John Hershey, J Cooper Roddey and Javier R Movellan, "Joint tracking of pose, expression, and texture using conditionally Gaussian filters", Advances in neural information processing systems, Vol. 17, pp. 889-896, 2005.
      BibTeX
      • @Article{marks2005joint,
      • author = {Marks, Tim K and Hershey, John and Roddey, J Cooper and Movellan, Javier R},
      • title = {Joint tracking of pose, expression, and texture using conditionally Gaussian filters},
      • journal = {Advances in neural information processing systems},
      • year = 2005,
      • volume = 17,
      • pages = {889--896}
      • }
    •  Tim K Marks, John Hershey, J Cooper Roddey and Javier R Movellan, "3d tracking of morphable objects using conditionally gaussian nonlinear filters", Computer Vision and Pattern Recognition Workshop, 2004. CVPRW'04. Conference on, 2004, pp. 190-190.
      BibTeX
      • @Inproceedings{marks20043d,
      • author = {Marks, Tim K and Hershey, John and Roddey, J Cooper and Movellan, Javier R},
      • title = {3d tracking of morphable objects using conditionally gaussian nonlinear filters},
      • booktitle = {Computer Vision and Pattern Recognition Workshop, 2004. CVPRW'04. Conference on},
      • year = 2004,
      • pages = {190--190},
      • organization = {IEEE}
      • }
    •  Tim K Marks and Javier R Movellan, "Diffusion networks, products of experts, and factor analysis", Proc. Int. Conf. on Independent Component Analysis, pp. 481-485, 2001.
      BibTeX
      • @Article{marks2001diffusion,
      • author = {Marks, Tim K and Movellan, Javier R},
      • title = {Diffusion networks, products of experts, and factor analysis},
      • journal = {Proc. Int. Conf. on Independent Component Analysis},
      • year = 2001,
      • pages = {481--485},
      • publisher = {Citeseer}
      • }

  • Software Downloads

  • Videos

  • MERL Issued Patents

    • Title: "Method and System for Determining 3D Object Poses and Landmark Points using Surface Patches"
      Inventors: Jones, Michael J.; Marks, Tim; Papazov, Chavdar
      Patent No.: 10,515,259
      Issue Date: Dec 24, 2019
    • Title: "Method and System for Multi-Modal Fusion Model"
      Inventors: Hori, Chiori; Hori, Takaaki; Hershey, John R.; Marks, Tim
      Patent No.: 10,417,498
      Issue Date: Sep 17, 2019
    • Title: "Method and System for Detecting Actions in Videos"
      Inventors: Jones, Michael J.; Tuzel, Oncel; Marks, Tim; Singh, Bharat
      Patent No.: 10,242,266
      Issue Date: Mar 26, 2019
    • Title: "Method and System for Detecting Actions in Videos using Contour Sequences"
      Inventors: Jones, Michael J.; Marks, Tim; Kulkarni, Kuldeep
      Patent No.: 10,210,391
      Issue Date: Feb 19, 2019
    • Title: "Method for Estimating Locations of Facial Landmarks in an Image of a Face using Globally Aligned Regression"
      Inventors: Tuzel, Oncel; Marks, Tim; Tambe, Salil
      Patent No.: 9,633,250
      Issue Date: Apr 25, 2017
    • Title: "Method for Generating Representations Polylines Using Piecewise Fitted Geometric Primitives"
      Inventors: Brand, Matthew E.; Marks, Tim; MV, Rohith
      Patent No.: 9,613,443
      Issue Date: Apr 4, 2017
    • Title: "Method for Determining Similarity of Objects Represented in Images"
      Inventors: Jones, Michael J.; Marks, Tim; Ahmed, Ejaz
      Patent No.: 9,436,895
      Issue Date: Sep 6, 2016
    • Title: "Method for Detecting 3D Geometric Boundaries in Images of Scenes Subject to Varying Lighting"
      Inventors: Marks, Tim; Tuzel, Oncel; Porikli, Fatih M.; Thornton, Jay E.; Ni, Jie
      Patent No.: 9,418,434
      Issue Date: Aug 16, 2016
    • Title: "Method for Factorizing Images of a Scene into Basis Images"
      Inventors: Tuzel, Oncel; Marks, Tim; Porikli, Fatih M.; Ni, Jie
      Patent No.: 9,384,553
      Issue Date: Jul 5, 2016
    • Title: "Method and System for Tracking People in Indoor Environments using a Visible Light Camera and a Low-Frame-Rate Infrared Sensor"
      Inventors: Marks, Tim; Jones, Michael J.; Kumar, Suren
      Patent No.: 9,245,196
      Issue Date: Jan 26, 2016
    • Title: "Method for Detecting and Tracking Objects in Image Sequences of Scenes Acquired by a Stationary Camera"
      Inventors: Marks, Tim; Jones, Michael J.; MV, Rohith
      Patent No.: 9,213,896
      Issue Date: Dec 15, 2015
    • Title: "Method and System for Segmenting Moving Objects from Images Using Foreground Extraction"
      Inventors: Veeraraghavan, Ashok N.; Marks, Tim; Taguchi, Yuichi
      Patent No.: 8,941,726
      Issue Date: Jan 27, 2015
    • Title: "Camera-Based 3D Climate Control"
      Inventors: Marks, Tim; Jones, Michael J.
      Patent No.: 8,929,592
      Issue Date: Jan 6, 2015
    • Title: "Method and System for Registering an Object with a Probe Using Entropy-Based Motion Selection and Rao-Blackwellized Particle Filtering"
      Inventors: Taguchi, Yuichi; Marks, Tim; Hershey, John R.
      Patent No.: 8,510,078
      Issue Date: Aug 13, 2013
    • Title: "Localization in Industrial Robotics Using Rao-Blackwellized Particle Filtering"
      Inventors: Marks, Tim; Taguchi, Yuichi
      Patent No.: 8,219,352
      Issue Date: Jul 10, 2012
    • Title: "Method for Synthetically Images of Objects"
      Inventors: Jones, Michael J.; Marks, Tim; Kumar, Ritwik
      Patent No.: 8,194,072
      Issue Date: Jun 5, 2012
    See All Patents for MERL