MERL - Mitsubishi Electric Research Laboratories

Suhas Lohit

Suhas Lohit
Suhas' Quick Links

  • Biography

    Before coming to MERL, Suhas worked as an intern at MERL (summer 2018), SRI International (summer 2017) and Nvidia (summer 2016). His research interests include computer vision, computational imaging and deep learning. Recently, his research focus has been on creating hybrid model- and data-driven neural architectures for various applications in imaging and vision. He won the Best Paper Award at the CVPR workshop on Computational Cameras and Displays in 2015 and the University Graduate Fellowship at ASU for 2015-16.

  • Recent News & Events

    •  NEWS    MERL researchers presenting four papers and organizing the VLAR-SMART101 Workshop at ICCV 2023
      Date: October 2, 2023 - October 6, 2023
      Where: Paris/France
      MERL Contacts: Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Toshiaki Koike-Akino; Suhas Lohit; Tim K. Marks; Pedro Miraldo; Kuan-Chuan Peng; Ye Wang
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Brief
      • MERL researchers are presenting 4 papers and organizing the VLAR-SMART-101 workshop at the ICCV 2023 conference, which will be held in Paris, France October 2-6. ICCV is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.

        1. Conference paper: “Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis,” by Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal Patel, and Tim K. Marks

        Conditional generative models typically demand large annotated training sets to achieve high-quality synthesis. As a result, there has been significant interest in plug-and-play generation, i.e., using a pre-defined model to guide the generative process. In this paper, we introduce Steered Diffusion, a generalized framework for fine-grained photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation. The key idea is to steer the image generation of the diffusion model during inference via designing a loss using a pre-trained inverse model that characterizes the conditional task. Our model shows clear qualitative and quantitative improvements over state-of-the-art diffusion-based plug-and-play models, while adding negligible computational cost.

        2. Conference paper: "BANSAC: A dynamic BAyesian Network for adaptive SAmple Consensus," by Valter Piedade and Pedro Miraldo

        We derive a dynamic Bayesian network that updates individual data points' inlier scores while iterating RANSAC. At each iteration, we apply weighted sampling using the updated scores. Our method works with or without prior data point scorings. In addition, we use the updated inlier/outlier scoring for deriving a new stopping criterion for the RANSAC loop. Our method outperforms the baselines in accuracy while needing less computational time.

        3. Conference paper: "Robust Frame-to-Frame Camera Rotation Estimation in Crowded Scenes," by Fabien Delattre, David Dirnfeld, Phat Nguyen, Stephen Scarano, Michael J. Jones, Pedro Miraldo, and Erik Learned-Miller

        We present a novel approach to estimating camera rotation in crowded, real-world scenes captured using a handheld monocular video camera. Our method uses a novel generalization of the Hough transform on SO3 to efficiently find the camera rotation most compatible with the optical flow. Because the setting is not addressed well by other data sets, we provide a new dataset and benchmark, with high-accuracy and rigorously annotated ground truth on 17 video sequences. Our method is more accurate by almost 40 percent than the next best method.

        4. Workshop paper: "Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection" by Manish Sharma*, Moitreya Chatterjee*, Kuan-Chuan Peng, Suhas Lohit, and Michael Jones

        While state-of-the-art object detection methods for RGB images have reached some level of maturity, the same is not true for Infrared (IR) images. The primary bottleneck towards bridging this gap is the lack of sufficient labeled training data in the IR images. Towards addressing this issue, we present TensorFact, a novel tensor decomposition method which splits the convolution kernels of a CNN into low-rank factor matrices with fewer parameters. This compressed network is first pre-trained on RGB images and then augmented with only a few parameters. This augmented network is then trained on IR images, while freezing the weights trained on RGB. This prevents it from over-fitting, allowing it to generalize better. Experiments show that our method outperforms state-of-the-art.

        5. “Vision-and-Language Algorithmic Reasoning (VLAR) Workshop and SMART-101 Challenge” by Anoop Cherian,  Kuan-Chuan Peng, Suhas Lohit, Tim K. Marks, Ram Ramrakhya, Honglu Zhou, Kevin A. Smith, Joanna Matthiesen, and Joshua B. Tenenbaum

        MERL researchers along with researchers from MIT, GeorgiaTech, Math Kangaroo USA, and Rutgers University are jointly organizing a workshop on vision-and-language algorithmic reasoning at ICCV 2023 and conducting a challenge based on the SMART-101 puzzles described in the paper: Are Deep Neural Networks SMARTer than Second Graders?. A focus of this workshop is to bring together outstanding faculty/researchers working at the intersections of vision, language, and cognition to provide their opinions on the recent breakthroughs in large language models and artificial general intelligence, as well as showcase their cutting edge research that could inspire the audience to search for the missing pieces in our quest towards solving the puzzle of artificial intelligence.

        Workshop link: https://wvlar.github.io/iccv23/
    •  
    •  NEWS    MERL researchers presenting four papers and co-organizing a workshop at CVPR 2023
      Date: June 18, 2023 - June 22, 2023
      Where: Vancouver/Canada
      MERL Contacts: Anoop Cherian; Michael J. Jones; Suhas Lohit; Kuan-Chuan Peng
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Brief
      • MERL researchers are presenting 4 papers and co-organizing a workshop at the CVPR 2023 conference, which will be held in Vancouver, Canada June 18-22. CVPR is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.

        1. “Are Deep Neural Networks SMARTer than Second Graders,” by Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Kevin Smith, and Joshua B. Tenenbaum

        We present SMART: a Simple Multimodal Algorithmic Reasoning Task and the associated SMART-101 dataset for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed for children in the 6-8 age group. Our experiments using SMART-101 reveal that powerful deep models are not better than random accuracy when analyzed for generalization. We also evaluate large language models (including ChatGPT) on a subset of SMART-101 and find that while these models show convincing reasoning abilities, their answers are often incorrect.

        Paper: https://arxiv.org/abs/2212.09993

        2. “EVAL: Explainable Video Anomaly Localization,” by Ashish Singh, Michael J. Jones, and Erik Learned-Miller

        This work presents a method for detecting unusual activities in videos by building a high-level model of activities found in nominal videos of a scene. The high-level features used in the model are human understandable and include attributes such as the object class and the directions and speeds of motion. Such high-level features allow our method to not only detect anomalous activity but also to provide explanations for why it is anomalous.

        Paper: https://arxiv.org/abs/2212.07900

        3. "Aligning Step-by-Step Instructional Diagrams to Video Demonstrations," by Jiahao Zhang, Anoop Cherian, Yanbin Liu, Yizhak Ben-Shabat, Cristian Rodriguez, and Stephen Gould

        The rise of do-it-yourself (DIY) videos on the web has made it possible even for an unskilled person (or a skilled robot) to imitate and follow instructions to complete complex real world tasks. In this paper, we consider the novel problem of aligning instruction steps that are depicted as assembly diagrams (commonly seen in Ikea assembly manuals) with video segments from in-the-wild videos. We present a new dataset: Ikea Assembly in the Wild (IAW) and propose a contrastive learning framework for aligning instruction diagrams with video clips.

        Paper: https://arxiv.org/pdf/2303.13800.pdf

        4. "HaLP: Hallucinating Latent Positives for Skeleton-Based Self-Supervised Learning of Actions," by Anshul Shah, Aniket Roy, Ketul Shah, Shlok Kumar Mishra, David Jacobs, Anoop Cherian, and Rama Chellappa

        In this work, we propose a new contrastive learning approach to train models for skeleton-based action recognition without labels. Our key contribution is a simple module, HaLP: Hallucinating Latent Positives for contrastive learning. HaLP explores the latent space of poses in suitable directions to generate new positives. Our experiments using HaLP demonstrates strong empirical improvements.

        Paper: https://arxiv.org/abs/2304.00387

        The 4th Workshop on Fair, Data-Efficient, and Trusted Computer Vision

        MERL researcher Kuan-Chuan Peng is co-organizing the fourth Workshop on Fair, Data-Efficient, and Trusted Computer Vision (https://fadetrcv.github.io/2023/) in conjunction with CVPR 2023 on June 18, 2023. This workshop provides a focused venue for discussing and disseminating research in the areas of fairness, bias, and trust in computer vision, as well as adjacent domains such as computational social science and public policy.
    •  

    See All News & Events for Suhas

  • Awards

    •  AWARD    Best Paper - Honorable Mention Award at WACV 2021
      Date: January 6, 2021
      Awarded to: Rushil Anirudh, Suhas Lohit, Pavan Turaga
      MERL Contact: Suhas Lohit
      Research Areas: Computational Sensing, Computer Vision, Machine Learning
      Brief
      • A team of researchers from Mitsubishi Electric Research Laboratories (MERL), Lawrence Livermore National Laboratory (LLNL) and Arizona State University (ASU) received the Best Paper Honorable Mention Award at WACV 2021 for their paper "Generative Patch Priors for Practical Compressive Image Recovery".

        The paper proposes a novel model of natural images as a composition of small patches which are obtained from a deep generative network. This is unlike prior approaches where the networks attempt to model image-level distributions and are unable to generalize outside training distributions. The key idea in this paper is that learning patch-level statistics is far easier. As the authors demonstrate, this model can then be used to efficiently solve challenging inverse problems in imaging such as compressive image recovery and inpainting even from very few measurements for diverse natural scenes.
    •  
    See All Awards for MERL

  • Research Highlights

  • Internships with Suhas

    • CV2084: Deep Learning for Cloud Removal from Satellite Images

      MERL is seeking an intern to conduct research for cloud removal from satellite images. The focus will be on building novel deep learning algorithms for this application. A good candidate is a PhD student with experience in deep learning and computational imaging with a publication record. Prior knowledge and experience in deep image restoration algorithms e.g., deep algorithm unrolling, using deep priors such as diffusion models are strongly preferred. Good Python and Pytorch skills are required. Publication of results in a conference or a journal is expected. The expected duration of the internship is 3 months and the start date is flexible.

    See All Internships at MERL

  • MERL Publications

    •  Carmichael, Z., Lohit, S., Cherian, A., Jones, M.J., Scheirer, W., "Pixel-Grounded Prototypical Part Networks", arXiv, October 2023.
      BibTeX arXiv
      • @article{Carmichael2023oct,
      • author = {Carmichael, Zachariah and Lohit, Suhas and Cherian, Anoop and Jones, Michael J. and Scheirer, Walter},
      • title = {Pixel-Grounded Prototypical Part Networks},
      • journal = {arXiv},
      • year = 2023,
      • month = oct,
      • url = {https://arxiv.org/abs/2309.14531}
      • }
    •  Sharma, M., Chatterjee, M., Peng, K.-C., Lohit, S., Jones, M.J., "Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection", IEEE International Conference on Computer Vision Workshops (ICCV), October 2023, pp. 924-932.
      BibTeX TR2023-125 PDF Presentation
      • @inproceedings{Sharma2023oct,
      • author = {Sharma, Manish and Chatterjee, Moitreya and Peng, Kuan-Chuan and Lohit, Suhas and Jones, Michael J.},
      • title = {Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection},
      • booktitle = {IEEE International Conference on Computer Vision Workshops (ICCV)},
      • year = 2023,
      • pages = {924--932},
      • month = oct,
      • url = {https://www.merl.com/publications/TR2023-125}
      • }
    •  Nair, N.G., Cherian, A., Lohit, S., Wang, Y., Koike-Akino, T., Patel, V.M., Marks, T.K., "Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis", IEEE International Conference on Computer Vision (ICCV), October 2023.
      BibTeX TR2023-126 PDF Presentation
      • @inproceedings{Nair2023sep,
      • author = {Nair, Nithin Gopalakrishnan and Cherian, Anoop and Lohit, Suhas and Wang, Ye and Koike-Akino, Toshiaki and Patel, Vishal M. and Marks, Tim K.},
      • title = {Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis},
      • booktitle = {IEEE International Conference on Computer Vision (ICCV)},
      • year = 2023,
      • month = oct,
      • url = {https://www.merl.com/publications/TR2023-126}
      • }
    •  Shenoy, V., Marks, T.K., Mansour, H., Lohit, S., "Unrolled IPPG: Video Heart Rate Esitmation via Unrolling Proximal Gradient Descent", IEEE International Conference on Image Processing (ICIP), September 2023.
      BibTeX TR2023-116 PDF Video
      • @inproceedings{Shenoy2023sep,
      • author = {Shenoy, Vineet and Marks, Tim K. and Mansour, Hassan and Lohit, Suhas},
      • title = {Unrolled IPPG: Video Heart Rate Esitmation via Unrolling Proximal Gradient Descent},
      • booktitle = {IEEE International Conference on Image Processing (ICIP)},
      • year = 2023,
      • month = sep,
      • url = {https://www.merl.com/publications/TR2023-116}
      • }
    •  Jeon, E.S., Lohit, S., Anirudh, R., Turaga, P., "Robust Time Series Recovery and Classification Using Test-time Noise Simulator Networks", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), DOI: 10.1109/​ICASSP49357.2023.10096888, May 2023.
      BibTeX TR2023-021 PDF Presentation
      • @inproceedings{Jeon2023may,
      • author = {Jeon, Eun Som and Lohit, Suhas and Anirudh, Rushil and Turaga, Pavan},
      • title = {Robust Time Series Recovery and Classification Using Test-time Noise Simulator Networks},
      • booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
      • year = 2023,
      • month = may,
      • publisher = {IEEE},
      • doi = {10.1109/ICASSP49357.2023.10096888},
      • url = {https://www.merl.com/publications/TR2023-021}
      • }
    See All Publications for Suhas

  • Other Publications

    •  Suhas Lohit, Qiao Wang and Pavan Turaga, "Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 12426-12435.
      BibTeX
      • @Inproceedings{lohit2019temporal,
      • author = {Lohit, Suhas and Wang, Qiao and Turaga, Pavan},
      • title = {Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
      • year = 2019,
      • pages = {12426--12435}
      • }
    •  Suhas Lohit, Kuldeep Kulkarni, Ronan Kerviche, Pavan Turaga and Amit Ashok, "Convolutional neural networks for noniterative reconstruction of compressively sensed images", IEEE Transactions on Computational Imaging, Vol. 4, No. 3, pp. 326-340, 2018.
      BibTeX
      • @Article{lohit2018convolutional,
      • author = {Lohit, Suhas and Kulkarni, Kuldeep and Kerviche, Ronan and Turaga, Pavan and Ashok, Amit},
      • title = {Convolutional neural networks for noniterative reconstruction of compressively sensed images},
      • journal = {IEEE Transactions on Computational Imaging},
      • year = 2018,
      • volume = 4,
      • number = 3,
      • pages = {326--340},
      • publisher = {IEEE}
      • }
    •  Suhas Lohit, Ankan Bansal, Nitesh Shroff, Jaishanker Pillai, Pavan Turaga and Rama Chellappa, "Predicting Dynamical Evolution of Human Activities from a Single Image", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 383-392.
      BibTeX
      • @Inproceedings{lohit2018predicting,
      • author = {Lohit, Suhas and Bansal, Ankan and Shroff, Nitesh and Pillai, Jaishanker and Turaga, Pavan and Chellappa, Rama},
      • title = {Predicting Dynamical Evolution of Human Activities from a Single Image},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
      • year = 2018,
      • pages = {383--392}
      • }
    •  Suhas Lohit and Pavan Turaga, "Learning invariant Riemannian geometric representations using deep nets", Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 1329-1338.
      BibTeX
      • @Inproceedings{lohit2017learning,
      • author = {Lohit, Suhas and Turaga, Pavan},
      • title = {Learning invariant Riemannian geometric representations using deep nets},
      • booktitle = {Proceedings of the IEEE International Conference on Computer Vision Workshops},
      • year = 2017,
      • pages = {1329--1338}
      • }
    •  Kuldeep Kulkarni, Suhas Lohit, Pavan Turaga, Ronan Kerviche and Amit Ashok, "Reconnet: Non-iterative reconstruction of images from compressively sensed measurements", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 449-458.
      BibTeX
      • @Inproceedings{kulkarni2016reconnet,
      • author = {Kulkarni, Kuldeep and Lohit, Suhas and Turaga, Pavan and Kerviche, Ronan and Ashok, Amit},
      • title = {Reconnet: Non-iterative reconstruction of images from compressively sensed measurements},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
      • year = 2016,
      • pages = {449--458}
      • }
    •  Suhas Lohit, Kuldeep Kulkarni and Pavan Turaga, "Direct inference on compressive measurements using convolutional neural networks", 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 1913-1917.
      BibTeX
      • @Inproceedings{lohit2016direct,
      • author = {Lohit, Suhas and Kulkarni, Kuldeep and Turaga, Pavan},
      • title = {Direct inference on compressive measurements using convolutional neural networks},
      • booktitle = {2016 IEEE International Conference on Image Processing (ICIP)},
      • year = 2016,
      • pages = {1913--1917},
      • organization = {IEEE}
      • }
    •  Qiao Wang, Suhas Lohit, Meynard John Toledo, Matthew P Buman and Pavan Turaga, "A statistical estimation framework for energy expenditure of physical activities from a wrist-worn accelerometer", 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016, pp. 2631-2635.
      BibTeX
      • @Inproceedings{wang2016statistical,
      • author = {Wang, Qiao and Lohit, Suhas and Toledo, Meynard John and Buman, Matthew P and Turaga, Pavan},
      • title = {A statistical estimation framework for energy expenditure of physical activities from a wrist-worn accelerometer},
      • booktitle = {2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
      • year = 2016,
      • pages = {2631--2635},
      • organization = {IEEE}
      • }
    •  Suhas Lohit, Kuldeep Kulkarni, Pavan Turaga, Jian Wang and Aswin C Sankaranarayanan, "Reconstruction-free inference on compressive measurements", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015, pp. 16-24.
      BibTeX
      • @Inproceedings{lohit2015reconstruction,
      • author = {Lohit, Suhas and Kulkarni, Kuldeep and Turaga, Pavan and Wang, Jian and Sankaranarayanan, Aswin C},
      • title = {Reconstruction-free inference on compressive measurements},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
      • year = 2015,
      • pages = {16--24}
      • }

  • Software & Data Downloads

  • Videos

  • MERL Issued Patents

    • Title: "Systems and Methods for Multi-Spectral Image Fusion Using Unrolled Projected Gradient Descent and Convolutinoal Neural Network"
      Inventors: Liu, Dehong; Lohit, Suhas; Mansour, Hassan; Boufounos, Petros T.
      Patent No.: 10,891,527
      Issue Date: Jan 12, 2021
    See All Patents for MERL