MERL - Mitsubishi Electric Research Laboratories

Anoop Cherian

Anoop Cherian
Anoop's Quick Links

  • Biography

    Anoop was a postdoctoral researcher in the LEAR group at Inria from 2012-2015 where his research was on the estimation and tracking of human poses in videos. From 2015-2017, he was a Research Fellow at the Australian National University, where he worked on the problem of recognizing human activities in video sequences. Anoop is the recipient of the Best Student Paper award at the Intl. Conference on Image Processing in 2012. Currently, his research focus is on modeling the semantics of video data.

  • Recent News & Events

    •  NEWS    MERL researchers presenting five papers at NeurIPS 2022
      Date: November 29, 2022 - December 9, 2022
      Where: NeurIPS 2022
      MERL Contacts: Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Suhas Lohit
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
      Brief
      • MERL researchers are presenting 5 papers at the NeurIPS Conference, which will be held in New Orleans from Nov 29-Dec 1st, with virtual presentations in the following week. NeurIPS is one of the most prestigious and competitive international conferences in machine learning.

        MERL papers in NeurIPS 2022:

        1. “AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments” by Sudipta Paul, Amit Roy-Chowdhary, and Anoop Cherian

        This work proposes a unified multimodal task for audio-visual embodied navigation where the navigating agent can also interact and seek help from a human/oracle in natural language when it is uncertain of its navigation actions. We propose a multimodal deep hierarchical reinforcement learning framework for solving this challenging task that allows the agent to learn when to seek help and how to use the language instructions. AVLEN agents can interact anywhere in the 3D navigation space and demonstrate state-of-the-art performances when the audio-goal is sporadic or when distractor sounds are present.

        2. “Learning Partial Equivariances From Data” by David W. Romero and Suhas Lohit

        Group equivariance serves as a good prior improving data efficiency and generalization for deep neural networks, especially in settings with data or memory constraints. However, if the symmetry groups are misspecified, equivariance can be overly restrictive and lead to bad performance. This paper shows how to build partial group convolutional neural networks that learn to adapt the equivariance levels at each layer that are suitable for the task at hand directly from data. This improves performance while retaining equivariance properties approximately.

        3. “Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation” by Moitreya Chatterjee, Narendra Ahuja, and Anoop Cherian

        There often exist strong correlations between the 3D motion dynamics of a sounding source and its sound being heard, especially when the source is moving towards or away from the microphone. In this paper, we propose an audio-visual scene-graph that learns and leverages such correlations for improved visually-guided audio separation from an audio mixture, while also allowing predicting the direction of motion of the sound source.

        4. “What Makes a "Good" Data Augmentation in Knowledge Distillation - A Statistical Perspective” by Huan Wang, Suhas Lohit, Michael Jones, and Yun Fu

        This paper presents theoretical and practical results for understanding what makes a particular data augmentation technique (DA) suitable for knowledge distillation (KD). We design a simple metric that works very well in practice to predict the effectiveness of DA for KD. Based on this metric, we also propose a new data augmentation technique that outperforms other methods for knowledge distillation in image recognition networks.

        5. “FeLMi : Few shot Learning with hard Mixup” by Aniket Roy, Anshul Shah, Ketul Shah, Prithviraj Dhar, Anoop Cherian, and Rama Chellappa

        Learning from only a few examples is a fundamental challenge in machine learning. Recent approaches show benefits by learning a feature extractor on the abundant and labeled base examples and transferring these to the fewer novel examples. However, the latter stage is often prone to overfitting due to the small size of few-shot datasets. In this paper, we propose a novel uncertainty-based criteria to synthetically produce “hard” and useful data by mixing up real data samples. Our approach leads to state-of-the-art results on various computer vision few-shot benchmarks.
    •  
    •  TALK    [MERL Seminar Series 2022] Prof. Jiajun Wu presents talk titled Understanding the Visual World Through Naturally Supervised Code
      Date & Time: Tuesday, November 1, 2022; 1:00 PM
      Speaker: Jiajun Wu, Stanford University
      MERL Host: Anoop Cherian
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Abstract
      • The visual world has its inherent structure: scenes are made of multiple identical objects; different objects may have the same color or material, with a regular layout; each object can be symmetric and have repetitive parts. How can we infer, represent, and use such structure from raw data, without hampering the expressiveness of neural networks? In this talk, I will demonstrate that such structure, or code, can be learned from natural supervision. Here, natural supervision can be from pixels, where neuro-symbolic methods automatically discover repetitive parts and objects for scene synthesis. It can also be from objects, where humans during fabrication introduce priors that can be leveraged by machines to infer regular intrinsics such as texture and material. When solving these problems, structured representations and neural nets play complementary roles: it is more data-efficient to learn with structured representations, and they generalize better to new scenarios with robustly captured high-level information; neural nets effectively extract complex, low-level features from cluttered and noisy visual data.
    •  

    See All News & Events for Anoop

  • Research Highlights

  • MERL Publications

    •  Ota, K., Tung, H.-Y., Smith, K., Cherian, A., Marks, T.K., Sullivan, A., Kanezaki, A., Tenenbaum, J.B., "H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions", IEEE International Conference on Robotics and Automation (ICRA), March 2023.
      BibTeX TR2023-009 PDF
      • @inproceedings{Ota2023mar,
      • author = {Ota, Kei and Tung, Hsiao-Yu and Smith, Kevin and Cherian, Anoop and Marks, Tim K. and Sullivan, Alan and Kanezaki, Asako and Tenenbaum, Joshua B.},
      • title = {H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions},
      • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
      • year = 2023,
      • month = mar,
      • url = {https://www.merl.com/publications/TR2023-009}
      • }
    •  Cherian, A., Jain, S., Marks, T.K., Sullivan, A., "Discriminative 3D Shape Modeling for Few-Shot Instance Segmentation", IEEE International Conference on Robotics and Automation (ICRA), March 2023.
      BibTeX TR2023-010 PDF
      • @inproceedings{Cherian2023mar,
      • author = {Cherian, Anoop and Jain, Siddarth and Marks, Tim K. and Sullivan, Alan},
      • title = {Discriminative 3D Shape Modeling for Few-Shot Instance Segmentation},
      • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
      • year = 2023,
      • month = mar,
      • url = {https://www.merl.com/publications/TR2023-010}
      • }
    •  Liu, T., Cherian, A., "Learning a Constrained Optimizer: A Primal Method", AAAI Conference on Artificial Intelligence, January 2023.
      BibTeX TR2023-003 PDF
      • @inproceedings{Liu2023jan,
      • author = {Liu, Tao and Cherian, Anoop},
      • title = {Learning a Constrained Optimizer: A Primal Method},
      • booktitle = {AAAI Conference on Artificial Intelligence},
      • year = 2023,
      • month = jan,
      • url = {https://www.merl.com/publications/TR2023-003}
      • }
    •  Cherian, A., Peng, K.-C., Lohit, S., Smith, K., Tenenbaum, J.B., "Are Deep Neural Networks SMARTer than Second Graders?", arXiv, December 2022.
      BibTeX arXiv
      • @inproceedings{Cherian2022dec,
      • author = {Cherian, Anoop and Peng, Kuan-Chuan and Lohit, Suhas and Smith, Kevin and Tenenbaum, Joshua B.},
      • title = {Are Deep Neural Networks SMARTer than Second Graders?},
      • booktitle = {arXiv},
      • year = 2022,
      • month = dec,
      • url = {https://arxiv.org/abs/2212.09993}
      • }
    •  Chatterjee, M., Ahuja, N., Cherian, A., "Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation", Advances in Neural Information Processing Systems (NeurIPS), November 2022.
      BibTeX TR2022-140 PDF
      • @inproceedings{Chatterjee2022nov,
      • author = {Chatterjee, Moitreya and Ahuja, Narendra and Cherian, Anoop},
      • title = {Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation},
      • booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
      • year = 2022,
      • month = nov,
      • url = {https://www.merl.com/publications/TR2022-140}
      • }
    See All Publications for Anoop

  • Other Publications

    •  Anoop Cherian and Stephen Gould, "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.
      BibTeX
      • @Article{cherian2018ijcv,
      • author = {Cherian, Anoop and Gould, Stephen},
      • title = {Second-order Temporal Pooling for Action Recognition},
      • journal = {International Journal of Computer Vision (IJCV)},
      • year = 2018,
      • publisher = {Springer}
      • }
    •  Rodrigo Santa Cruz, Basura Fernando, Anoop Cherian and Stephen Gould, "Visual Permutation Learning", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.
      BibTeX
      • @Article{cherian2018permutation,
      • author = {Santa Cruz, Rodrigo and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
      • title = {Visual Permutation Learning},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2018,
      • publisher = {IEEE}
      • }
    •  Jue Wang, Anoop Cherian, Fatih Porikli and Stephen Gould, "Video Representation Learning Using Discriminative Pooling", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_representation_cvpr18,
      • author = {Wang, Jue and Cherian, Anoop and Porikli, Fatih and Gould, Stephen},
      • title = {Video Representation Learning Using Discriminative Pooling},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Suryansh Kumar, Anoop Cherian, Yuchao Dai and Hongdong Li, "Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_rigid_cvpr18,
      • author = {Kumar, Suryansh and Cherian, Anoop and Dai, Yuchao and Li, Hongdong},
      • title = {Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Anoop Cherian, Suvrit Sra, Stephen Gould and Richard Hartley, "Non-Linear Temporal Subspace Representations for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_temporal_cvpr18,
      • author = {Cherian, Anoop and Sra, Suvrit and Gould, Stephen and Hartley, Richard},
      • title = {Non-Linear Temporal Subspace Representations for Activity Recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Anoop Cherian, Basura Fernando, Mehrtash Harandi and Stephen Gould, "Generalized Rank Pooling for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
      BibTeX
      • @Inproceedings{cherian2017generalized,
      • author = {Cherian, Anoop and Fernando, Basura and Harandi, Mehrtash and Gould, Stephen},
      • title = {Generalized Rank Pooling for Activity Recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2017
      • }
    •  Anoop Cherian, Panagiotis Stanitsas, Mehrtash Harandi, Vassilios Morellas and Nikolaos Papanikolopoulos, "Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices", International Conference on Computer Vision (ICCV), 2017.
      BibTeX
      • @Inproceedings{cherian_rigid_iccv17,
      • author = {Cherian, Anoop and Stanitsas, Panagiotis and Harandi, Mehrtash and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices},
      • booktitle = {International Conference on Computer Vision (ICCV)},
      • year = 2017
      • }
    •  Rodrigo Santa Cruz, Basura Fernando, Anoop Cherian and Stephen Gould, "DeepPermNet: Visual Permutation Learning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
      BibTeX
      • @Inproceedings{cruz2017deeppermnet,
      • author = {Cruz, Rodrigo Santa and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
      • title = {DeepPermNet: Visual Permutation Learning},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2017
      • }
    •  Anoop Cherian, Vassilios Morellas and Nikolaos Papanikolopoulos, "Bayesian Non-Parametric clustering for positive definite matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.
      BibTeX
      • @Article{cherian2016bayesian,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Bayesian Non-Parametric clustering for positive definite matrices},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2016,
      • publisher = {IEEE}
      • }
    •  Piotr Koniusz and Anoop Cherian, "Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
      BibTeX
      • @Inproceedings{koniusz2016sparse,
      • author = {Koniusz, Piotr and Cherian, Anoop},
      • title = {Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2016
      • }
    •  Piotr Koniusz, Anoop Cherian and Fatih Porikli, "Tensor representations via kernel linearization for action recognition from 3D skeletons", European Conference on Computer Vision (ECCV), 2016.
      BibTeX
      • @Inproceedings{koniusz2016tensor,
      • author = {Koniusz, Piotr and Cherian, Anoop and Porikli, Fatih},
      • title = {Tensor representations via kernel linearization for action recognition from 3D skeletons},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2016,
      • organization = {Springer}
      • }
    •  Anoop Cherian, Julien Mairal, Karteek Alahari and Cordelia Schmid, "Mixing body-part sequences for human pose estimation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
      BibTeX
      • @Inproceedings{cherian2014mixing,
      • author = {Cherian, Anoop and Mairal, Julien and Alahari, Karteek and Schmid, Cordelia},
      • title = {Mixing body-part sequences for human pose estimation},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2014
      • }
    •  Anoop Cherian, "Nearest neighbors using compact sparse codes", International Conference on Machine Learning (ICML), 2014.
      BibTeX
      • @Inproceedings{cherian2014nearest,
      • author = {Cherian, Anoop},
      • title = {Nearest neighbors using compact sparse codes},
      • booktitle = {International Conference on Machine Learning (ICML)},
      • year = 2014
      • }
    •  Anoop Cherian and Suvrit Sra, "Riemannian sparse coding for positive definite matrices", European Conference on Computer Vision (ECCV), 2014.
      BibTeX
      • @Inproceedings{cherian2014riemannian,
      • author = {Cherian, Anoop and Sra, Suvrit},
      • title = {Riemannian sparse coding for positive definite matrices},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2014,
      • organization = {Springer}
      • }
    •  Anoop Cherian, Suvrit Sra, Arindam Banerjee and Nikolaos Papanikolopoulos, "Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2013.
      BibTeX
      • @Article{cherian2013jensen,
      • author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
      • title = {Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2013,
      • publisher = {IEEE}
      • }
    •  Anoop Cherian, Vassilios Morellas, Nikolaos Papanikolopoulos and Saad J Bedros, "Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications", Computer Vision and Pattern Recognition (CVPR), 2011.
      BibTeX
      • @Inproceedings{cherian2011dirichlet,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos and Bedros, Saad J},
      • title = {Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications},
      • booktitle = {Computer Vision and Pattern Recognition (CVPR)},
      • year = 2011
      • }
    •  Anoop Cherian, Suvrit Sra, Arindam Banerjee and Nikolaos Papanikolopoulos, "Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence", International Conference on Computer Vision (ICCV), 2011.
      BibTeX
      • @Inproceedings{cherian2011efficient,
      • author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
      • title = {Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence},
      • booktitle = {International Conference on Computer Vision (ICCV)},
      • year = 2011
      • }
    •  Suvrit Sra and Anoop Cherian, "Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval", Machine Learning and Knowledge Discovery in Databases (ECML), 2011.
      BibTeX
      • @Article{sra2011generalized,
      • author = {Sra, Suvrit and Cherian, Anoop},
      • title = {Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval},
      • journal = {Machine Learning and Knowledge Discovery in Databases (ECML)},
      • year = 2011
      • }
    •  Anoop Cherian, Vassilios Morellas and Nikolaos Papanikolopoulos, "Accurate 3D ground plane estimation from a single image", International Conference on Robotics and Automation, 2009.
      BibTeX
      • @Inproceedings{cherian2009accurate,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Accurate 3D ground plane estimation from a single image},
      • booktitle = {International Conference on Robotics and Automation},
      • year = 2009
      • }

  • Software Downloads

  • Videos

  • MERL Issued Patents

    • Title: "Scene-Aware Video Encoder System and Method"
      Inventors: Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan; Marks, Tim; Sullivan, Alan
      Patent No.: 11,582,485
      Issue Date: Feb 14, 2023
    • Title: "Low-latency Captioning System"
      Inventors: Hori, Chiori; Hori, Takaaki; Cherian, Anoop; Marks, Tim; Le Roux, Jonathan
      Patent No.: 11,445,267
      Issue Date: Sep 13, 2022
    • Title: "Anomaly Detector for Detecting Anomaly using Complementary Classifiers"
      Inventors: Cherian, Anoop; Wang, Jue
      Patent No.: 11,423,698
      Issue Date: Aug 23, 2022
    • Title: "System and Method for a Dialogue Response Generation System"
      Inventors: Hori, Chiori; Cherian, Anoop; Marks, Tim; Hori, Takaaki
      Patent No.: 11,264,009
      Issue Date: Mar 1, 2022
    • Title: "Scene-Aware Video Dialog"
      Inventors: Geng, Shijie; Gao, Peng; Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan
      Patent No.: 11,210,523
      Issue Date: Dec 28, 2021
    See All Patents for MERL