Anoop Cherian

Anoop Cherian
  • Biography

    Anoop was a postdoctoral researcher in the LEAR group at Inria from 2012-2015 where his research was on the estimation and tracking of human poses in videos. From 2015-2017, he was a Research Fellow at the Australian National University, where he worked on the problem of recognizing human activities in video sequences. Anoop is the recipient of the Best Student Paper award at the Intl. Conference on Image Processing in 2012. Currently, his research focus is on modeling the semantics of video data.

  • Recent News & Events

    •  NEWS    MERL researchers presenting five papers at NeurIPS 2022
      Date: November 29, 2022 - December 9, 2022
      Where: NeurIPS 2022
      MERL Contacts: Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Suhas Lohit
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning, Speech & Audio
      Brief
      • MERL researchers are presenting 5 papers at the NeurIPS Conference, which will be held in New Orleans from Nov 29-Dec 1st, with virtual presentations in the following week. NeurIPS is one of the most prestigious and competitive international conferences in machine learning.

        MERL papers in NeurIPS 2022:

        1. “AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments” by Sudipta Paul, Amit Roy-Chowdhary, and Anoop Cherian

        This work proposes a unified multimodal task for audio-visual embodied navigation where the navigating agent can also interact and seek help from a human/oracle in natural language when it is uncertain of its navigation actions. We propose a multimodal deep hierarchical reinforcement learning framework for solving this challenging task that allows the agent to learn when to seek help and how to use the language instructions. AVLEN agents can interact anywhere in the 3D navigation space and demonstrate state-of-the-art performances when the audio-goal is sporadic or when distractor sounds are present.

        2. “Learning Partial Equivariances From Data” by David W. Romero and Suhas Lohit

        Group equivariance serves as a good prior improving data efficiency and generalization for deep neural networks, especially in settings with data or memory constraints. However, if the symmetry groups are misspecified, equivariance can be overly restrictive and lead to bad performance. This paper shows how to build partial group convolutional neural networks that learn to adapt the equivariance levels at each layer that are suitable for the task at hand directly from data. This improves performance while retaining equivariance properties approximately.

        3. “Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation” by Moitreya Chatterjee, Narendra Ahuja, and Anoop Cherian

        There often exist strong correlations between the 3D motion dynamics of a sounding source and its sound being heard, especially when the source is moving towards or away from the microphone. In this paper, we propose an audio-visual scene-graph that learns and leverages such correlations for improved visually-guided audio separation from an audio mixture, while also allowing predicting the direction of motion of the sound source.

        4. “What Makes a "Good" Data Augmentation in Knowledge Distillation - A Statistical Perspective” by Huan Wang, Suhas Lohit, Michael Jones, and Yun Fu

        This paper presents theoretical and practical results for understanding what makes a particular data augmentation technique (DA) suitable for knowledge distillation (KD). We design a simple metric that works very well in practice to predict the effectiveness of DA for KD. Based on this metric, we also propose a new data augmentation technique that outperforms other methods for knowledge distillation in image recognition networks.

        5. “FeLMi : Few shot Learning with hard Mixup” by Aniket Roy, Anshul Shah, Ketul Shah, Prithviraj Dhar, Anoop Cherian, and Rama Chellappa

        Learning from only a few examples is a fundamental challenge in machine learning. Recent approaches show benefits by learning a feature extractor on the abundant and labeled base examples and transferring these to the fewer novel examples. However, the latter stage is often prone to overfitting due to the small size of few-shot datasets. In this paper, we propose a novel uncertainty-based criteria to synthetically produce “hard” and useful data by mixing up real data samples. Our approach leads to state-of-the-art results on various computer vision few-shot benchmarks.
    •  
    •  TALK    [MERL Seminar Series 2022] Prof. Jiajun Wu presents talk titled Understanding the Visual World Through Naturally Supervised Code
      Date & Time: Tuesday, November 1, 2022; 1:00 PM
      Speaker: Jiajun Wu, Stanford University
      MERL Host: Anoop Cherian
      Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
      Abstract
      • The visual world has its inherent structure: scenes are made of multiple identical objects; different objects may have the same color or material, with a regular layout; each object can be symmetric and have repetitive parts. How can we infer, represent, and use such structure from raw data, without hampering the expressiveness of neural networks? In this talk, I will demonstrate that such structure, or code, can be learned from natural supervision. Here, natural supervision can be from pixels, where neuro-symbolic methods automatically discover repetitive parts and objects for scene synthesis. It can also be from objects, where humans during fabrication introduce priors that can be leveraged by machines to infer regular intrinsics such as texture and material. When solving these problems, structured representations and neural nets play complementary roles: it is more data-efficient to learn with structured representations, and they generalize better to new scenarios with robustly captured high-level information; neural nets effectively extract complex, low-level features from cluttered and noisy visual data.
    •  

    See All News & Events for Anoop
  • Research Highlights

  • Internships with Anoop

    • CV1911: Few-shot Action Recognition

      MERL is looking for a self-motivated intern to work on problems at the intersection of video understanding and graph representation learning for solving few-shot action recognition problems. The ideal candidate would be a PhD student with a strong mathematical background in machine learning and computer vision and who has published at least one paper in a top-tier machine learning or computer vision venue (NIPS/CVPR/ECCV/ICCV/ICML/PAMI etc.). The candidate must have prior experience in using deep learning methods for video understanding (such as using Transformers) and self-supervised/unsupervised methods. Experience in approaches using skeleton/pose-based action recognition will be a plus. Proficiency in PyTorch is expected and familiarity with neural language models will be useful. The intern will conduct original research with MERL researchers towards scientific publications.

    • CV1912: Multimodal Embodied AI

      MERL is looking for a self-motivated intern to work on problems at the intersection of visual understanding, audio processing, language models, and embodied navigation AI (see our recent NeurIPS 2022 paper for the context). The ideal candidate would be a senior PhD student with a strong background in machine learning and computer vision, as demonstrated by top-tier publications. The candidate must have prior experience in developing deep learning methods for audio-visual-language data. Expertise in popular embodied AI simulation environments as well as a strong background in reinforcement learning will be beneficial. The intern is expected to collaborate with researchers in computer vision and speech teams at MERL to develop algorithms and prepare manuscripts for scientific publications.

    • CV1930: Meta-Algorithmic Learning for Vision-based Robotic Manipulation

      MERL is looking for a self-motivated intern to work on problems at the intersection of computer vision and robotic manipulation for solving tasks such as vision-based robotic tool manipulation. The ideal candidate would be a PhD student with strong mathematical background in machine learning/reinforcement learning, modeling contact-physics for object manipulation, and experience in working with and training deep models on large scale computer vision datasets. Proficiency in PyTorch and (differentiable) robotic simulators is expected. Knowledge of meta-learning, hierarchical RL, self-supervised learning, and scene graph based visual reasoning would be useful. The intern will conduct original research with MERL researchers towards scientific publications.

    See All Internships at MERL
  • MERL Publications

    •  Chatterjee, M., Ahuja, N., Cherian, A., "Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation", Advances in Neural Information Processing Systems (NeurIPS), November 2022.
      BibTeX TR2022-140 PDF
      • @inproceedings{Chatterjee2022nov,
      • author = {Chatterjee, Moitreya and Ahuja, Narendra and Cherian, Anoop},
      • title = {Learning Audio-Visual Dynamics Using Scene Graphs for Audio Source Separation},
      • booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
      • year = 2022,
      • month = nov,
      • url = {https://www.merl.com/publications/TR2022-140}
      • }
    •  Paul, S., Roy Chowdhury, A.K., Cherian, A., "AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments", Advances in Neural Information Processing Systems (NeurIPS), October 2022.
      BibTeX TR2022-131 PDF Video
      • @inproceedings{Paul2022oct2,
      • author = {Paul, Sudipta and Roy Chowdhury, Amit K and Cherian, Anoop},
      • title = {AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments},
      • booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
      • year = 2022,
      • month = oct,
      • url = {https://www.merl.com/publications/TR2022-131}
      • }
    •  Chatterjee, M., Ahuja, N., Cherian, A., "Quantifying Predictive Uncertainty for Stochastic Video Synthesis from Audio", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
      BibTeX TR2022-082 PDF
      • @inproceedings{Chatterjee2022jun,
      • author = {Chatterjee, Moitreya and Ahuja, Narendra and Cherian, Anoop},
      • title = {Quantifying Predictive Uncertainty for Stochastic Video Synthesis from Audio},
      • booktitle = {Sight and Sound Workshop at CVPR 2022},
      • year = 2022,
      • month = jun,
      • url = {https://www.merl.com/publications/TR2022-082}
      • }
    •  Shah, A.P., Geng, S., Gao, P., Cherian, A., Hori, T., Marks, T.K., Le Roux, J., Hori, C., "Audio-Visual Scene-Aware Dialog and Reasoning Using Audio-Visual Transformers with Joint Student-Teacher Learning", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), April 2022, pp. 7732-7736.
      BibTeX TR2022-019 PDF
      • @inproceedings{Shah2022apr,
      • author = {Shah, Ankit Parag and Geng, Shijie and Gao, Peng and Cherian, Anoop and Hori, Takaaki and Marks, Tim K. and Le Roux, Jonathan and Hori, Chiori},
      • title = {Audio-Visual Scene-Aware Dialog and Reasoning Using Audio-Visual Transformers with Joint Student-Teacher Learning},
      • booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
      • year = 2022,
      • pages = {7732--7736},
      • month = apr,
      • publisher = {IEEE},
      • issn = {1520-6149},
      • isbn = {978-1-6654-0540-9},
      • url = {https://www.merl.com/publications/TR2022-019}
      • }
    •  Hori, C., Shah, A.P., Geng, S., Gao, P., Cherian, A., Hori, T., Le Roux, J., Marks, T.K., "Overview of Audio Visual Scene-Aware Dialog with Reasoning Track for Natural Language Generation in DSTC10", The 10th Dialog System Technology Challenge Workshop at AAAI, February 2022.
      BibTeX TR2022-016 PDF
      • @inproceedings{Hori2022feb,
      • author = {Hori, Chiori and Shah, Ankit Parag and Geng, Shijie and Gao, Peng and Cherian, Anoop and Hori, Takaaki and Le Roux, Jonathan and Marks, Tim K.},
      • title = {Overview of Audio Visual Scene-Aware Dialog with Reasoning Track for Natural Language Generation in DSTC10},
      • booktitle = {The 10th Dialog System Technology Challenge Workshop at AAAI},
      • year = 2022,
      • month = feb,
      • url = {https://www.merl.com/publications/TR2022-016}
      • }
    See All Publications for Anoop
  • Other Publications

    •  Anoop Cherian and Stephen Gould, "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.
      BibTeX
      • @Article{cherian2018ijcv,
      • author = {Cherian, Anoop and Gould, Stephen},
      • title = {Second-order Temporal Pooling for Action Recognition},
      • journal = {International Journal of Computer Vision (IJCV)},
      • year = 2018,
      • publisher = {Springer}
      • }
    •  Rodrigo Santa Cruz, Basura Fernando, Anoop Cherian and Stephen Gould, "Visual Permutation Learning", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.
      BibTeX
      • @Article{cherian2018permutation,
      • author = {Santa Cruz, Rodrigo and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
      • title = {Visual Permutation Learning},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2018,
      • publisher = {IEEE}
      • }
    •  Jue Wang, Anoop Cherian, Fatih Porikli and Stephen Gould, "Video Representation Learning Using Discriminative Pooling", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_representation_cvpr18,
      • author = {Wang, Jue and Cherian, Anoop and Porikli, Fatih and Gould, Stephen},
      • title = {Video Representation Learning Using Discriminative Pooling},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Suryansh Kumar, Anoop Cherian, Yuchao Dai and Hongdong Li, "Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_rigid_cvpr18,
      • author = {Kumar, Suryansh and Cherian, Anoop and Dai, Yuchao and Li, Hongdong},
      • title = {Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Anoop Cherian, Suvrit Sra, Stephen Gould and Richard Hartley, "Non-Linear Temporal Subspace Representations for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
      BibTeX
      • @Inproceedings{cherian_temporal_cvpr18,
      • author = {Cherian, Anoop and Sra, Suvrit and Gould, Stephen and Hartley, Richard},
      • title = {Non-Linear Temporal Subspace Representations for Activity Recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2018
      • }
    •  Anoop Cherian, Basura Fernando, Mehrtash Harandi and Stephen Gould, "Generalized Rank Pooling for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
      BibTeX
      • @Inproceedings{cherian2017generalized,
      • author = {Cherian, Anoop and Fernando, Basura and Harandi, Mehrtash and Gould, Stephen},
      • title = {Generalized Rank Pooling for Activity Recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2017
      • }
    •  Anoop Cherian, Panagiotis Stanitsas, Mehrtash Harandi, Vassilios Morellas and Nikolaos Papanikolopoulos, "Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices", International Conference on Computer Vision (ICCV), 2017.
      BibTeX
      • @Inproceedings{cherian_rigid_iccv17,
      • author = {Cherian, Anoop and Stanitsas, Panagiotis and Harandi, Mehrtash and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices},
      • booktitle = {International Conference on Computer Vision (ICCV)},
      • year = 2017
      • }
    •  Rodrigo Santa Cruz, Basura Fernando, Anoop Cherian and Stephen Gould, "DeepPermNet: Visual Permutation Learning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
      BibTeX
      • @Inproceedings{cruz2017deeppermnet,
      • author = {Cruz, Rodrigo Santa and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
      • title = {DeepPermNet: Visual Permutation Learning},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2017
      • }
    •  Anoop Cherian, Vassilios Morellas and Nikolaos Papanikolopoulos, "Bayesian Non-Parametric clustering for positive definite matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.
      BibTeX
      • @Article{cherian2016bayesian,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Bayesian Non-Parametric clustering for positive definite matrices},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2016,
      • publisher = {IEEE}
      • }
    •  Piotr Koniusz and Anoop Cherian, "Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
      BibTeX
      • @Inproceedings{koniusz2016sparse,
      • author = {Koniusz, Piotr and Cherian, Anoop},
      • title = {Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2016
      • }
    •  Piotr Koniusz, Anoop Cherian and Fatih Porikli, "Tensor representations via kernel linearization for action recognition from 3D skeletons", European Conference on Computer Vision (ECCV), 2016.
      BibTeX
      • @Inproceedings{koniusz2016tensor,
      • author = {Koniusz, Piotr and Cherian, Anoop and Porikli, Fatih},
      • title = {Tensor representations via kernel linearization for action recognition from 3D skeletons},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2016,
      • organization = {Springer}
      • }
    •  Anoop Cherian, Julien Mairal, Karteek Alahari and Cordelia Schmid, "Mixing body-part sequences for human pose estimation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
      BibTeX
      • @Inproceedings{cherian2014mixing,
      • author = {Cherian, Anoop and Mairal, Julien and Alahari, Karteek and Schmid, Cordelia},
      • title = {Mixing body-part sequences for human pose estimation},
      • booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2014
      • }
    •  Anoop Cherian, "Nearest neighbors using compact sparse codes", International Conference on Machine Learning (ICML), 2014.
      BibTeX
      • @Inproceedings{cherian2014nearest,
      • author = {Cherian, Anoop},
      • title = {Nearest neighbors using compact sparse codes},
      • booktitle = {International Conference on Machine Learning (ICML)},
      • year = 2014
      • }
    •  Anoop Cherian and Suvrit Sra, "Riemannian sparse coding for positive definite matrices", European Conference on Computer Vision (ECCV), 2014.
      BibTeX
      • @Inproceedings{cherian2014riemannian,
      • author = {Cherian, Anoop and Sra, Suvrit},
      • title = {Riemannian sparse coding for positive definite matrices},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2014,
      • organization = {Springer}
      • }
    •  Anoop Cherian, Suvrit Sra, Arindam Banerjee and Nikolaos Papanikolopoulos, "Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2013.
      BibTeX
      • @Article{cherian2013jensen,
      • author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
      • title = {Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices},
      • journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
      • year = 2013,
      • publisher = {IEEE}
      • }
    •  Anoop Cherian, Vassilios Morellas, Nikolaos Papanikolopoulos and Saad J Bedros, "Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications", Computer Vision and Pattern Recognition (CVPR), 2011.
      BibTeX
      • @Inproceedings{cherian2011dirichlet,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos and Bedros, Saad J},
      • title = {Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications},
      • booktitle = {Computer Vision and Pattern Recognition (CVPR)},
      • year = 2011
      • }
    •  Anoop Cherian, Suvrit Sra, Arindam Banerjee and Nikolaos Papanikolopoulos, "Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence", International Conference on Computer Vision (ICCV), 2011.
      BibTeX
      • @Inproceedings{cherian2011efficient,
      • author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
      • title = {Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence},
      • booktitle = {International Conference on Computer Vision (ICCV)},
      • year = 2011
      • }
    •  Suvrit Sra and Anoop Cherian, "Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval", Machine Learning and Knowledge Discovery in Databases (ECML), 2011.
      BibTeX
      • @Article{sra2011generalized,
      • author = {Sra, Suvrit and Cherian, Anoop},
      • title = {Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval},
      • journal = {Machine Learning and Knowledge Discovery in Databases (ECML)},
      • year = 2011
      • }
    •  Anoop Cherian, Vassilios Morellas and Nikolaos Papanikolopoulos, "Accurate 3D ground plane estimation from a single image", International Conference on Robotics and Automation, 2009.
      BibTeX
      • @Inproceedings{cherian2009accurate,
      • author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
      • title = {Accurate 3D ground plane estimation from a single image},
      • booktitle = {International Conference on Robotics and Automation},
      • year = 2009
      • }
  • Software Downloads

  • Videos

  • MERL Issued Patents

    • Title: "Low-latency Captioning System"
      Inventors: Hori, Chiori; Hori, Takaaki; Cherian, Anoop; Marks, Tim; Le Roux, Jonathan
      Patent No.: 11,445,267
      Issue Date: Sep 13, 2022
    • Title: "Anomaly Detector for Detecting Anomaly using Complementary Classifiers"
      Inventors: Cherian, Anoop
      Patent No.: 11,423,698
      Issue Date: Aug 23, 2022
    • Title: "System and Method for a Dialogue Response Generation System"
      Inventors: Hori, Chiori; Cherian, Anoop; Marks, Tim; Hori, Takaaki
      Patent No.: 11,264,009
      Issue Date: Mar 1, 2022
    • Title: "Scene-Aware Video Dialog"
      Inventors: Geng, Shijie; Gao, Peng; Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan
      Patent No.: 11,210,523
      Issue Date: Dec 28, 2021
    See All Patents for MERL