Anoop Cherian
- Phone: 617-621-7519
- Email:
-
Position:
Research / Technical Staff
Senior Principal Research Scientist -
Education:
Ph.D., University of Minnesota, 2013 -
Research Areas:
External Links:
Anoop's Quick Links
-
Biography
Anoop was a postdoctoral researcher in the LEAR group at Inria from 2012-2015 where his research was on the estimation and tracking of human poses in videos. From 2015-2017, he was a Research Fellow at the Australian National University, where he worked on the problem of recognizing human activities in video sequences. Anoop is the recipient of the Best Student Paper award at the Intl. Conference on Image Processing in 2012. Currently, his research focus is on modeling the semantics of video data.
-
Recent News & Events
-
NEWS MERL at the International Conference on Robotics and Automation (ICRA) 2024 Date: May 13, 2024 - May 17, 2024
Where: Yokohama, Japan
MERL Contacts: Anoop Cherian; Radu Corcodel; Stefano Di Cairano; Chiori Hori; Siddarth Jain; Devesh K. Jha; Jonathan Le Roux; Diego Romeres; William S. Yerazunis
Research Areas: Artificial Intelligence, Machine Learning, Optimization, Robotics, Speech & AudioBrief- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
MERL was a Bronze sponsor of the conference, and exhibited a live robotic demonstration, which attracted a large audience. The demonstration showcased an Autonomous Robotic Assembly technology executed on MELCO's Assista robot arm and was the collaborative effort of the Optimization and Robotics Team together with the Advanced Technology department at Mitsubishi Electric.
MERL researchers from the Optimization and Robotics, Speech & Audio, and Control for Autonomy teams also presented 8 papers and 2 invited talks covering topics on robotic assembly, applications of LLMs to robotics, human robot interaction, safe and robust path planning for autonomous drones, transfer learning, perception and tactile sensing.
- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
-
NEWS MERL Papers and Workshops at CVPR 2024 Date: June 17, 2024 - June 21, 2024
Where: Seattle, WA
MERL Contacts: Petros T. Boufounos; Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Toshiaki Koike-Akino; Jonathan Le Roux; Suhas Lohit; Tim K. Marks; Pedro Miraldo; Jing Liu; Kuan-Chuan Peng; Pu (Perry) Wang; Ye Wang; Matthew Brand
Research Areas: Artificial Intelligence, Computational Sensing, Computer Vision, Machine Learning, Speech & AudioBrief- MERL researchers are presenting 5 conference papers, 3 workshop papers, and are co-organizing two workshops at the CVPR 2024 conference, which will be held in Seattle, June 17-21. CVPR is one of the most prestigious and competitive international conferences in computer vision. Details of MERL contributions are provided below.
CVPR Conference Papers:
1. "TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models" by H. Ni, B. Egger, S. Lohit, A. Cherian, Y. Wang, T. Koike-Akino, S. X. Huang, and T. K. Marks
This work enables a pretrained text-to-video (T2V) diffusion model to be additionally conditioned on an input image (first video frame), yielding a text+image to video (TI2V) model. Other than using the pretrained T2V model, our method requires no ("zero") training or fine-tuning. The paper uses a "repeat-and-slide" method and diffusion resampling to synthesize videos from a given starting image and text describing the video content.
Paper: https://www.merl.com/publications/TR2024-059
Project page: https://merl.com/research/highlights/TI2V-Zero
2. "Long-Tailed Anomaly Detection with Learnable Class Names" by C.-H. Ho, K.-C. Peng, and N. Vasconcelos
This work aims to identify defects across various classes without relying on hard-coded class names. We introduce the concept of long-tailed anomaly detection, addressing challenges like class imbalance and dataset variability. Our proposed method combines reconstruction and semantic modules, learning pseudo-class names and utilizing a variational autoencoder for feature synthesis to improve performance in long-tailed datasets, outperforming existing methods in experiments.
Paper: https://www.merl.com/publications/TR2024-040
3. "Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling" by X. Liu, Y-W. Tai, C-T. Tang, P. Miraldo, S. Lohit, and M. Chatterjee
This work presents a new strategy for rendering dynamic scenes from novel viewpoints. Our approach is based on stratifying the scene into regions based on the extent of motion of the region, which is automatically determined. Regions with higher motion are permitted a denser spatio-temporal sampling strategy for more faithful rendering of the scene. Additionally, to the best of our knowledge, ours is the first work to enable tracking of objects in the scene from novel views - based on the preferences of a user, provided by a click.
Paper: https://www.merl.com/publications/TR2024-042
4. "SIRA: Scalable Inter-frame Relation and Association for Radar Perception" by R. Yataka, P. Wang, P. T. Boufounos, and R. Takahashi
Overcoming the limitations on radar feature extraction such as low spatial resolution, multipath reflection, and motion blurs, this paper proposes SIRA (Scalable Inter-frame Relation and Association) for scalable radar perception with two designs: 1) extended temporal relation, generalizing the existing temporal relation layer from two frames to multiple inter-frames with temporally regrouped window attention for scalability; and 2) motion consistency track with a pseudo-tracklet generated from observational data for better object association.
Paper: https://www.merl.com/publications/TR2024-041
5. "RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation" by Z. Yang, J. Liu, P. Chen, A. Cherian, T. K. Marks, J. L. Roux, and C. Gan
We leverage Large Language Models (LLM) for zero-shot semantic audio visual navigation. Specifically, by employing multi-modal models to process sensory data, we instruct an LLM-based planner to actively explore the environment by adaptively evaluating and dismissing inaccurate perceptual descriptions.
Paper: https://www.merl.com/publications/TR2024-043
CVPR Workshop Papers:
1. "CoLa-SDF: Controllable Latent StyleSDF for Disentangled 3D Face Generation" by R. Dey, B. Egger, V. Boddeti, Y. Wang, and T. K. Marks
This paper proposes a new method for generating 3D faces and rendering them to images by combining the controllability of nonlinear 3DMMs with the high fidelity of implicit 3D GANs. Inspired by StyleSDF, our model uses a similar architecture but enforces the latent space to match the interpretable and physical parameters of the nonlinear 3D morphable model MOST-GAN.
Paper: https://www.merl.com/publications/TR2024-045
2. “Tracklet-based Explainable Video Anomaly Localization” by A. Singh, M. J. Jones, and E. Learned-Miller
This paper describes a new method for localizing anomalous activity in video of a scene given sample videos of normal activity from the same scene. The method is based on detecting and tracking objects in the scene and estimating high-level attributes of the objects such as their location, size, short-term trajectory and object class. These high-level attributes can then be used to detect unusual activity as well as to provide a human-understandable explanation for what is unusual about the activity.
Paper: https://www.merl.com/publications/TR2024-057
MERL co-organized workshops:
1. "Multimodal Algorithmic Reasoning Workshop" by A. Cherian, K-C. Peng, S. Lohit, M. Chatterjee, H. Zhou, K. Smith, T. K. Marks, J. Mathissen, and J. Tenenbaum
Workshop link: https://marworkshop.github.io/cvpr24/index.html
2. "The 5th Workshop on Fair, Data-Efficient, and Trusted Computer Vision" by K-C. Peng, et al.
Workshop link: https://fadetrcv.github.io/2024/
3. "SuperLoRA: Parameter-Efficient Unified Adaptation for Large Vision Models" by X. Chen, J. Liu, Y. Wang, P. Wang, M. Brand, G. Wang, and T. Koike-Akino
This paper proposes a generalized framework called SuperLoRA that unifies and extends different variants of low-rank adaptation (LoRA). Introducing new options with grouping, folding, shuffling, projection, and tensor decomposition, SuperLoRA offers high flexibility and demonstrates superior performance up to 10-fold gain in parameter efficiency for transfer learning tasks.
Paper: https://www.merl.com/publications/TR2024-062
- MERL researchers are presenting 5 conference papers, 3 workshop papers, and are co-organizing two workshops at the CVPR 2024 conference, which will be held in Seattle, June 17-21. CVPR is one of the most prestigious and competitive international conferences in computer vision. Details of MERL contributions are provided below.
See All News & Events for Anoop -
-
Research Highlights
-
MERL Publications
- "LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models", International Conference on Learning Representations (ICLR), October 2024.BibTeX
- @article{Cherian2024oct,
- author = {Cherian, Anoop and Corcodel, Radu and Jain, Siddarth and Romeres, Diego}},
- title = {LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models},
- journal = {International Conference on Learning Representations (ICLR)},
- year = 2024,
- month = oct
- }
, - "Disentangled Acoustic Fields For Multimodal Physical Scene Understanding", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2024.BibTeX TR2024-125 PDF
- @inproceedings{Yin2024sep,
- author = {Yin, Jie and Luo, Andrew and Du, Yilun and Cherian, Anoop and Marks, Tim K. and Le Roux, Jonathan and Gan, Chuang}},
- title = {Disentangled Acoustic Fields For Multimodal Physical Scene Understanding},
- booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2024,
- month = sep,
- url = {https://www.merl.com/publications/TR2024-125}
- }
, - "Few-shot Transparent Instance Segmentation for Bin Picking", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 2024.BibTeX TR2024-127 PDF
- @inproceedings{Cherian2024sep,
- author = {Cherian, Anoop and Jain, Siddarth and Marks, Tim K.}},
- title = {Few-shot Transparent Instance Segmentation for Bin Picking},
- booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
- year = 2024,
- month = sep,
- url = {https://www.merl.com/publications/TR2024-127}
- }
, - "Temporally Grounding Instructional Diagrams in Unconstrained Videos", arXiv, July 2024.BibTeX arXiv
- @article{Zhang2024jul4,
- author = {Zhang, Jiahao and Zhang, Frederic and Rodriguez, Cristian and Ben-Shabat, Itzik and Cherian, Anoop and Gould, Stephen}},
- title = {Temporally Grounding Instructional Diagrams in Unconstrained Videos},
- journal = {arXiv},
- year = 2024,
- month = jul,
- url = {https://arxiv.org/abs/2407.12066}
- }
, - "Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads", arXiv, June 2024.BibTeX arXiv
- @article{Cherian2024jun,
- author = {Cherian, Anoop and Peng, Kuan-Chuan and Lohit, Suhas and Matthiesen, Joanna and Smith, Kevin and Tenenbaum, Joshua B.}},
- title = {Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads},
- journal = {arXiv},
- year = 2024,
- month = jun,
- url = {https://arxiv.org/abs/2406.15736}
- }
,
- "LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models", International Conference on Learning Representations (ICLR), October 2024.
-
Other Publications
- "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.BibTeX
- @Article{cherian2018ijcv,
- author = {Cherian, Anoop and Gould, Stephen},
- title = {Second-order Temporal Pooling for Action Recognition},
- journal = {International Journal of Computer Vision (IJCV)},
- year = 2018,
- publisher = {Springer}
- }
, - "Visual Permutation Learning", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.BibTeX
- @Article{cherian2018permutation,
- author = {Santa Cruz, Rodrigo and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
- title = {Visual Permutation Learning},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2018,
- publisher = {IEEE}
- }
, - "Video Representation Learning Using Discriminative Pooling", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_representation_cvpr18,
- author = {Wang, Jue and Cherian, Anoop and Porikli, Fatih and Gould, Stephen},
- title = {Video Representation Learning Using Discriminative Pooling},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_rigid_cvpr18,
- author = {Kumar, Suryansh and Cherian, Anoop and Dai, Yuchao and Li, Hongdong},
- title = {Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Non-Linear Temporal Subspace Representations for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_temporal_cvpr18,
- author = {Cherian, Anoop and Sra, Suvrit and Gould, Stephen and Hartley, Richard},
- title = {Non-Linear Temporal Subspace Representations for Activity Recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Generalized Rank Pooling for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.BibTeX
- @Inproceedings{cherian2017generalized,
- author = {Cherian, Anoop and Fernando, Basura and Harandi, Mehrtash and Gould, Stephen},
- title = {Generalized Rank Pooling for Activity Recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2017
- }
, - "Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices", International Conference on Computer Vision (ICCV), 2017.BibTeX
- @Inproceedings{cherian_rigid_iccv17,
- author = {Cherian, Anoop and Stanitsas, Panagiotis and Harandi, Mehrtash and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices},
- booktitle = {International Conference on Computer Vision (ICCV)},
- year = 2017
- }
, - "DeepPermNet: Visual Permutation Learning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.BibTeX
- @Inproceedings{cruz2017deeppermnet,
- author = {Cruz, Rodrigo Santa and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
- title = {DeepPermNet: Visual Permutation Learning},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2017
- }
, - "Bayesian Non-Parametric clustering for positive definite matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.BibTeX
- @Article{cherian2016bayesian,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Bayesian Non-Parametric clustering for positive definite matrices},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2016,
- publisher = {IEEE}
- }
, - "Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.BibTeX
- @Inproceedings{koniusz2016sparse,
- author = {Koniusz, Piotr and Cherian, Anoop},
- title = {Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2016
- }
, - "Tensor representations via kernel linearization for action recognition from 3D skeletons", European Conference on Computer Vision (ECCV), 2016.BibTeX
- @Inproceedings{koniusz2016tensor,
- author = {Koniusz, Piotr and Cherian, Anoop and Porikli, Fatih},
- title = {Tensor representations via kernel linearization for action recognition from 3D skeletons},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2016,
- organization = {Springer}
- }
, - "Mixing body-part sequences for human pose estimation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.BibTeX
- @Inproceedings{cherian2014mixing,
- author = {Cherian, Anoop and Mairal, Julien and Alahari, Karteek and Schmid, Cordelia},
- title = {Mixing body-part sequences for human pose estimation},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2014
- }
, - "Nearest neighbors using compact sparse codes", International Conference on Machine Learning (ICML), 2014.BibTeX
- @Inproceedings{cherian2014nearest,
- author = {Cherian, Anoop},
- title = {Nearest neighbors using compact sparse codes},
- booktitle = {International Conference on Machine Learning (ICML)},
- year = 2014
- }
, - "Riemannian sparse coding for positive definite matrices", European Conference on Computer Vision (ECCV), 2014.BibTeX
- @Inproceedings{cherian2014riemannian,
- author = {Cherian, Anoop and Sra, Suvrit},
- title = {Riemannian sparse coding for positive definite matrices},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2014,
- organization = {Springer}
- }
, - "Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2013.BibTeX
- @Article{cherian2013jensen,
- author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
- title = {Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2013,
- publisher = {IEEE}
- }
, - "Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications", Computer Vision and Pattern Recognition (CVPR), 2011.BibTeX
- @Inproceedings{cherian2011dirichlet,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos and Bedros, Saad J},
- title = {Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications},
- booktitle = {Computer Vision and Pattern Recognition (CVPR)},
- year = 2011
- }
, - "Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence", International Conference on Computer Vision (ICCV), 2011.BibTeX
- @Inproceedings{cherian2011efficient,
- author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
- title = {Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence},
- booktitle = {International Conference on Computer Vision (ICCV)},
- year = 2011
- }
, - "Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval", Machine Learning and Knowledge Discovery in Databases (ECML), 2011.BibTeX
- @Article{sra2011generalized,
- author = {Sra, Suvrit and Cherian, Anoop},
- title = {Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval},
- journal = {Machine Learning and Knowledge Discovery in Databases (ECML)},
- year = 2011
- }
, - "Accurate 3D ground plane estimation from a single image", International Conference on Robotics and Automation, 2009.BibTeX
- @Inproceedings{cherian2009accurate,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Accurate 3D ground plane estimation from a single image},
- booktitle = {International Conference on Robotics and Automation},
- year = 2009
- }
,
- "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.
-
Software & Data Downloads
-
ComplexVAD Dataset -
Pixel-Grounded Prototypical Part Networks -
Steered Diffusion -
Simple Multimodal Algorithmic Reasoning Task Dataset -
Audio-Visual-Language Embodied Navigation in 3D Environments -
3D MOrphable STyleGAN -
Instance Segmentation GAN -
Audio Visual Scene-Graph Segmentor -
Generalized One-class Discriminative Subspaces -
Generating Visual Dynamics from Sound and Context -
Adversarially-Contrastive Optimal Transport -
Landmarks’ Location, Uncertainty, and Visibility Likelihood -
Gradient-based Nikaido-Isoda -
Discriminative Subspace Pooling
-
-
Videos
-
MERL Issued Patents
-
Title: "A Method and System for Scene-Aware Audio-Video Representation"
Inventors: Cherian, Anoop; Chatterjee, Moitreya; Le Roux, Jonathan
Patent No.: 12,056,213
Issue Date: Aug 6, 2024 -
Title: "Artificial Intelligence System for Classification of Data Based on Contrastive Learning"
Inventors: Cherian, Anoop; Aeron, Shuchin
Patent No.: 11,809,988
Issue Date: Nov 7, 2023 -
Title: "System and Method for Manipulating Two-Dimensional (2D) Images of Three-Dimensional (3D) Objects"
Inventors: Marks, Tim; Medin, Safa; Cherian, Anoop; Wang, Ye
Patent No.: 11,663,798
Issue Date: May 30, 2023 -
Title: "InSeGAN: A Generative Approach to Instance Segmentation in Depth Images"
Inventors: Cherian, Anoop; Pais, Goncalo; Marks, Tim; Sullivan, Alan
Patent No.: 11,651,497
Issue Date: May 16, 2023 -
Title: "Method and System for Scene-Aware Interaction"
Inventors: Hori, Chiori; Cherian, Anoop; Chen, Siheng; Marks, Tim; Le Roux, Jonathan; Hori, Takaaki; Harsham, Bret A.; Vetro, Anthony; Sullivan, Alan
Patent No.: 11,635,299
Issue Date: Apr 25, 2023 -
Title: "Scene-Aware Video Encoder System and Method"
Inventors: Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan; Marks, Tim; Sullivan, Alan
Patent No.: 11,582,485
Issue Date: Feb 14, 2023 -
Title: "Low-latency Captioning System"
Inventors: Hori, Chiori; Hori, Takaaki; Cherian, Anoop; Marks, Tim; Le Roux, Jonathan
Patent No.: 11,445,267
Issue Date: Sep 13, 2022 -
Title: "Anomaly Detector for Detecting Anomaly using Complementary Classifiers"
Inventors: Cherian, Anoop; Wang, Jue
Patent No.: 11,423,698
Issue Date: Aug 23, 2022 -
Title: "System and Method for a Dialogue Response Generation System"
Inventors: Hori, Chiori; Cherian, Anoop; Marks, Tim; Hori, Takaaki
Patent No.: 11,264,009
Issue Date: Mar 1, 2022 -
Title: "Scene-Aware Video Dialog"
Inventors: Geng, Shijie; Gao, Peng; Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan
Patent No.: 11,210,523
Issue Date: Dec 28, 2021
-
Title: "A Method and System for Scene-Aware Audio-Video Representation"