Anoop Cherian

- Phone: 617-621-7519
- Email:
-
Position:
Research / Technical Staff
Senior Principal Research Scientist -
Education:
Ph.D., University of Minnesota, 2013 -
Research Areas:
- Computer Vision
- Machine Learning
- Artificial Intelligence
- Speech & Audio
- Human-Computer Interaction
- Optimization
- Robotics
External Links:
Anoop's Quick Links
-
Biography
Anoop was a postdoctoral researcher in the LEAR group at Inria from 2012-2015 where his research was on the estimation and tracking of human poses in videos. From 2015-2017, he was a Research Fellow at the Australian National University, where he worked on the problem of recognizing human activities in video sequences. Anoop is the recipient of the Best Student Paper award at the Intl. Conference on Image Processing in 2012. Currently, his research focus is on modeling the semantics of video data.
-
Recent News & Events
-
NEWS Anoop Cherian gives a podcast interview with AI Business Date: September 26, 2023
Where: Virtual
MERL Contact: Anoop Cherian
Research Areas: Artificial Intelligence, Computer Vision, Machine LearningBrief- Anoop Cherian, a Senior Principal Research Scientist in the Computer Vision team at MERL, gave a podcast interview with award-winning journalist, Deborah Yao. Deborah is the editor of AI Business -- a leading content platform for artificial intelligence and its applications in the real world, delivering its readers up-to-the-minute insights into how AI technologies are currently affecting the global economy and society. The podcast was based on the recent research that Anoop and his colleagues did at MERL with his collaborators at MIT; this research attempts to objectively answer the pertinent question: are current deep neural networks smarter than second graders? The podcast discusses shortcomings in the recent artificial general intelligence systems with regard to their capabilities for knowledge abstraction, learning, and generalization, which are brought out by this research.
-
NEWS MERL researchers presenting four papers and organizing the VLAR-SMART101 Workshop at ICCV 2023 Date: October 2, 2023 - October 6, 2023
Where: Paris/France
MERL Contacts: Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Toshiaki Koike-Akino; Suhas Lohit; Tim K. Marks; Pedro Miraldo; Kuan-Chuan Peng; Ye Wang
Research Areas: Artificial Intelligence, Computer Vision, Machine LearningBrief- MERL researchers are presenting 4 papers and organizing the VLAR-SMART-101 workshop at the ICCV 2023 conference, which will be held in Paris, France October 2-6. ICCV is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.
1. Conference paper: “Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis,” by Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal Patel, and Tim K. Marks
Conditional generative models typically demand large annotated training sets to achieve high-quality synthesis. As a result, there has been significant interest in plug-and-play generation, i.e., using a pre-defined model to guide the generative process. In this paper, we introduce Steered Diffusion, a generalized framework for fine-grained photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation. The key idea is to steer the image generation of the diffusion model during inference via designing a loss using a pre-trained inverse model that characterizes the conditional task. Our model shows clear qualitative and quantitative improvements over state-of-the-art diffusion-based plug-and-play models, while adding negligible computational cost.
2. Conference paper: "BANSAC: A dynamic BAyesian Network for adaptive SAmple Consensus," by Valter Piedade and Pedro Miraldo
We derive a dynamic Bayesian network that updates individual data points' inlier scores while iterating RANSAC. At each iteration, we apply weighted sampling using the updated scores. Our method works with or without prior data point scorings. In addition, we use the updated inlier/outlier scoring for deriving a new stopping criterion for the RANSAC loop. Our method outperforms the baselines in accuracy while needing less computational time.
3. Conference paper: "Robust Frame-to-Frame Camera Rotation Estimation in Crowded Scenes," by Fabien Delattre, David Dirnfeld, Phat Nguyen, Stephen Scarano, Michael J. Jones, Pedro Miraldo, and Erik Learned-Miller
We present a novel approach to estimating camera rotation in crowded, real-world scenes captured using a handheld monocular video camera. Our method uses a novel generalization of the Hough transform on SO3 to efficiently find the camera rotation most compatible with the optical flow. Because the setting is not addressed well by other data sets, we provide a new dataset and benchmark, with high-accuracy and rigorously annotated ground truth on 17 video sequences. Our method is more accurate by almost 40 percent than the next best method.
4. Workshop paper: "Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection" by Manish Sharma*, Moitreya Chatterjee*, Kuan-Chuan Peng, Suhas Lohit, and Michael Jones
While state-of-the-art object detection methods for RGB images have reached some level of maturity, the same is not true for Infrared (IR) images. The primary bottleneck towards bridging this gap is the lack of sufficient labeled training data in the IR images. Towards addressing this issue, we present TensorFact, a novel tensor decomposition method which splits the convolution kernels of a CNN into low-rank factor matrices with fewer parameters. This compressed network is first pre-trained on RGB images and then augmented with only a few parameters. This augmented network is then trained on IR images, while freezing the weights trained on RGB. This prevents it from over-fitting, allowing it to generalize better. Experiments show that our method outperforms state-of-the-art.
5. “Vision-and-Language Algorithmic Reasoning (VLAR) Workshop and SMART-101 Challenge” by Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Tim K. Marks, Ram Ramrakhya, Honglu Zhou, Kevin A. Smith, Joanna Matthiesen, and Joshua B. Tenenbaum
MERL researchers along with researchers from MIT, GeorgiaTech, Math Kangaroo USA, and Rutgers University are jointly organizing a workshop on vision-and-language algorithmic reasoning at ICCV 2023 and conducting a challenge based on the SMART-101 puzzles described in the paper: Are Deep Neural Networks SMARTer than Second Graders?. A focus of this workshop is to bring together outstanding faculty/researchers working at the intersections of vision, language, and cognition to provide their opinions on the recent breakthroughs in large language models and artificial general intelligence, as well as showcase their cutting edge research that could inspire the audience to search for the missing pieces in our quest towards solving the puzzle of artificial intelligence.
Workshop link: https://wvlar.github.io/iccv23/
- MERL researchers are presenting 4 papers and organizing the VLAR-SMART-101 workshop at the ICCV 2023 conference, which will be held in Paris, France October 2-6. ICCV is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.
See All News & Events for Anoop -
-
Research Highlights
-
Internships with Anoop
-
CV2113: Embodied Multimodal Large Language Models
MERL is looking for a self-motivated intern to work on problems at the intersection of multimodal large language models and embodied AI in dynamic indoor environments. The ideal candidate would be a PhD student with a strong background in machine learning and computer vision, as demonstrated by top-tier publications. The candidate must have prior experience in audio-visual AI, large language models, and simulators such as Habitat/SoundSpaces. Hands on experience in using animated 3D human shape models (e.g., SMPL and variants) is desired. The intern is expected to collaborate with researchers in computer vision and speech teams at MERL to develop algorithms and prepare manuscripts for scientific publications.
-
CV2121: Simulators and Generative AI for Task Driven Data Generation
MERL is looking for a self-motivated intern to develop a general-purpose simulation platform for generating computer vision datasets defined by downstream tasks. The ideal intern must have strong background in computer graphics, computer vision, and machine learning, as well as experience in using the latest graphics simulation toolboxes and physics engines. Working knowledge of recent multimodal generative AI methods will be a strong plus. The intern is expected to collaborate with researchers in the computer vision team at MERL to develop algorithms and prepare manuscripts for scientific publications.
-
-
MERL Publications
- "Pixel-Grounded Prototypical Part Networks", arXiv, October 2023. ,
- "Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis", IEEE International Conference on Computer Vision (ICCV), October 2023.BibTeX TR2023-126 PDF Presentation
- @inproceedings{Nair2023sep,
- author = {Nair, Nithin Gopalakrishnan and Cherian, Anoop and Lohit, Suhas and Wang, Ye and Koike-Akino, Toshiaki and Patel, Vishal M. and Marks, Tim K.},
- title = {Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis},
- booktitle = {IEEE International Conference on Computer Vision (ICCV)},
- year = 2023,
- month = oct,
- url = {https://www.merl.com/publications/TR2023-126}
- }
, - "Active Sparse Conversations for Improved Audio-Visual Embodied Navigation", arXiv, June 2023. ,
- "H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/ICRA48891.2023.10160575, May 2023, pp. 7272-7278.BibTeX TR2023-009 PDF
- @inproceedings{Ota2023may,
- author = {Ota, Kei and Tung, Hsiao-Yu and Smith, Kevin and Cherian, Anoop and Marks, Tim K. and Sullivan, Alan and Kanezaki, Asako and Tenenbaum, Joshua B.},
- title = {H-SAUR: Hypothesize, Simulate, Act, Update, and Repeat for Understanding Object Articulations from Interactions},
- booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
- year = 2023,
- pages = {7272--7278},
- month = may,
- publisher = {IEEE},
- doi = {10.1109/ICRA48891.2023.10160575},
- url = {https://www.merl.com/publications/TR2023-009}
- }
, - "Discriminative 3D Shape Modeling for Few-Shot Instance Segmentation", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/ICRA48891.2023.10160644, May 2023, pp. 9296-9302.BibTeX TR2023-010 PDF Presentation
- @inproceedings{Cherian2023may,
- author = {Cherian, Anoop and Jain, Siddarth and Marks, Tim K. and Sullivan, Alan},
- title = {Discriminative 3D Shape Modeling for Few-Shot Instance Segmentation},
- booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
- year = 2023,
- pages = {9296--9302},
- month = may,
- publisher = {IEEE},
- doi = {10.1109/ICRA48891.2023.10160644},
- url = {https://www.merl.com/publications/TR2023-010}
- }
,
-
Other Publications
- "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.BibTeX
- @Article{cherian2018ijcv,
- author = {Cherian, Anoop and Gould, Stephen},
- title = {Second-order Temporal Pooling for Action Recognition},
- journal = {International Journal of Computer Vision (IJCV)},
- year = 2018,
- publisher = {Springer}
- }
, - "Visual Permutation Learning", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018.BibTeX
- @Article{cherian2018permutation,
- author = {Santa Cruz, Rodrigo and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
- title = {Visual Permutation Learning},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2018,
- publisher = {IEEE}
- }
, - "Video Representation Learning Using Discriminative Pooling", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_representation_cvpr18,
- author = {Wang, Jue and Cherian, Anoop and Porikli, Fatih and Gould, Stephen},
- title = {Video Representation Learning Using Discriminative Pooling},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_rigid_cvpr18,
- author = {Kumar, Suryansh and Cherian, Anoop and Dai, Yuchao and Li, Hongdong},
- title = {Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Non-Linear Temporal Subspace Representations for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.BibTeX
- @Inproceedings{cherian_temporal_cvpr18,
- author = {Cherian, Anoop and Sra, Suvrit and Gould, Stephen and Hartley, Richard},
- title = {Non-Linear Temporal Subspace Representations for Activity Recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2018
- }
, - "Generalized Rank Pooling for Activity Recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.BibTeX
- @Inproceedings{cherian2017generalized,
- author = {Cherian, Anoop and Fernando, Basura and Harandi, Mehrtash and Gould, Stephen},
- title = {Generalized Rank Pooling for Activity Recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2017
- }
, - "Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices", International Conference on Computer Vision (ICCV), 2017.BibTeX
- @Inproceedings{cherian_rigid_iccv17,
- author = {Cherian, Anoop and Stanitsas, Panagiotis and Harandi, Mehrtash and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Learning Discriminative Alpha-Beta Divergences for Positive Definite Matrices},
- booktitle = {International Conference on Computer Vision (ICCV)},
- year = 2017
- }
, - "DeepPermNet: Visual Permutation Learning", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.BibTeX
- @Inproceedings{cruz2017deeppermnet,
- author = {Cruz, Rodrigo Santa and Fernando, Basura and Cherian, Anoop and Gould, Stephen},
- title = {DeepPermNet: Visual Permutation Learning},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2017
- }
, - "Bayesian Non-Parametric clustering for positive definite matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.BibTeX
- @Article{cherian2016bayesian,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Bayesian Non-Parametric clustering for positive definite matrices},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2016,
- publisher = {IEEE}
- }
, - "Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.BibTeX
- @Inproceedings{koniusz2016sparse,
- author = {Koniusz, Piotr and Cherian, Anoop},
- title = {Sparse coding for third-order super-symmetric tensor descriptors with application to texture recognition},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2016
- }
, - "Tensor representations via kernel linearization for action recognition from 3D skeletons", European Conference on Computer Vision (ECCV), 2016.BibTeX
- @Inproceedings{koniusz2016tensor,
- author = {Koniusz, Piotr and Cherian, Anoop and Porikli, Fatih},
- title = {Tensor representations via kernel linearization for action recognition from 3D skeletons},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2016,
- organization = {Springer}
- }
, - "Mixing body-part sequences for human pose estimation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.BibTeX
- @Inproceedings{cherian2014mixing,
- author = {Cherian, Anoop and Mairal, Julien and Alahari, Karteek and Schmid, Cordelia},
- title = {Mixing body-part sequences for human pose estimation},
- booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
- year = 2014
- }
, - "Nearest neighbors using compact sparse codes", International Conference on Machine Learning (ICML), 2014.BibTeX
- @Inproceedings{cherian2014nearest,
- author = {Cherian, Anoop},
- title = {Nearest neighbors using compact sparse codes},
- booktitle = {International Conference on Machine Learning (ICML)},
- year = 2014
- }
, - "Riemannian sparse coding for positive definite matrices", European Conference on Computer Vision (ECCV), 2014.BibTeX
- @Inproceedings{cherian2014riemannian,
- author = {Cherian, Anoop and Sra, Suvrit},
- title = {Riemannian sparse coding for positive definite matrices},
- booktitle = {European Conference on Computer Vision (ECCV)},
- year = 2014,
- organization = {Springer}
- }
, - "Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2013.BibTeX
- @Article{cherian2013jensen,
- author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
- title = {Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices},
- journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
- year = 2013,
- publisher = {IEEE}
- }
, - "Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications", Computer Vision and Pattern Recognition (CVPR), 2011.BibTeX
- @Inproceedings{cherian2011dirichlet,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos and Bedros, Saad J},
- title = {Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications},
- booktitle = {Computer Vision and Pattern Recognition (CVPR)},
- year = 2011
- }
, - "Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence", International Conference on Computer Vision (ICCV), 2011.BibTeX
- @Inproceedings{cherian2011efficient,
- author = {Cherian, Anoop and Sra, Suvrit and Banerjee, Arindam and Papanikolopoulos, Nikolaos},
- title = {Efficient similarity search for covariance matrices via the Jensen-Bregman LogDet divergence},
- booktitle = {International Conference on Computer Vision (ICCV)},
- year = 2011
- }
, - "Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval", Machine Learning and Knowledge Discovery in Databases (ECML), 2011.BibTeX
- @Article{sra2011generalized,
- author = {Sra, Suvrit and Cherian, Anoop},
- title = {Generalized dictionary learning for symmetric positive definite matrices with application to nearest neighbor retrieval},
- journal = {Machine Learning and Knowledge Discovery in Databases (ECML)},
- year = 2011
- }
, - "Accurate 3D ground plane estimation from a single image", International Conference on Robotics and Automation, 2009.BibTeX
- @Inproceedings{cherian2009accurate,
- author = {Cherian, Anoop and Morellas, Vassilios and Papanikolopoulos, Nikolaos},
- title = {Accurate 3D ground plane estimation from a single image},
- booktitle = {International Conference on Robotics and Automation},
- year = 2009
- }
,
- "Second-order Temporal Pooling for Action Recognition", International Journal of Computer Vision (IJCV), 2018.
-
Software & Data Downloads
-
Simple Multimodal Algorithmic Reasoning Task Dataset -
Audio-Visual-Language Embodied Navigation in 3D Environments -
Instance Segmentation GAN -
Audio Visual Scene-Graph Segmentor -
Generalized One-class Discriminative Subspaces -
Generating Visual Dynamics from Sound and Context -
Adversarially-Contrastive Optimal Transport -
Landmarks’ Location, Uncertainty, and Visibility Likelihood -
Gradient-based Nikaido-Isoda -
Discriminative Subspace Pooling
-
-
Videos
-
MERL Issued Patents
-
Title: "Artificial Intelligence System for Classification of Data Based on Contrastive Learning"
Inventors: Cherian, Anoop; Aeron, Shuchin
Patent No.: 11,809,988
Issue Date: Nov 7, 2023 -
Title: "System and Method for Manipulating Two-Dimensional (2D) Images of Three-Dimensional (3D) Objects"
Inventors: Marks, Tim; Medin, Safa; Cherian, Anoop; Wang, Ye
Patent No.: 11,663,798
Issue Date: May 30, 2023 -
Title: "InSeGAN: A Generative Approach to Instance Segmentation in Depth Images"
Inventors: Cherian, Anoop; Pais, Goncalo; Marks, Tim; Sullivan, Alan
Patent No.: 11,651,497
Issue Date: May 16, 2023 -
Title: "Method and System for Scene-Aware Interaction"
Inventors: Hori, Chiori; Cherian, Anoop; Chen, Siheng; Marks, Tim; Le Roux, Jonathan; Hori, Takaaki; Harsham, Bret A.; Vetro, Anthony; Sullivan, Alan
Patent No.: 11,635,299
Issue Date: Apr 25, 2023 -
Title: "Scene-Aware Video Encoder System and Method"
Inventors: Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan; Marks, Tim; Sullivan, Alan
Patent No.: 11,582,485
Issue Date: Feb 14, 2023 -
Title: "Low-latency Captioning System"
Inventors: Hori, Chiori; Hori, Takaaki; Cherian, Anoop; Marks, Tim; Le Roux, Jonathan
Patent No.: 11,445,267
Issue Date: Sep 13, 2022 -
Title: "Anomaly Detector for Detecting Anomaly using Complementary Classifiers"
Inventors: Cherian, Anoop; Wang, Jue
Patent No.: 11,423,698
Issue Date: Aug 23, 2022 -
Title: "System and Method for a Dialogue Response Generation System"
Inventors: Hori, Chiori; Cherian, Anoop; Marks, Tim; Hori, Takaaki
Patent No.: 11,264,009
Issue Date: Mar 1, 2022 -
Title: "Scene-Aware Video Dialog"
Inventors: Geng, Shijie; Gao, Peng; Cherian, Anoop; Hori, Chiori; Le Roux, Jonathan
Patent No.: 11,210,523
Issue Date: Dec 28, 2021
-
Title: "Artificial Intelligence System for Classification of Data Based on Contrastive Learning"