Detection, Tracking and 3D Modeling of Objects with Sparse RGB-D SLAM and Interactive Perception

    •  Almeida, D., Ataer-Cansizoglu, E., Corcodel, R., "Detection, Tracking and 3D Modeling of Objects with Sparse RGB-D SLAM and Interactive Perception", IEEE-RAS International Conference on Humanoid Robots, DOI: 10.1109/​Humanoids43949.2019.9035001, October 2019, pp. 1-8.
      BibTeX TR2019-119 PDF
      • @inproceedings{Almeida2019oct,
      • author = {Almeida, Diogo and Ataer-Cansizoglu, Esra and Corcodel, Radu},
      • title = {Detection, Tracking and 3D Modeling of Objects with Sparse RGB-D SLAM and Interactive Perception},
      • booktitle = {IEEE-RAS International Conference on Humanoid Robots},
      • year = 2019,
      • pages = {1--8},
      • month = oct,
      • doi = {10.1109/Humanoids43949.2019.9035001},
      • url = {}
      • }
  • MERL Contact:
  • Research Areas:

    Computer Vision, Robotics


We present an interactive perception system that enables an autonomous agent to deliberately interact with its environment and produce 3D object models. Our system verifies object hypotheses through interaction and simultaneously maintains 3D SLAM maps for each rigidly moving object hypothesis in the scene. We rely on depth-based segmentation and a multigroup registration scheme to classify features into various object maps. Our main contribution lies in the employment of a novel segment classification scheme that allows the system to handle incorrect object hypotheses, common in cluttered environments due to touching objects or occlusion. We start with a single map and initiate further object maps based on the outcome of depth segment classification. For each existing map, we select a segment to interact with and execute a manipulation primitive with the goal of disturbing it. If the resulting set of depth segments has at least one segment that did not follow the dominant motion pattern of its respective map, we split the map, thus yielding updated object hypotheses. We show qualitative results with a Fetch manipulator and objects of various shapes, which showcase the viability of the method for identifying and modelling multiple objects through repeated interactions.


  • Related News & Events

    •  NEWS    Radu Corcodel to present invited seminar at NYU on Robot Vision
      Date: May 4, 2022
      MERL Contact: Radu Corcodel
      Research Areas: Computer Vision, Robotics
      • Radu Corcodel, a Principal Research Scientist in MERL's Computer Vision Group, will present an overview of the Robot Perception research published by MERL for advanced manipulation. The talk will mainly cover topics pertaining to robotic manipulation in unstructured environments such as machine vision, tactile sensing and autonomous grasping. The seminar will also cover specific perception problems in non-prehensile interactions such as Contact-Implicit Trajectory Optimization and Tactile Classification, and is intended for a broader audience.