TR2025-076

PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector


    •  Li, K., Zhang, T., Peng, K.-C., Wang, G., "PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector", IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop, June 2025.
      BibTeX TR2025-076 PDF
      • @inproceedings{Li2025jun,
      • author = {Li, Kaidong and Zhang, Tianxiao and Peng, Kuan-Chuan and Wang, Guanghui},
      • title = {{PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector}},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop},
      • year = 2025,
      • month = jun,
      • url = {https://www.merl.com/publications/TR2025-076}
      • }
  • MERL Contact:
  • Research Areas:

    Artificial Intelligence, Computer Vision, Machine Learning

Abstract:

3D object detection is crucial for autonomous driving, leveraging both LiDAR point clouds for precise depth in- formation and camera images for rich semantic informa- tion. Therefore, the multi-modal methods that combine both modalities offer more robust detection results. However, efficiently fusing LiDAR points and images remains challenging due to the domain gaps. In addition, the performance of many models is limited by the amount of high quality labeled data, which is expensive to create. The recent advances in foundation models, which use large-scale pre- training on different modalities, enable better multi-modal fusion. Combining the prompt engineering techniques for efficient training, we propose the Prompted Foundational 3D Detector (PF3Det), which integrates foundation model encoders and soft prompts to enhance LiDAR-camera feature fusion. PF3Det achieves the state-of-the-art results under limited training data, improving NDS by 1.19% and mAP by 2.42% on the nuScenes dataset, demonstrating its efficiency in 3D detection.

 

  • Related Publication

  •  Li, K., Zhang, T., Peng, K.-C., Wang, G., "PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector", arXiv, April 2025.
    BibTeX arXiv
    • @article{Li2025apr,
    • author = {Li, Kaidong and Zhang, Tianxiao and Peng, Kuan-Chuan and Wang, Guanghui},
    • title = {{PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector}},
    • journal = {arXiv},
    • year = 2025,
    • month = apr,
    • url = {https://arxiv.org/abs/2504.03563}
    • }