Computational Sensing
Utilizing computation to improve sensing capabilities.
Our research in the area of computational sensing focuses on signal acquisition and design, signal modeling and reconstruction algorithms, including inverse problems, as well as array signal processing techniques.
Quick Links
-
Researchers
Petros T.
Boufounos
Hassan
Mansour
Dehong
Liu
Pu
(Perry)
WangPhilip V.
Orlik
Toshiaki
Koike-Akino
Yanting
Ma
Kieran
Parsons
Karl
Berntorp
Wataru
Tsujita
Mouhacine
Benosman
Ye
Wang
Joshua
Rapp
Anthony
Vetro
Jianlin
Guo
Radu
Corcodel
Vedang M.
Deshpande
Siddarth
Jain
Suhas
Lohit
Saviz
Mowlavi
Kuan-Chuan
Peng
Abraham P.
Vinod
Wael
Hajj Ali
James
Queeney
Ryoma
Yataka
-
Awards
-
AWARD MERL’s Paper on Wi-Fi Sensing Earns Top 3% Paper Recognition at ICASSP 2023, Selected as a Best Student Paper Award Finalist Date: June 9, 2023
Awarded to: Cristian J. Vaca-Rubio, Pu Wang, Toshiaki Koike-Akino, Ye Wang, Petros Boufounos and Petar Popovski
MERL Contacts: Petros T. Boufounos; Toshiaki Koike-Akino; Pu (Perry) Wang; Ye Wang
Research Areas: Artificial Intelligence, Communications, Computational Sensing, Dynamical Systems, Machine Learning, Signal ProcessingBrief- A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.
Performed during Cristian’s stay at MERL first as a visiting Marie Skłodowska-Curie Fellow and then as a full-time intern in 2022, this work capitalizes on standards-compliant Wi-Fi signals to perform indoor localization and sensing. The paper uses a neural dynamic learning framework to address technical issues such as low sampling rate and irregular sampling intervals.
ICASSP, a flagship conference of the IEEE Signal Processing Society (SPS), was hosted on the Greek island of Rhodes from June 04 to June 10, 2023. ICASSP 2023 marked the largest ICASSP in history, boasting over 4000 participants and 6128 submitted papers, out of which 2709 were accepted.
- A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.
-
AWARD Joshua Rapp wins Best Dissertation Award from the IEEE Signal Processing Society Date: December 20, 2021
Awarded to: Joshua Rapp
MERL Contact: Joshua Rapp
Research Areas: Computational Sensing, Signal ProcessingBrief- Joshua Rapp has won the 2021 Best PhD Dissertation Award from the IEEE Signal Processing Society.
The award recognizes a PhD thesis completed on a signal processing subject within the past three years for its relevant work in signal processing while stimulating further research in the field.
Dr. Rapp completed his PhD at Boston University in 2020 with a thesis entitled "Probabilistic Modeling for Single-Photon Lidar." The dissertation tackles challenges of the acquisition and processing of 3D depth maps reconstructed from time-of-flight data captured one photon at a time.
The award will be presented at the 2022 IEEE International Conference on Image Processing (ICIP) in France.
- Joshua Rapp has won the 2021 Best PhD Dissertation Award from the IEEE Signal Processing Society.
-
AWARD Petros Boufounos Elevated to IEEE Fellow Date: January 1, 2022
Awarded to: Petros T. Boufounos
MERL Contact: Petros T. Boufounos
Research Areas: Computational Sensing, Signal ProcessingBrief- MERL’s Petros Boufounos has been elevated to IEEE Fellow, effective January 2022, for “contributions to compressed sensing.”
IEEE Fellow is the highest grade of membership of the IEEE. It honors members with an outstanding record of technical achievements, contributing importantly to the advancement or application of engineering, science and technology, and bringing significant value to society. Each year, following a rigorous evaluation procedure, the IEEE Fellow Committee recommends a select group of recipients for elevation to IEEE Fellow. Less than 0.1% of voting members are selected annually for this member grade elevation.
- MERL’s Petros Boufounos has been elevated to IEEE Fellow, effective January 2022, for “contributions to compressed sensing.”
See All Awards for Computational Sensing -
-
News & Events
-
TALK [MERL Seminar Series 2023] Dr. Kristina Monakhova presents talk titled Robust and Physics-informed machine learning for low light imaging Date & Time: Tuesday, November 28, 2023; 12:00 PM
Speaker: Kristina Monakhova, MIT and Cornell
MERL Host: Joshua Rapp
Research Areas: Computational Sensing, Computer Vision, Machine Learning, Signal ProcessingAbstract- Imaging in low light settings is extremely challenging due to low photon counts, both in photography and in microscopy. In photography, imaging under low light, high gain settings often results in highly structured, non-Gaussian sensor noise that’s hard to characterize or denoise. In this talk, we address this by developing a GAN-tuned physics-based noise model to more accurately represent camera noise at the lowest light, and highest gain settings. Using this noise model, we train a video denoiser using synthetic data and demonstrate photorealistic videography at starlight (submillilux levels of illumination) for the first time.
For multiphoton microscopy, which is a form a scanning microscopy, there’s a trade-off between field of view, phototoxicity, acquisition time, and image quality, often resulting in noisy measurements. While deep learning-based methods have shown compelling denoising performance, can we trust these methods enough for critical scientific and medical applications? In the second part of this talk, I’ll introduce a learned, distribution-free uncertainty quantification technique that can both denoise and predict pixel-wise uncertainty to gauge how much we can trust our denoiser’s performance. Furthermore, we propose to leverage this learned, pixel-wise uncertainty to drive an adaptive acquisition technique that rescans only the most uncertain regions of a sample. With our sample and algorithm-informed adaptive acquisition, we demonstrate a 120X improvement in total scanning time and total light dose for multiphoton microscopy, while successfully recovering fine structures within the sample.
- Imaging in low light settings is extremely challenging due to low photon counts, both in photography and in microscopy. In photography, imaging under low light, high gain settings often results in highly structured, non-Gaussian sensor noise that’s hard to characterize or denoise. In this talk, we address this by developing a GAN-tuned physics-based noise model to more accurately represent camera noise at the lowest light, and highest gain settings. Using this noise model, we train a video denoiser using synthetic data and demonstrate photorealistic videography at starlight (submillilux levels of illumination) for the first time.
-
NEWS MERL Researchers give a Tutorial Talk on Quantum Machine Learning for Sensing and Communications at IEEE VCC Date: November 28, 2023 - November 30, 2023
Where: Virtual
MERL Contacts: Toshiaki Koike-Akino; Pu (Perry) Wang
Research Areas: Artificial Intelligence, Communications, Computational Sensing, Machine Learning, Signal ProcessingBrief- On November 28, 2023, MERL researchers Toshiaki Koike-Akino and Pu (Perry) Wang will give a 3-hour tutorial presentation at the first IEEE Virtual Conference on Communications (VCC). The talk, titled "Post-Deep Learning Era: Emerging Quantum Machine Learning for Sensing and Communications," addresses recent trends, challenges, and advances in sensing and communications. P. Wang presents use cases, industry trends, signal processing, and deep learning for Wi-Fi integrated sensing and communications (ISAC), while T. Koike-Akino discusses the future of deep learning, giving a comprehensive overview of artificial intelligence (AI) technologies, natural computing, emerging quantum AI, and their diverse applications. The tutorial is conducted virtually.
IEEE VCC is a new fully virtual conference launched from the IEEE Communications Society, gathering researchers from academia and industry who are unable to travel but wish to present their recent scientific results and engage in conducive interactive discussions with fellow researchers working in their fields. It is designed to resolve potential hardship such as pandemic restrictions, visa issues, travel problems, or financial difficulties.
- On November 28, 2023, MERL researchers Toshiaki Koike-Akino and Pu (Perry) Wang will give a 3-hour tutorial presentation at the first IEEE Virtual Conference on Communications (VCC). The talk, titled "Post-Deep Learning Era: Emerging Quantum Machine Learning for Sensing and Communications," addresses recent trends, challenges, and advances in sensing and communications. P. Wang presents use cases, industry trends, signal processing, and deep learning for Wi-Fi integrated sensing and communications (ISAC), while T. Koike-Akino discusses the future of deep learning, giving a comprehensive overview of artificial intelligence (AI) technologies, natural computing, emerging quantum AI, and their diverse applications. The tutorial is conducted virtually.
See All News & Events for Computational Sensing -
-
Research Highlights
-
Internships
-
ST2083: Deep Learning for Radar Perception
The Computation Sensing team at MERL is seeking a highly motivated intern to conduct fundamental research in radar perception. Expertise in deep learning-based object detection, multiple object tracking, data association, and representation learning (detection points, heatmaps, and raw radar waveforms) is required. Previous hands-on experience on open indoor/outdoor radar datasets is a plus. Familiarity with the concept of FMCW, MIMO, and range-Doppler-angle spectrum is an asset. The intern will collaborate with a small group of MERL researchers to develop novel algorithms, design experiments with MERL in-house testbed, and prepare results for patents and publication. The expected duration of the internship is 3 months with a flexible start date.
-
ST1763: Technologies for Multimodal Tracking and Imaging
MERL is seeking a motivated intern to assist in developing hardware and algorithms for multimodal imaging applications. The project involves integration of radar, camera, and depth sensors in a variety of sensing scenarios. The ideal candidate should have experience with FMCW radar and/or depth sensing, and be fluent in Python and scripting methods. Familiarity with optical tracking of humans and experience with hardware prototyping is desired. Good knowledge of computational imaging and/or radar imaging methods is a plus.
-
ST2082: Integrated Sensing and Communication (ISAC)
The Computational Sensing team at MERL is seeking a highly motivated intern to conduct fundamental research in integrated sensing and communication (ISAC) with a focus on signal processing, model-based learning, and optimization. Expertise in joint waveform/sequence optimization, integrated ISAC precoder/combiner design, model-based learning for ISAC, and downlink/uplink/active sensing under timing and frequency offsets is highly desired. Familiarity with IEEE 802.11 (ac/ax/ad/ay) standards is a plus but not required. The intern will collaborate with a small group of MERL researchers to develop novel algorithms, design experiments using MERL in-house testbed, and prepare results for publication. The expected duration of the internship is 3 months with a flexible start date.
See All Internships for Computational Sensing -
-
Recent Publications
- "Object Trajectory Estimation with Multi-Band Wi-Fi Neural Dynamic Fusion", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2024.BibTeX TR2024-019 PDF
- @inproceedings{Kato2024mar,
- author = {Kato, Sorachi and Wang, Pu and Koike-Akino, Toshiaki and Fujihashi, Takuya and Mansour, Hassan and Boufounos, Petros T.},
- title = {Object Trajectory Estimation with Multi-Band Wi-Fi Neural Dynamic Fusion},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2024,
- month = mar,
- url = {https://www.merl.com/publications/TR2024-019}
- }
, - "A model of spatial resolution uncertainty for Compton camera imaging", International Conference on Advancements in Nuclear Instrumentation Measurement Methods and their Applications (ANIMMA), DOI: 10.1051/epjconf/202328810002, January 2024, pp. 10002.BibTeX TR2024-005 PDF
- @inproceedings{Ma2024jan,
- author = {Ma, Yanting and Rapp, Joshua and Boufounos, Petros T. and Mansour, Hassan},
- title = {A model of spatial resolution uncertainty for Compton camera imaging},
- booktitle = {Advancements in Nuclear Instrumentation Measurement Methods and their Applications (ANIMMA)},
- year = 2024,
- pages = 10002,
- month = jan,
- publisher = {EPJ Web of Conferences, 288},
- doi = {10.1051/epjconf/202328810002},
- url = {https://www.merl.com/publications/TR2024-005}
- }
, - "The Role of Detection Times in Reflectivity Estimation with Single-Photon Lidar", IEEE Journal of Selected Topics in Quantum Electronics, DOI: 10.1109/JSTQE.2023.3333834, Vol. 30, No. 1, pp. 8800114:1-14, January 2024.BibTeX TR2024-003 PDF
- @article{Kitichotkul2024jan,
- author = {Kitichotkul, Ruangrawee and Rapp, Joshua and Goyal, Vivek K},
- title = {The Role of Detection Times in Reflectivity Estimation with Single-Photon Lidar},
- journal = {IEEE Journal of Selected Topics in Quantum Electronics},
- year = 2024,
- volume = 30,
- number = 1,
- pages = {8800114:1--14},
- month = jan,
- doi = {10.1109/JSTQE.2023.3333834},
- url = {https://www.merl.com/publications/TR2024-003}
- }
, - "Dual Parametric and State Estimation for Partial Differential Equations", IEEE Conference on Decision and Control, DOI: 10.1109/CDC49753.2023.10384246, December 2023, pp. 8156-8161.BibTeX TR2023-145 PDF
- @inproceedings{Mowlavi2023dec,
- author = {Mowlavi, Saviz and Benosman, Mouhacine},
- title = {Dual Parametric and State Estimation for Partial Differential Equations},
- booktitle = {IEEE Conference on Decision and Control (CDC)},
- year = 2023,
- pages = {8156--8161},
- month = dec,
- publisher = {IEEE},
- doi = {10.1109/CDC49753.2023.10384246},
- issn = {2576-2370},
- isbn = {979-8-3503-0125-0},
- url = {https://www.merl.com/publications/TR2023-145}
- }
, - "RIS-Assisted Joint Preamble Detection and Localization", IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), DOI: 10.1109/CAMSAP58249.2023.10403493, December 2023.BibTeX TR2023-142 PDF
- @inproceedings{Nuti2023dec,
- author = {Nuti, Pooja and Kim, Kyeong Jin and Wang, Pu and Koike-Akino, Toshiaki and Parsons, Kieran},
- title = {RIS-Assisted Joint Preamble Detection and Localization},
- booktitle = {IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)},
- year = 2023,
- month = dec,
- doi = {10.1109/CAMSAP58249.2023.10403493},
- isbn = {979-8-3503-4453-0},
- url = {https://www.merl.com/publications/TR2023-142}
- }
, - "Learning-Based THz Multi-Layer Imaging With Model-Based Masks", International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz), DOI: 10.1109/IRMMW-THz57677.2023.10299043, September 2023.BibTeX TR2024-017 PDF
- @inproceedings{Wang2023sep2,
- author = {Wang, Pu and Koike-Akino, Toshiaki and Boufounos, Petros T. and Tsujita, Wataru and Yamashita, Genki},
- title = {Learning-Based THz Multi-Layer Imaging With Model-Based Masks},
- booktitle = {International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz)},
- year = 2023,
- month = sep,
- doi = {10.1109/IRMMW-THz57677.2023.10299043},
- issn = {2162-2035},
- isbn = {979-8-3503-3661-0},
- url = {https://www.merl.com/publications/TR2024-017}
- }
, - "Sensing and Machine Learning for Automotive Perception: A Review", IEEE Sensors Journal, DOI: 10.1109/JSEN.2023.3262134, Vol. 23, No. 11, pp. 11097-11115, June 2023.BibTeX TR2023-089 PDF
- @article{Pandharipande2023jun,
- author = {Pandharipande, Ashish and Cheng, Chih-Hong and Dauwels, Justin and Gurbuz, Sevgi and Ibanez-Guzman, Javier and Li, Guofa and Piazzoni, Andrea and Wang, Pu and Santra, Avik},
- title = {Sensing and Machine Learning for Automotive Perception: A Review},
- journal = {IEEE Sensors Journal},
- year = 2023,
- volume = 23,
- number = 11,
- pages = {11097--11115},
- month = jun,
- doi = {10.1109/JSEN.2023.3262134},
- issn = {1558-1748},
- url = {https://www.merl.com/publications/TR2023-089}
- }
,
- "Object Trajectory Estimation with Multi-Band Wi-Fi Neural Dynamic Fusion", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2024.
-
Videos
-
Software & Data Downloads