TR2026-142

Test-Time Attention: Can Robots Better Follow Commands?

    •  Liu, J., Wang, Y., Suzuki, K., Koike-Akino, T., "Test-Time Attention: Can Robots Better Follow Commands?", IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IARL Workshop, September 2026.
      BibTeX TR2026-142 PDF
      • @inproceedings{Liu2026sep,
      • author = {Liu, Jing and Wang, Ye and Suzuki, Kei and Koike-Akino, Toshiaki},
      • title = {{Test-Time Attention: Can Robots Better Follow Commands?}},
      • booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IARL Workshop},
      • year = 2026,
      • month = sep,
      • url = {https://www.merl.com/publications/TR2026-142}
      • }
  • MERL Contacts:
  • Research Areas:

    Artificial Intelligence, Machine Learning

Abstract:

Generalist Vision-Language-Action (VLA) policies promise broad manipulation capabilities, but the same breadth can weaken fidelity to a narrow deployment command. Repeated scene–object–action correlations and the large number of visual tokens relative to language tokens may cause visually suggested behaviors to compete with the operator’s current command. We ask whether an already deployed VLA can be made more command-focused by redistributing attention only at inference time, without new demonstrations or policy retraining. Using pretrained pi 0.5 policy on a R1 Pro wheeled humanoid, we propose and evaluate 7 heuristic training-free interventions that either strengthen the current subtask text or reduce the influence of less command-relevant visual information in an outfitting toolbox setting. Because task success alone can conceal interactions with objects not specified by the current command, we introduce the Command Focus Ratio (CFR), which measures interaction selectivity, and the Target Interaction Rate (TIR), which measures how often the commanded target is engaged. Empirical studies suggest that test-time attention methods can improve the command fidelity of VLA policies without additional data collection or retraining, while the remaining gap in CFR highlights the need for stronger test-time mechanisms to suppress interactions with non-target objects.

 

  • Related News & Events

    •  NEWS    MERL contributes to IROS 2026
      Date: September 27, 2026 - October 1, 2026
      Where: The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
      MERL Contacts: Siddarth Jain; Toshiaki Koike-Akino; Jing Liu; Daniel N. Nikovski; Arvind Raghunathan; Alexander Schperberg; Kei Suzuki; Ye Wang
      Research Areas: Artificial Intelligence, Computer Vision, Control, Machine Learning, Optimization, Robotics, Signal Processing
      Brief
      • MERL made broad contributions to the technical program and robotics community at the 2026 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026), held in Pittsburgh, Pennsylvania. MERL researchers presented one main-conference paper and five workshop papers, participated in an editorial board meeting, competed in the Humanoid IKEA Assembly Challenge, and contributed to the organization of an IROS workshop.

        Main Conference Paper
        • ORIGAMI: Object Representation Inferred Geometrically for Articulated ManIpulation, Yunfu Deng and Daniel N. Nikovski (TR2026-141)


          • The work introduces a geometric representation for articulated-object manipulation, enabling robots to infer compact representations of previously unknown articulated mechanisms for downstream learning and control.

        Workshop Papers
        • Deliberate Practice: Learning Robot Skills under a Budget Shivam Vats, Sudarshan Harithas, Mete Akbulut, Arvind Raghunathan, and George Konidaris (TR2026-147)


        • ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies, Haodi Hu, Chung-Ta Huang, Jing Liu, Ye Wang, Kei Suzuki, Matthew Brand, and Toshiaki Koike-Akino (TR2026-148)


        • Test-Time Attention: Can Robots Better Follow Commands? Jing Liu, Ye Wang, Kei Suzuki, and Toshiaki Koike-Akino (TR2026-142)


        • Read the Manual: Grounding Behavior Tree Synthesis for Multi-Machine Factory Operation, Chak Lam Shek, Ye Wang, Jing Liu, Kei Suzuki, Pratap Tokekar, and Toshiaki Koike-Akino (TR2026-149)


        • DYNAMIT: Dynamic Zero-Shot Manipulation via Instruction-Grounded Visual Tracking and Interception in Industrial Conveyor Environments, Harsh Singh, Kei Suzuki, Ye Wang, Jing Liu, Paola Cascante-Bonilla, and Toshiaki Koike-Akino (TR2026-146)

          Together, these works address a range of challenges in modern robotics, including articulated-object manipulation, efficient robot skill learning, vision-language-action policies, test-time adaptation, autonomous factory operation, and dynamic manipulation. The paper on ReCoVLA was nominated as a spotlight talk.


        Robotics Community Contributions

        MERL Principal Research Scientist Dr. Siddarth Jain participated in the IEEE Robotics and Automation Letters (RA-L) editorial board meeting at IROS. Dr. Jain serves as an Associate Editor of RA-L, contributing to the peer-review and editorial activities of the robotics research community.

        Former MERL scientist Dr. Diego Romeres was also among the organizers of the 1st International Workshop on Industrial Applications of Robot Learning (IARL 2026). The workshop brought together researchers from academia and industry to discuss how advances in robot learning can be translated into reliable and scalable real-world industrial robotic systems.

        Humanoid IKEA Assembly Challenge

        MERL also participated in the IROS 2026 Humanoid IKEA Assembly Challenge with Team MEL-Craft. The team achieved first place in the competition, demonstrating autonomous humanoid manipulation capabilities for a challenging furniture-assembly task. The MEL-Craft team included MERL researchers Kei Suzuki, Jing Liu, Alexander Schperberg, Toshiaki Koike-Akino, and Ye Wang, with contributions from MERL interns Haodi Hu, Harsh Singh, Chak Lam Shek, and Maxwell Asselmeier. The competition achievement is highlighted separately in MERL's related award announcement.

        About IROS

        The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) is a major international robotics conference bringing together researchers, engineers, and industry practitioners working across intelligent robots and systems. IROS 2026 took place in Pittsburgh from September 27 to October 1, 2026.

    •