Abstract:

Dexterous food manipulation requires control under deformation, occlusion, and uncertain contact. We present TacSushi, a Cosmos3-based policy that gives current fingertip touch a feature-wise gated path to the deployed action head while using logged future visual, tactile, progress, and contact outcomes only as training supervision. Failed demonstrations retain consequence targets, but their actions are masked from imitation. Across 600 real-robot roll-outs, the full model reaches 68.3% average success on three in-distribution tasks and 37.5% on two held-out ingredients under a per-rollout human–VLM visual-quality rule. Matched ablations support complementary benefits from gated tactile fusion and training-only consequence prediction, while also revealing limited ingredient-level transfer and open questions about which predicted outcome matters most