Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language
Abstract
Tactile sensing is essential for robots to achieve human-like gentle manipulation capabilities. However, existing Vision-Language-Action (VLA) models struggle to exploit tactile feedback for gentle manipulation due to the scarcity of aligned vision-tactile-language data and the lack of effective closed-loop force feedback mechanisms. To address these challenges, we introduce Tabero, a benchmark and model suite for gentle, language-conditioned robotic manipulation that demands fine-grained contact force perception. First, the Tabero benchmark addresses the scarcity of tactile data by presenting a data-efficient pipeline that repurposes open-source robot manipulation trajectories to generate a diverse set of vision-tactile-language tasks, and establishes a multidimensional evaluation protocol that measures task success alongside physical interaction quality. Second, we propose Tabero-VTLA, a Vision-Tactile-Language-Action architecture featuring a decoupled force-position command interface; the resulting force-position commands are executed by a fixed hybrid controller to enable real-time, force-aware manipulation. Evaluated on Tabero, our model maintains high task success while reducing average grip force by over 70% under gentle instructions, demonstrating its ability to modulate interaction forces based on multimodal experience.
Key Contributions
- Tabero Benchmark: A scalable pipeline that repurposes open-source robot trajectories in a high-fidelity tactile simulator (Isaac Lab + Taxim/FOTS) to generate diverse vision-tactile-language datasets, along with the first standardized protocol for quantifying gentleness in language-conditioned manipulation.
- Tabero-VTLA: A suite of force-aware VLA models that introduce a decoupled force-position command interface, enabling substantially reduced contact forces while preserving high task success through closed-loop tactile feedback.
- Comprehensive Evaluation: New process-aware metrics (Average/Maximum Grip Force, Average/Maximum Applied Force) that go beyond binary success rates to assess the quality of physical interaction.
Model Inference Results
Tabero-VTLA modulates grip force according to natural language instructions. Below we show the model executing the same task under different force instructions.
Task: Pick up the cream cheese and place it in the basket
Gentle Success
Instruction: "Gently pick up the cream cheese and place it in the basket." Significantly reduced grip force while completing the task.
Gentle Failure Case
Under extreme low-force conditions (10%), the object slips due to insufficient grip — illustrating the inherent trade-off between gentleness and reliability.
Task: Pick up the chocolate pudding and place it in the basket
Firm Success
Instruction: "Tightly pick up the chocolate pudding and place it in the basket." Standard force level applied.
Gentle Success
Instruction: "Softly pick up the chocolate pudding and place it in the basket." Force reduced by ~70% while maintaining stable grasping.
Real-World Demonstrations
Real World
Simulation
We provide real-robot data demonstrations to show that the sim-to-real gap is small, and this example will be continuously updated in the codebase.
Cube (Real Robot)
This real-robot example demonstrates stable grasping on an object with a prismatic structure.
Cube (Simulation)
This simulation example demonstrates replay-transferred real-robot trajectories on a cube object.
Cup (Real Robot)
This real-robot example demonstrates reliable grasping behavior on an object with round geometry.
Cup (Simulation)
This simulation example demonstrates replay-transferred real-robot trajectories on a cup object.
BibTeX
@misc{wu2026taberolearninggentlemanipulation,
title = {Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language},
author = {Qiwei Wu and Rui Zhang and Xin Xiang and Tao Li and Weihua Zhang and Junjie Lai and Renjing Xu},
year = {2026},
eprint = {2605.27886},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2605.27886}
}