ViNL: Visual Navigation and Locomotion Over Obstacles

Georgia Institute of Technology

International Conference on Robotics and Automation (ICRA), 2023
Best Paper Award at Learning for Agile Robotics Workshop (CoRL) , 2022

arXiv Navigation GitHub Locomotion GitHub Press Coverage

We present Visual Navigation and Locomotion over obstacles (ViNL), which enables a quadrupedal robot to navigate unseen apartments while stepping over small obstacles that lie in its path (e.g., shoes, toys, cables), similar to how humans and pets lift their feet over objects as they walk. ViNL consists of: (1) a visual navigation policy that outputs linear and angular velocity commands that guides the robot to a goal coordinate in unfamiliar indoor environments; and (2) a visual locomotion policy that controls the robot's joints to avoid stepping on obstacles while following provided velocity commands. Both the policies are entirely "model-free", i.e. sensors-to-actions neural networks trained end-to-end. The two are trained independently in two entirely different simulators and then seamlessly co-deployed by feeding the velocity commands from the navigator to the locomotor, entirely "zero-shot" (without any co-training). While prior works have developed learning methods for visual navigation or visual locomotion, to the best of our knowledge, this is the first fully learned approach that leverages vision to accomplish both (1) intelligent navigation in new environments, and (2) intelligent visual locomotion that aims to traverse cluttered environments without disrupting obstacles. On the task of navigation to distant goals in unknown environments, ViNL using just egocentric vision significantly outperforms prior work on robust locomotion using privileged terrain maps (+32.8% success and -4.42 collisions per meter). Additionally, we ablate our locomotion policy to show that each aspect of our approach helps reduce obstacle collisions.

ViNL: Visual Navigation and Locomotion over obstacles

Navigation with AlienGo using egocentric depth vision in novel indoor environments

The RGB image (left) and top down map (right) is for visualization only and is not provided to the robot.

Locomotion with AlienGo using egocentric depth vision

Locomotion with AlienGo using privileged terrain maps

Here's a summary if you're short on time :)

Press Coverage


    title={{ViNL: Visual Navigation and Locomotion Over Obstacles}}, 
    author={Simar Kareer and Naoki Yokoyama and Dhruv Batra and Sehoon Ha and Joanne Truong}, 
    booktitle={International Conference on Robotics and Automation (ICRA)},

Related Projects

Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation
Joanne Truong, Max Rudolph, Naoki Yokoyama, Sonia Chernova, Dhruv Batra, Akshara Rai
CoRL 2022, Spotlight at Sim-to-Real Robot Learning: Locomotion and Beyond Workshop
paper / project page