Virtual Reality for Robots

Markku Suomalainen, Alexandra Q. Nilles, Steven M. LaValle

Published 2019-09-16Version 1

This paper applies the principles of Virtual Reality (VR) to robots, rather than living organisms. A simulator, of either physical states or information states, renders outputs to custom displays that fool the robot's sensors. This enables the design of targeted experiences for the robot that are more realistic than pure simulation, yet more feasible and controllable than real-world experiences. Potential applications include testing procedures that are better than simulation, reverse engineering of unknown robots, the study of spoofing attacks and anti-spoofing techniques, and sample generation for machine learning. A general mathematical framework is presented, along with a simple experiment, detailed examples, and discussion of the implications.