Controlling robots and interacting with VR environments using a Leap Motion controller is nothing new. As this video shows, it is possible to rely on it to control a Pepper robot, teleoperating its arm to move objects.
The robot can pick up a bottle and pour drinks. Its camera is used to give the operator a pilot’s view. Pretty cool, don’t you think?