Physics-simulated runtime interaction has the potential of offering one of the most realistic and generalizable solutions for grasping virtual objects in immersive Virtual Reality (VR). However, existing research is limited to a specific type of, camera-based, hardware for hand tracking. Furthermore, even though there is substantial research in robotics for grasping using Reinforcement Learning (RL), this approach hasn’t yet been applied in VR. We propose a series of experiments to evaluate physics-driven grasping for different tracking methods based on a platform and device-agnostic library. For the cases of hardware with limited finger control, such as controller tracking, we also propose the training of an RL agent that will be responsible for the finger motions. The evaluations will take into account the users’ performance and preference, but also the Sense of Embodiment, an aspect that hasn’t been explored in this context of physics-based VR interaction.
Javascript must be enabled to continue!
Physics-based Grasping in VR Using Different Tracking Methods and Reinforcement Learning: the Effects on the Sense of Embodiment
CHI Greece
Connecting the Community