Ma Meng, , P. Fallavollita, N. Navab Natural user interface for ambient objects International Symposium on Mixed and Augmented Reality (ISMAR), 2015 (bib)
	To help the computing device always understand the spacial relationship between the user’s gesture and the ambient objects, a methodology is proposed to find the user’s virtual eye center in the wearable camera coordinate system and then calculate accurately where a user is pointing at to perform the natural interaction. First, the wearable RGB-D sensor is affixed around the user forehead. A tool-free calibration is done by having the user move their fingers along their lines of sight from his eye center to the random selected targets. The fingertips are detected in the depth camera and then the interaction of these lines of sight is calculated. Then we present how to find where the user is pointing at in different scenarios with a depth map, a detected object and a controlled virtual element. To validate our methods, we perform a point-to-screen experiment. Results demonstrate that when a user is interacting with a display up to 1.5 meters away, our natural gesture interface has an average error of 2.1cm. In conclusion, the presented technique is a viable option for a reliable user interaction.
	This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each authors copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.