Interacting with holograms

There are basically five ways that you can interact with holograms and HoloLens. Using your Gaze, Gesture, and Voice and with Spatial Audio and Spatial Mapping. Spatial Mapping provides a detailed illustration of the real-world surfaces in the environment around HoloLens. This allows developers to understand the digitalized real environments and mix holograms into the world around you. Gaze is the most usual and common one, and we start the interaction with it. At any time, HoloLens would know what you are looking at using Gaze. Based on that, the device can take further decisions on the gesture and voice that should be targeted. Spatial Audio is the sound coming out from HoloLens and we use spatial audio to inflate the MR experience beyond the visual.

So, here are the five key elements for HoloLens interaction:

• Gaze: Indicates where the user is currently focusing
• Gesture: A user action performed on any of the gazed items
• Voice: Executing commands based on your Voice Commands
• Spatial Mapping: Understands your real environment and helps mix your holograms to the real surface 
• Spatial Audio: Spatial Audio to expand the MR experience beyond our visual senses

Before we go into more detail on all these interaction models, we can consider Gaze, Gesture, and Voice, to make it more logical, as input for HoloLens from users, whereas Spatial Mapping takes inputs from the environments to understand your real environment. On the other hand, Spatial Audio can be considered as an output interaction for HoloLens.

Let's explore them in detail.