Reality Plane

Posted by
Igal Nassima

Date: 2017-06-18
Updated: 2017-06-29


Reality Plane is a haptic tablet using Hololens, Vuforia and BLE bridging the gap between the virtual and reality, allowing new interactions with a familiar interface.   "The physical plane, in emanationist metaphysics taught in Neoplatonism, Hermeticism, Hinduism, and Theosophy, refers to the visible reality of space and time, energy and matter" As we subscribe ourselves to worlds created or altered in Virtual or Augmented Reality, we are leaving and redefining our current plane of reality. In this mode of transition, it will become important to explore how our existing bodies simultaneously interact with physical and virtual environments. The Reality Plane begins to explore our interaction in Mixed Reality through added sensory cues in a familiar input device, a tablet.

Hardware Setup

To build the haptic response component, we used the following hardware

Tablet Design and Vuforia Integration

We did some experiments to balance the weight and improve the resonance of the haptic feedback through the tablet. The marker size was especially challenging because of the small viewport restriction of Hololens and the desired tracking quality we wanted to achieve.

How we used the tablet

Reality Plane presents a completely blank surface onto which information can be projected in mixed or virtual reality. It has the potential to support natural interactions (touching, sliding, pinching, et cetera) and provides haptic feedback for interactions. Developers can create their own experiences while users experience a more natural mode of interactions (as compared with air gestures or game controllers) The orientation of the Reality Plane allows for different types of information to be displayed. In the demo game we created, when orientated horizontally a game map appears and displays the location of crystals to collect. Vertical orientation allows you to engage with the crystals that are floating in your environment. When you catch the crystals you simultaneously feel a buzz and hear the sound that each crystal emits.

Challenges and Source Code

Source code coming soon!

  • We built a BLE library for unity from scratch, which took a long time to get working with consistency--it's already a great start but there's more to do
  • We had to design haptic tablet in multiple iterations and test it to create a balanced physical UX. The viewport scale of
  • Hololens creates a challenge in terms of designing the interaction between tablet, navigation and gestures that change the environment at any time.
  • Vuforia performance create a delay in the content movement, we designed in the interface to minimize its use.