Mixed Reality/Spatial Computing
MR, essentially a holographic computer, removes the boundaries between real and virtual interaction via occlusion (blocking) and is a combination of AR and VR. The physical world becomes one’s computing interface! The most popular MR headset is the Microsoft HoloLens, which has a pair of translucent screens that allows users to see the real world while interacting with holograms within their world. The visor, resembling a Microsoft Kinect, has cameras that scan one’s surroundings and collects data while a user wears it.
The HoloLens has all the computing power within the headset, so it is not tethered to a computer or other peripheral device; it is designed for people, such as factory workers or soldiers, who work primarily with their hands and find it cumbersome to integrate a computer while on the job. The first HoloLens created an electronic mesh that covered the real world, allowing digital objects to be placed in it. With the new HoloLens 2, the device has a better understanding of a user’s space thanks to cloud processing and AI. The eye tracking allows users to simply look at where they would like a hologram to be, and it will move there; plus, the cameras will scan users’ retinas to securely log them in to the device.
Like MR, spatial computing is a form of immersive technology that uses computer vision and AI to integrate visual and intelligent content into the user’s real world. The content “knows” it is in physical space with the user, therefore the interaction is more engaging. MR and spatial computing do pretty much the same thing, but the competing companies use different terms: Magic Leap (magicleap.com) calls its technology spatial computing; Microsoft calls it MR. Magic Leap includes Lightwear (a headset), which has tinted lenses acting as small projectors with eye- and hand-tracking capabilities. Lightwear is connected via one cable to its Lightpack, which does all the processing and battery work. One controller is included, but not all applications require it. Magic Leap and the HoloLens have visualized a way for computing to come outside the physical confines of a computer and to allow a room to become their interface. Everything now becomes the interface.
Again, XR has the potential to bring people, places, and experiences closer together; as a result, one of Magic Leap’s primary goals is to get people to start looking at each other again, instead of staring at individual screens and being more isolated. St. Petersburg College science students recently prepared for an Anatomy and Physiology (A&P) exam using Magic Leap and HoloLens headsets. The students preferred this method of study because many of the 3D models they use in class do not allow them to see underneath some of the body’s systems/parts. With immersive technology, they were able to easily remove parts in the spatial computing/MR application so they could see how the entire system is connected. These students were also able to use co-location to work with the same A&P hologram in a shared room with different headsets while being able to interact in a face-to-face setting. Adding assessments and metrics to immersive technologies are helping to improve learning too. Teaching 3D concepts on traditional 2D boards/displays/textbooks works, but it will be more meaningful when a student starts to learn 3D concepts actually in 3D.
The Future?
Immersive technologies are starting to gain traction. Once the hardware becomes ubiquitous and does not require bulky and expensive headset computers, then society will start to see the growth of XR, making everything in our world interactive. Festivals like SXSW and Cannes are now including XR tracks that cover VR/AR/MR and highlight immersive technologies, including virtual cinema. XR is becoming more mainstream, so society is going to see more meaningful connections between entertainment and XR technologies, which will allow stories to be told in more immersive ways. Gaming will be indistinguishable from reality in the future, which terrifies some people, but excites others.
XR is revolutionizing education. Perhaps “blended learning” will truly become blended once learning environments are better designed for extending a student’s reality so they can learn in multiple modalities. Any 3D room now can be uploaded to an XR space to give students a different classroom setting (e.g., a surgery room or architecture design) to make learning more meaningful and to enrich a variety of topics. Learning will become more transformational as a result.
All of this has serious potential for distance learners and project-based systems, whereby users can become location-agnostic and work together no matter where anyone is physically located. There will be no time and space limits, as the physical world continues to blend, almost seamlessly, with the digital. Ultimately, it is important to know what is coming so educators can help students prepare for the workforce of the future, be ready to adapt to changes, and ultimately succeed.
