We experienced the future firsthand, and we lived to tell you about it
By Osmond Chia
Of all digital realities – VR, AR and MR – Mixed Reality is undoubtedly the hardest to get your hands on.
A technological baby, MR is being pioneered by very few companies - the one we know best is Microsoft. To make things more difficult, Microsoft’s only MR solution, the Hololens, is only sold to developers with valid justification – not to mention the tedious process to get it on to our shores.
While it may seem distant, its technology is going to change the way we live in so many ways – from the way we shop, game, study - to the way we see the world – and we got a glimpse of just how useful this piece of tech is.
We were given a very thorough demo of the Hololens courtesy of Helloholo, a local start-up that provides developers and interested parties access to the Hololens. They’ll teach you how to use it, how to develop apps with it, and provide bespoke MR solutions to issues you may be facing. They also provide rental and assist with tricky Hololens acquisitions.
Yeo Jun Jie, the Marketing Director of Helloholo brought the Hololens to us. Right away you get the sense of how well-built the headset is. It’s lightweight, premium-looking and didn’t make you look ridiculous like a HTC Vive. Microsoft are indeed serious about bringing MR to the world.
SKYPE ON THE HOLOLENS
The first app Yeo loaded up was something familiar to us all, but given a whole new dimension of usefulness, and that app is Skype. He had to scan the room, so that the Hololens knew its dimensions and space, then he handed us a laptop that showed us what he could see through the lens.
The difference here is digital objects can be brought into what we see. The Hololens knows that that’s a wall, and using a pen tool, Jun Jie was able to draw symbols and markings which stay fixed on the wall and don’t shift even though we look away. The tracking on this thing is phenomenal – there was no lag as if all these digital objects had some sense of physicality.
Microsoft doesn’t expect other parties to make a S$5000 investment either; the other party doesn’t need a headset to enjoy the benefits of video calls and other applications as they are able to stream what the user sees. Imagine this in education and how helpful that would be! Being able to teach students from afar through the Hololens, and creating digital objects and interacting with them as with a usual class. So many industries and individuals can be benefitted by this.
ENTERTAINMENT AND GAMES
We moved on to a bit of fun and games – an alien invasion game called RoboRaid. As with most apps, we had to scan our perimeters, and before we knew it, parts of the wall began crumbling away and little alien squids began crawling through to kill us.
Admittedly, it’s not as immersive as VR, (which would literally being taken to a whole new digitally rendered realm) but MR adds to our own world to let us interact with virtual elements in a way never seen before. It retains your awareness of where you actually are, and adds another dimension to it. If it were more natural and integrated, it could be some kind of sixth sense.
The game that seriously impressed us was a crime scene investigation simulator, Fragments, that Yeo set up for the team.
The scanning process was far more thorough; the app required us to scan every nook and cranny of the room, under real objects and behind chairs – it’s quite commendable how the machine picked it all up.
Our room was then transformed into a crime scene, with digital objects rendered on top of our reality. We had to familiarize ourselves with this new set up, as there were plenty of items added, like a cracked wall, frames, a corpse on the floor and smashed objects. These could all be interacted with, by peeking through cracks, opening boxes and other contextual actions as you move around your actual room.
While it wasn’t the most fun game ever made, it certainly unveiled the potential of the device as a gaming device.
The Hololens isn't quite ready for the market yet.
We say it lacks a natural feel party because of its limited controls. There are two moves: to access the menu, you need to stick your fist up in front of its sensors and open it up. You may then select things by pinching icons, strictly with your other fingers tucked in.
This applies to “shooting” in video games as well – it can’t quite recognise objects as substitutes for weapons and thus we’re left with this awkward way to shoot enemies that simply doesn’t feel very immersive. Onlookers probably wouldn’t be able to tell we were in a game.
But perhaps the Hololens’ greatest setback today is its poor field of view – look straight and the screen only fills about 80% of your view. Digital objects may be cut off, and you have to look around to fully capture the object within the screen. As a result, you get no situational awareness and it’s tiring to keep moving your neck around – it doesn’t track eyeball movement.
Having said that, this is most problematic for games like RoboRaid, where peripheral vision is essential to locate aliens creeping up from beside you. Alas, where Hololens matters most – developers – the limited field of view won’t be a deal breaker. You can still render digital constructs and share them between users for projects, demonstrations and digital tours, but its an setback we hope to see future models addressing.
There’s a huge future for MR, especially if developers are willing to make that step to incorporate this fascinating tech into their work. “Hololens represents limited resources made unlimited”, Yeo told us, and began to explain how real companies can really benefit from it.
The final app we tried was probably the simplest of the lot, but with real practical benefits companies can use right away. Room scanned, Yeo brought in virtual objects like tables and rendered cats that we could place around the room. The device recognised real world items like a couch, and digital objects sat realistically above it, without sinking through or being awkwardly positioned.
Scale this up for developers; imagine IKEA incorporating MR into their catalogs. You could wear a Hololens, and visualise how your dream setup would be with digitally rendered IKEA furniture, planning your future purchases whether for home or office.
Helloholo are spearheading MR on our shores, and is already working with shipyards and other major corporations to create virtual renditions of spaces they have, as they all recognise the efficiency and flexibility of this piece of tech. Yeo hopes to see more developers jump on board to “promote more innovation in Singapore and be more creative”, with the help of MR.
And given what we’ve seen today, we are excited for what MR can bring to the table, especially in years to com