Microsoft has a long tradition of spicing up relatively dull product announcements with compelling tech demos, and the Windows 10 announcement was no exception. The software giant used the opportunity to create a fair amount of buzz about the HoloLens, a futuristic headset that offers a glimpse into the future of Augmented Reality (AR). However, Microsoft also has a tradition of spectacular hardware flops, which peaked under the Ballmer regime. Remember the Kin phone? Neither do I.
The introduction of HoloLens probably won’t be such a flop for a number of reasons. First of all, the HoloLens still has a long way to go before it becomes a commercially viable device – it could be about a few quarters, or a couple of years. Secondly, the concept behind it is sound, and builds on a few promising emerging industry trends, such as wearable tech and Virtual Reality (VR) headsets. The HoloLens is trying to be somewhat different by bundling a lot of functionality into a single device, but in this Microsoft HoloLens review we will take a look at what’s already out there and what is in the works.
Since this is an engineering blog designed for VR professionals and other engineers, I won’t spend much time answering the question “What is HoloLens?” and explaining the difference between AR and VR. Augmented reality technology has a range of potential applications in various industries, but limited applications in entertainment. Virtual reality is more geared towards entertainment, although it has some professional applications as well.
Both technologies still have a lot of limitations, and numerous technical challenges have to be overcome in order to gain mass market appeal. This is a gradual process that will take years rather than months. The technology needed to create such products without breaking the bank is simply not ready, but it’s slowly getting there.
Let’s take a look at what’s out there and what’s missing.
Hardware Limitations – Google Glass vs. Oculus Rift vs. Microsoft HoloLens
Google Glass was announced in early 2012 and started shipping a year later at a cost of $1,500. The high price tag meant it was reserved for a very small niche – early adopters described as “explorers” by Google’s PR and marketing machine. The device offered limited AR functionality, and contained a small prism projector with a resolution of 640×360 pixels, powered by an outdated processor.
While it managed to captivate the public for a while, Google Glass can hardly be described as a success. App developers who were keen to jump on the bandwagon started losing interest, along with “explorers” who appeared to get over the fad in a matter of months. The latest rumours point to a new version of Google Glass with Intel silicon inside, so it might be a bit too early for an obituary. Either way, Google Glass was not a big success no matter how you look at it.
Oculus Rift is perhaps the most talked about VR system at the moment, but unlike Google Glass, it has yet to launch. Oculus VR has been working on the device for years, and in the process the company went through two generations of development kits. The consumer version is expected to launch sometime in 2015, with a revised spec. In March 2014, Oculus VR was bought by Facebook for more than $2bn in cash and Facebook stock.
Samsung’s Gear VR offers a different approach, as it utilises the Galaxy Note 4 phablet in lieu of a built-in screen, but it relies on some technology developed by Oculus. I find the modular concept interesting, as a similar approach could be employed with a range of mobile devices from different vendors that would allow users to effectively upgrade hardware every time they got a new phone. Qualcomm’s Vuforia platform boasts some promising features for mobile devices and potential AR/VR applications.
So, what’s missing? The simple answer might be processing power, but it’s a bit more complicated than that.
The problem with both concepts is that they are still ahead of their time, and the technology still needs to catch up. Microsoft’s HoloLens is bound to suffer from the same teething problems, but Microsoft’s concept is somewhat different, and therefore stands a chance of overcoming at least some of these issues.
Google Glass was designed as a lightweight wearable, which resulted in a number of compromises. The device featured a single display on a thick prism in front of the user’s right eye. The resolution was very limited given the field of view (FOV). For example, smartwatch displays tend to feature similar vertical resolutions for a device that takes up just a couple of degrees of the user’s field of view. Google Glass was based on an antiquated System on Chip (SoC) and had limited battery life.
Designing mobile devices is not easy, and always involves a number of trade-offs. Higher resolution displays require more GPU power, necessitating the use of bigger SoCs with more powerful GPUs working at a higher load, which then requires a larger battery and so on. It’s a fine balancing act, and an AR headset is simply too small to accommodate a large battery like those used in high-resolution tablets.
At first glance, Oculus Rift does not appear to suffer from similar shortcomings on the hardware front, since it does not have compromises for the sake of battery life and portability. It does not rely on an integrated SoC, and a 1080p display sounds desirable; but, in reality it’s not nearly enough for photorealism. The device has a very large FOV, and pixel density is still insufficient.
To overcome this problem, VR devices would have to use higher resolution 4K/UHD displays, or even 8K displays at some point in the future. The technology is almost there, but it does not come cheap, and is anything but portable.
If you want to run the latest AAA games on a 4K display with the highest possible detail settings, you need two high-end discrete graphics cards. For example, Nvidia and AMD cards based on flagship Maxwell and Hawaii generation GPUs. To eliminate frame tearing (using technologies similar to Nvidia’s G-Sync or AMD’s FreeSync) you need a bit more power, and to do proper 3D for both eyes you need even more GPU power.
The bottom line is: to power a 4K VR device using currently available technology, you would need at least two GPUs with a total of 12-14bn transistors in 28nm, consuming 350W to 500W of power, not counting the CPU and rest of the system. This is a conservative estimate, based on currently available GPU and CPUs – and let’s not even discuss the idea of powering two 4K screens, one per eye.
Nvidia’s latest mobile SoC, the Tegra K1 64-bit used in the Google Nexus 9, features 192 CUDA cores based on Kepler architecture, not the more efficient Maxwell. The company’s current flagship discrete graphics cards sport 2048 Maxwell CUDA cores running at higher clocks than Kepler cores in mobile Tegra SoCs.
Portable VR devices with photorealistic graphics are clearly unavailable for years to come, and even wired devices like Oculus Rift have a long way to go. The overall platform cost is another concern. Gaming PCs capable of pumping out playable frame rates at 1080p are relatively cheap, since mainstream GPUs are fast enough to do the job. But at 2160p you need four times the GPU muscle, backed by more memory and a faster CPU.
There is another way of tackling this problem, and I will go over it later.
So What Did Microsoft Get Right?
Remember Facebook’s Oculus Rift deal I mentioned earlier? Just a few days after it was announced, it emerged that Microsoft bought intellectual property (IP) assets related to augmented reality and wearable computers from the Osterhout Design Group (ODG). Some of the patents covered “see-through near-eye display glasses” with a partially transmitting optical element.
In other words, Microsoft bought the IP needed to create HoloLens; and the deal reportedly covered dozens of ODG patents, including a few dozen more patent applications in progress. Meanwhile, Oculus VR is said to have just a single patent, which vaguely describes “a virtual reality headset”.
Microsoft appears to be trying to get the best of both worlds – a wide FOV usually associated with VR devices, and a transparent display surface suitable for AR applications. The approach should allow HoloLens to utilize a lot less processing power than VR devices, while at the same time offering more functionality thanks to the wide FOV. Instead of trying to render photorealistic content, HoloLens could get away with a slightly lower resolution and image quality due to the limited opacity of displayed content. There is no need to create an illusion of reality, so there is a lot less hardware overhead involved. A lot of off-the-shelf technology could allow the HoloLens to reduce or eliminate aliasing and generate good looking composites, since the backdrop is already there.
This fact limits HoloLens’ appeal in the entertainment niche, as opposed to true VR headsets; but, it opens up a number of possibilities in other industries, ranging from engineering and healthcare to architecture and defense. HoloLens could be used to assist healthcare professionals, engineers, operators of industrial machinery, soldiers, and law enforcement.
However, HoloLens still has applications in the consumer space. Microsoft’s Phil Spencer said HoloLens needs to be a successful standalone product, adding that the company is already looking into ways of using it in unison with PCs and Xbox One consoles. The device could serve as a heads up display (HUD) for gamers, or even for fitness buffs in gyms.
Via: ( toptal.com )
