Virtual Reality (VR) video is being touted as one of the next big innovations and revenue opportunities for content and service providers, enabling them to deliver more immersive experiences. Market research estimates that by 2020, VR video services will capture $1 billion in revenue, with live sporting events representing more than one-third of that amount.
Yet, from a technical perspective VR has not been practicable until recently because it required very high bandwidth (at least 15 Mbps to 20 Mbps) while providing poor video quality. However, a state-of-the-art technology called VR tiling resolves those issues, reducing bandwidth requirements by an order of magnitude and enabling fully immersive UHD VR to be enjoyed by viewers on legacy head-mounted displays (HMDs).
Bandwidth Challenges Explained
The largest hurdle preventing mass adoption of VR video services is video quality. High-quality VR video requires about 20 Mbps for transmission, and even then the video resolution is below HD. Part of the problem is that displays, such as HMDs, are relatively low resolution. The other problem is that users are only watching a small percentage of the full panorama. Yet, recent developments in technology are demonstrating that the perceived quality can be significantly enhanced by maximizing available bandwidth efficiency.
VR Tiling Technology
As the industry works on ways to improve the quality of experience (QoE) on HMDs, one solution that has emerged is a VR tiling technology capable of transmitting a complete UHD field of view. VR tiling technology works with all relevant devices, including the Oculus Rift, HTC’s Vive, and Samsung’s Gear VR.
To save bandwidth, tiling technology only transmits the pixels that are watched. The equivalent in quality of an UHD legacy VR transmission can then be delivered in HD, consuming only ~5 Mbps using HEVC compression. What’s more, the full UHD-native experience can be transmitted at around 20 Mbps, which is completely achievable in advanced networks such as DOCSIS 3.x, fiber, and 4G+/5G.
For tiling technology to work, the panoramic video needs to be encoded to where it’s divided into tiles after the compression step. This can be done easiest with HEVC encoders, which offer native support for tiles. HEVC encoding must be done with tiles of the desired size, and the encoder should enforce a few restrictions, notably that tiles are “motion-constrained.” Typically, there will be over 100 such tiles. These tiles are then extracted from the bitstream, packaged and stored on a standard CDN, in combination with special metadata that aids reconstruction of a legal bitstream at the decoding side. The client then requests all the tiles that are fully or partially in the viewport of the HMD, combines them into a single, standards-compliant bitstream, decodes that bitstream, and then arranges the decoded tiles (i.e., the decoded pixels) for rendering on the device.
Since it takes approximately 20 to 40 ms to retrieve tiles from the network under good network conditions, precautions are needed to prevent black areas or picture freezes appearing in the HMD when users turn their head. The ideal solution is to use an extra layer at a much lower resolution of the entire panorama. When a user moves the attention to a different part of the panorama, this “fallback” layer ensures that there are no black holes for the 20 ms to 40 ms it takes for the high-resolution tiles to arrive.
Advantages of Tiled Streaming
There are several key benefits of VR tiling technology that make it attractive for commercial services. First and foremost, it allows streaming of VR content at a very high quality, using realistic and affordable bitrates. With this method and optimized video encoding, bitrates can be reduced by up to 80 percent without quality loss. Moreover, significant improvements can be made to the quality at bitrates similar to those used by a system that streams the entire sphere.
Since the low-resolution base layer is always present, users will not experience “black holes” or content freezes. Rather, a low-quality version of the content will always be available. In addition, since tiles are retrieved individually, when bandwidth degrades, the center of the viewport might still display high-quality tiles while the corners of the viewport will degrade when showing the fallback layer. The use of adaptive bitrate streaming techniques further ensures that users can experience optimal quality given the available bitrate at any point in time, e.g., on a residential internet connection or even on a cellular 3G or 4G network.
VR tiling technology supports both VOD and live content, a stark contrast with other approaches, and the full panorama only has to be encoded once compared with other approaches that need to encode the panorama up to 30 times (once for each viewport). This makes VR tiling an inexpensive and easy to deploy solution. Even better, VR tiling works with HMDs as well as flat screens such as tablets, phones, and even set-top boxes.
From a scalability perspective, VR tiling makes the distribution of virtual reality video services to a massive number of simultaneous users possible by relying on the same standard HTTP streaming concepts that are used for virtually all commercially deployed streaming services. This means no changes to standard servers or CDNs are necessary, either over the top or on a managed network.
Real-World Success Stories
VR tiling technology is being tested and is used effectively in the real world. At the 2017 NAB Show, Harmonic teamed up with the Blue Man Group to showcase 360-degree VR video produced in 8K using Tiledmedia’s Tiled VR Streaming technology integrated with the Harmonic PURE Compression Engine, Viaccess-Orca’s Connected Sentinel Player for secure playback, and Samsung Gear VR headsets.
There are also instances where operators are successfully deploying legacy VR systems. A great example is the recent use of Harmonic’s Electra VS video system by PCCW Global for VR live delivery during the 2017 Cathay Pacific/HSBC Hong Kong Sevens rugby tournament. Utilizing Electra VS for compression, PCCW Global was able to distribute UHD VR content to consumers on a variety of devices, including PCs, smartphones, and Samsung Gear VR HMDs.
VR Video Is in Our Future
Today, there’s a large focus on increasing viewer engagement and monetization, as content and service providers face increased competition. Beyond providing a level of immersiveness that is not achievable with standard 180-degree views, 360-degree VR video opens up untapped revenue opportunities, especially for live sports applications.
However, the quality of VR video has to be superb, and bandwidth efficiency is a concern. VR tiling technology reduces the bandwidth and improves quality for VR streaming services, while providing compatibility with existing coding systems and CDNs, massive scalability, and suitability for use with a wide range of device types, from HMDs to smartphones. Using VR tiling technology — with appropriate video coding optimization — content providers can reduce the bitrate by up to 80 percent without quality loss for live and on-demand VR video.
Thierry Fautier is Vice President of Video Strategy at Harmonic.