Driving Virtual Reality from Linux

Presented by Keith Packard
Friday 11:40 a.m.–12:25 p.m. in Great Hall CB01.05.009
Target audience: Developer


Head-Mounted Displays (HMD) are the most common device for presenting virtual reality to the user. They project independent images to each eye to construct a stereo view and track the position and orientation of the head using both inertial measurement and external beacons. All of the elements in the system introduce varying amounts of delay between the motion of the head and the perception of the image by the user. Computing the projected images thus requires predicting where the user will be facing when the image is finally emitted from the display. As the user is strapped into the HMD and their visual field is limited to that provided by the HMD, timing errors in the VR presentation can cause disorientation leading to nausea or other effects. This leads to strong latency bounds through the entire application pipeline. A traditional window system, running multiple applications, cannot easily provide the guarantees needed by a VR system. Competing applications may consume window system resources and cause jitter in the VR presentation. Separating the VR application from the window system can allow it to operate without interference from other applications and improve performance. Started at LCA last year in Hobart, the new DRM Leasing mechanism allows applications to take over a set of display resources from the window system and drive them directly. These changes involved modifications to the Linux kernel, X window system and Mesa Vulkan implementation. This new system brings proposed Vulkan extensions for direct display operation to the DRM environment. The presentation will discuss the design and implementation of the system, along with highlights of the development process and a live demonstration of the resulting system. All of this free software was developed under contract with Valve, who are using it to provide support for the HTC VIVE HMD.

Presented by

Keith Packard

Keith Packard has been developing free software since 1986, working on the X Window System, Linux, and rocketry electronics. He is currently a Distinguished Linux Technologist at Hewlett Packard Enterprise working as the Chief Architect for Linux on The Machine. Keith received a Usenix Lifetime Achievement award in 1999, an O'Reilly Open Source award in 2011, sits on the X.org foundation board and is a member of the Debian Technical Committee. He has spoken at numerous free software events around the world, including Linux Con, the Plumber's Conference, Linux Conf Australia, FOSDEM, FISL, Guadec, Akademy, OSCON and many others.