Head-Mounted Displays (HMD) are the most common device for presenting
virtual reality to the user. They project independent images to each
eye to construct a stereo view and track the position and orientation
of the head using both inertial measurement and external beacons.
All of the elements in the system introduce varying amounts of delay
between the motion of the head and the perception of the image by the
user. Computing the projected images thus requires predicting where
the user will be facing when the image is finally emitted from the
As the user is strapped into the HMD and their visual field is limited
to that provided by the HMD, timing errors in the VR presentation can
cause disorientation leading to nausea or other effects. This leads to
strong latency bounds through the entire application pipeline.
A traditional window system, running multiple applications, cannot
easily provide the guarantees needed by a VR system. Competing
applications may consume window system resources and cause jitter in
the VR presentation. Separating the VR application from the window
system can allow it to operate without interference from other
applications and improve performance.
Started at LCA last year in Hobart, the new DRM Leasing mechanism
allows applications to take over a set of display resources from the
window system and drive them directly. These changes involved
modifications to the Linux kernel, X window system and Mesa Vulkan
implementation. This new system brings proposed Vulkan extensions for
direct display operation to the DRM environment.
The presentation will discuss the design and implementation of the
system, along with highlights of the development process and a
live demonstration of the resulting system.
All of this free software was developed under contract with Valve, who
are using it to provide support for the HTC VIVE HMD.
Keith Packard has been developing free software since 1986, working on
the X Window System, Linux, and rocketry electronics. He is currently
a Distinguished Linux Technologist at Hewlett Packard Enterprise
working as the Chief Architect for Linux on The Machine. Keith
received a Usenix Lifetime Achievement award in 1999, an O'Reilly Open
Source award in 2011, sits on the X.org foundation board and is a
member of the Debian Technical Committee. He has spoken at numerous
free software events around the world, including Linux Con, the
Plumber's Conference, Linux Conf Australia, FOSDEM, FISL, Guadec,
Akademy, OSCON and many others.