Dipping elementary OS' toes into Virtual Reality
Read how I managed to render elementary OS Desktop windows in Virtual Reality using xrdesktop.
elementary OS is rendering its windows in Virtual Reality
Thanks to the plugin I wrote to support xrdesktop in elementary's window manager "Gala", we are now able to mirror desktop windows to Virtual or Augmented Reality. Although the support is still in a very early stage, its already possible to interact with the windows using the Virtual Reality Controllers and doing some re-positioning stuff. On the other hand, mouse or keyboard interaction is not possible as of now and the current state has been tested in SteamVR using a HTC Vive only.
Mirroring the elementary OS desktop to Virtual Reality needs three pieces to work together:
- 2.The possibility to adjust some settings on how the mirroring should work
wingpanel-indicator-xrdesktop to enable/disable desktop mirroring
In addition, the backend which does all the mirroring work is developed as a Gala plugin. Fortunately, there is a reference implementation for the GNOME desktop environment available. So the main work was to translate the existing C code into Vala. To do so several VAPI's for the xrdesktop libraries were needed as well as doing some OpenGL stuff in Vala.
Mouse and keyboard interaction is still missing. So this would be a natural next step to implement. Unfortunately, Gala uses
libmutter-6to do so - wherelse xrdesktop only supports
libmutter-5in libinputsynth. Another potential solution for this would be to add Wayland support. However, this may lead to a bunch of other problems on elementary.
There is no way a regular user can adjust xrdesktop's settings in elementary OS yet. I think the best way to do so, would be to develop a Switchboard Plug with the UI needed to tinker with the settings.
Monado is the reference runtime for the OpenXR specification. At the time I started development, SteamVR provided the better end user experience and was much more stable. Therefore I chose SteamVR over Monado to jump start initial work. However, in the long rung we really want to support open software - therefore getting things working on Monado is definitely something we need to work on.