2024-06-29 Using Monado OpenXR runtime on OpenSUSE with the Valve Index

A couple of months ago, I decided to try out this whole VR thing, and maybe get some cardio in by playing Beat Saber. I purchased the Valve Index VR headset and the SteamVR 2.0 Base Stations. It required some fiddling to get it to work on the OS I use, OpenSUSE Tumbleweed, but eventually I got it to work. That was with Steam's built-in VR runtime SteamVR, so I then started looking into using the open-source Monado OpenXR Runtime instead. This post is a record of all the issues I ran into and the current state of things. If you are an OpenSUSE Tumbleweed user looking to play Steam games on a Valve Index, the information in this post may help you get started faster.

SteamVR in a sandbox?

I don't like running closed-source software unsandboxed because it can't be trusted. Steam is especially problematic because a) it has a history of deleting a person's home directory due to a bug, b) games are an easy way for malicious software to be distributed (gamers are often willing to click "Allow" as many times as necessary if it lets them play their game), and c) it ends up polluting my OS install with a bunch of 32-bit library dependencies that nothing else needs.

So for the longest time I did not use OpenSUSE's steam package. Instead I run a podman container of a "non-oss" base image (Debian 12 + Debian's steam package + a bunch of other libraries for audio etc) with its /home/arnavion mounted from a separate ~/non-oss-root/steam directory on the host, and some other host files mounted at the appropriate places. [1] I also use this container to run Discord (the web version doesn't support calls in Firefox) and used to use it for MS Teams back when it still had an Electron version.

However I could not get SteamVR to work in this container, because it did not seem to detect the headset despite all my attempts to mount the headset's USB dev nodes into the container. SteamVR has very baroque errors which just show a number and no details, which frustrate users even on "normal" setups. Even in a normal setup there are parts of SteamVR that are broken, eg it tries to launch a webview that flashes a white window and then immediately segfaults. Since it's closed-source, I couldn't just look at its code to see what the problem was, and SteamVR is complicated enough that it made strace debugging too noisy to be useful.

An alternative "standard" for such sandboxing is Flatpak, and Steam does have one, however it was also unable to detect the headset.

Ultimately I had to relent and install Steam on my host OS via my distro's Steam package. At least I was able to get Beat Saber working fine. This did however strengthen my resolve to replace as much of the closed-source software with open-source alternatives as I could, hence why I discovered Monado and started trying to make it work.

Monado

OpenGL is a standard that applications can invoke to draw triangles, and the implementation is provided by something specific to the environment, like the graphics driver on Windows or the Mesa library on Linux (which then has graphics-driver-specific backends). In the same way, OpenXR is a standard that applications can use to detect where the player's headset and controllers are and draw the appropriate binocular 3D triangles, and SteamVR is an OpenXR runtime that does so using the headset's DRM device and position tracking capabilities. Monado is another OpenXR runtime that is open-source.

OpenSUSE did have a package for Monado, though it was only in the hardware:xr OBS repository and not in the Factory OSS repo. In any case, this package had not been compiling succesfully for almost a year and nobody had cared to figure out why and fix it. The version of Monado itself was 3 years old (v21.0.0 from 2021-01-28) and upstream had had numerous fixes since then but not yet cut a release. I made my own OBS package of the git tip-of-tree with an updated RPM specfile to compile it. I wouldn't have been able to contribute this back to the hardware:xr repository in this state, but fortunately Monado did end up releasing v24.0.0 on 2024-06-07 and I upstreamed that to the hardware:xr repository. If you are an OpenSUSE user who wants to use Monado, this repository's package should work for you. (But if you want to use it for playing Steam games, keep reading.)

Before I tried this with Steam games, I figured I would test it first with a simple host application. Just like OpenGL has glgears, an application that just shows a window with three spinning gears rendered via OpenGL, Monado has its own xrgears that shows a 3D skybox, the same three spinning gears, and a bunch of floating pictures. OpenSUSE does not have this packaged anywhere so I've packaged it in my own OBS repository for now. Upstream hasn't had a new release in three years, and it seems to have had fixes since then that are necessary, so I'll wait to sr it to hardware:xr until upstream makes a new release.

Direct mode

There is a nuance to how Monado uses the headset, which depends on how the compositor exposes it. One way is that the compositor exposes the headset as another monitor-like output, with its own position and resolution like a monitor would, except that its width is twice of what one eye can see because it expects the images for both eyes to be rendered side-by-side. Another way is that the compositor detects it as a "non-monitor" output and exposes it as such. The latter is preferrable and is what Monado calls "direct mode". This depends on the kernel identifying the VR headset device as a non-monitor via a list of hard-coded EDIDs, and then exposing this information to userspace. X11 or Wayland compositors can then make use of this information to expose the headset to clients like Monado accordingly.

I use the Sway Wayland compositor, which does support the drm-lease-v1 Wayland protocol that makes it possible for clients like Monado to bind the output in direct mode, so that part should've worked fine. However running Monado would keep rendering a gray window on my regular monitors and nothing on my headset. Initially I didn't even know this was a problem; I figured the window on my monitors was supposed to be a preview of what it was also rendering to the headset, so it was just the headset rendering that was broken. However through a bunch of debugging of monado-service in gdb, I eventually figured out that this was actually a consequence of Monado not having the Wayland direct backend compiled-in at all so it was falling back to the non-direct Wayland backend, which expects you to move that gray window to the headset "monitor" yourself. This was because Monado's build automatically disabled the Wayland direct backend if the required build-time dependencies for it were not present, and I had not noticed the build output that indicated this backend was disabled (because I didn't know to look for it). This was also compounded by the fact that the documentation of direct mode for Wayland on Monado's website was outdated at the time and implied that the Wayland compositor side of the feature was still waiting on the protocol to be developed. I updated the Monado package build to install the Wayland direct backend's dependencies and enforce that the build fails if the dependencies change in the future and cause the backend to become disabled again. (This was before I submitted the v24.0.0 package to hardware:xr, so the hardware:xr package already has the fix.) I also submitted an MR to fix the Monado doc so that is also up-to-date now.

Since I have SteamVR base stations, the appropriate Monado backend for base stations (aka lighthouses) is the libsurvive backend. I followed this documentation for setting it up. There is also more general detail related to Monado binaries that is useful for troubleshooting, and just understanding how it all fits together, here.

The setup doc talks about two ways to configure libsurvive for tracking the base stations - either doing it from scratch or importing it from SteamVR's Room Setup. I tried both of these approaches but the tracking seemed to be very bad. Holding the headset still would frequently send the rendered view tumbling at a high speed, and even if it didn't do that it would frequently twitch in a random direction by many degrees, lag behind any movement of the headset, and various other problems. I found discussions with other people complaining about it too.

Fortunately Monado v24 has a new driver for tracking the SteamVR base stations that actually just uses the SteamVR driver itself, which has much better tracking. The Monado build does not enable this driver by default unless the build environment has a ~/.steam directory, which would obviously not be the case for a distro package builder, so I had to configure the build to explicitly enable the driver. This build change is also in the hardware:xr repository's monado package now. Note that the LH_DRIVER=steamvr env var mentioned in the Reddit post is outdated and will not work with Monado v24; the correct env var is STEAMVR_LH_ENABLE=true.

Since libsurvive is not being used any more, its env vars can be removed. Furthermore, XRT_COMPOSITOR_SCALE_PERCENTAGE already defaults to 140 so it does not need to be set, so the final set of env vars for monado-service is:

STEAMVR_LH_ENABLE=true
XRT_COMPOSITOR_COMPUTE=1

(XRT_COMPOSITOR_COMPUTE=1 will also become unnecessary once it becomes the default.)

The setup doc doesn't say it, but OpenSUSE users running Monado via its systemd user service can just add a dropin to set these automatically, ie by creating ~/.config/systemd/user/monado.service.d/override.conf with the content:

[Service]
Environment=STEAMVR_LH_ENABLE=true
Environment=XRT_COMPOSITOR_COMPUTE=1

I now had xrgears working fine. The next step was to make it work with Steam games.

Monado for Steam Linux-native games

(At the time I did the steps I describe in this section, I didn't realize there were additional considerations for Windows-native games running under Proton. So in retrospect, this section by itself is only sufficient for running Linux-native OpenXR Steam games, and I'm not sure if any of those actually exist. To get Windows-native games working, you'll need both this section and the next section.)

I talked at the start of this post how I've had to run Steam on my host instead of a sandbox. That said, Steam itself provides some sandboxing for the games themselves. Steam runs games using Pressure Vessel, which uses Bubblewrap to run the game in a mount namespace with a "Steam Runtime" base image. The latest Steam Runtime v3 "Sniper" base image for example is a Debian 11 base plus a bunch of audio/video libraries preinstalled. Of course Steam does this primarily to make it easier for games to target a single Linux distro's libraries rather than the large number of distros their players might use, not for security (for example it mounts the host's /home as-is; separate /home sandboxes for each game is not planned right now). This means that, among other things, /usr for the game process is actually the Steam runtime container image's /usr, not the host's. The host's /usr is mounted at /run/host/usr instead.

Monado has a client-server architecture. The host runs an application monado-service that actually handles the hardware drivers, and listens on a Unix domain socket monado_comp_ipc. The OpenXR runtime library loaded by an OpenXR application is libopenxr_monado.so which then connects to monado_comp_ipc to talk to monado-service. Furthermore the way that OpenXR applications look up the runtime library they should load is by having the user configure a runtime to be the default, via a file ~/.config/openxr/1/active_runtime.json. For using Monado, this file looks like this:

{
    "file_format_version" : "1.0.0",
    "runtime" : {
        "library_path" : "../../../../../usr/lib64/libopenxr_monado.so",
        "name" : "Monado"
    }
}

Notice how the library_path is relative to the location of the active_runtime.json. Monado recommends doing this but doesn't say why; the reason is that this allows the library to be located even if the /home and /usr that contain the active_runtime.json and libopenxr_monado.so are mounted under some arbitrary path rather than under /.

You might think this is exactly the case with the Steam Runtime sandbox, but alas as I said, the host's /usr is mounted at /run/host/usr but the host's /home is still mounted at /home, so this relative link still does not resolve. One way to resolve this is to put the relative path to /run/host/usr/lib64/libopenxr_monado.so in the manifest, but that would then break clients running on the host. A better option is to tell Steam games to use a different manifest via the XR_RUNTIME_JSON env var. The easiest such manifest is /usr/share/openxr/1/openxr_monado.json since it's already part of the monado package and contains a path relative to itself, "library_path" : "../../../lib64/libopenxr_monado.so". This means running games with XR_RUNTIME_JSON=/run/host/usr/share/openxr/1/openxr_monado.json so that the relative path resolves to /run/host/usr/lib64/libopenxr_monado.so, which is what we want.

Next, because the client library needs access to the monado_comp_ipc socket, another env var is needed to mount this inside the game's mount namespace. The socket is expected to be under $XDG_RUNTIME_DIR, so the additional env var is PRESSURE_VESSEL_FILESYSTEMS_RW=$XDG_RUNTIME_DIR/monado_comp_ipc

There is one last problem. We're asking the game to load /run/host/usr/lib64/libopenxr_monado.so which is a library compiled for OpenSUSE, but the game is running in a Debian 11 mount namespace (the base OS of the Steam Runtime v3 "Sniper") and is likely itself compiled for Debian 11. It's usually a bad idea to mix libraries from other distributions because distributions differ in compile flags (eg rpath in libraries), directory layouts (eg plugins in /usr/lib vs /usr/lib64 vs /usr/libexec, config files in /usr/share vs /usr/lib vs /etc) and so on. There's also the matter of the dependencies of libopenxr_monado.so itself - they could exist in /run/host/usr/lib64 but the loader that the game uses would only look in /usr/lib/.... We could set LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/run/host/usr/lib64, but that could cause other libraries from the host to be loaded instead of the container's. Indeed, this last problem is exactly what happens with the monado package from the hardware:xr repository - it is compiled to link dynamically to OpenSUSE's libcjson.so but this library is not present in the Steam Runtime container image, so the libopenxr_monado.so fails to load because of missing dependencies and the game ignores it.

Fortunately libcjson.so is the only dependency that's missing, and Monado knows about this problem and thus makes it possible to link to its own static copy of libcjson by setting the -DXRT_HAVE_SYSTEM_CJSON=OFF cmake flag at build time. Unfortunately the maintainer of the OpenSUSE monado package was not happy with this approach, because this still assumes that a library compiled for OpenSUSE can be loaded by a process running in Debian 11. Indeed libopenxr_monado.so has many other dynamically linked library dependencies apart from libcjson, which only happen to not be a problem because they happen to be provided by the Steam Runtime container image in a compatible way. Ideally one would have a separate monado-steam package that contains its own libopenxr_monado.so independent from the monado package (and corresponding independent openxr_monado.json manifest) that is compiled in the Steam Runtime build environment.

Alas, the Steam Runtime build environment is itself a container image (registry.gitlab.steamos.cloud/steamrt/sniper/sdk), and I was not able to find a way for an OBS build to run a build in a container that would work inside the chroot that osc build uses. OBS can build Docker images, and OBS can build Debian 11 packages in a Debian 11 chroot, but the situation here is neither. We want to build an OpenSUSE RPM package where the build step is to run a docker / podman container with the source directory mounted inside, then back in the build chroot use the resulting libopenxr_monado.so in the final package.

The SteamVR lighthouse driver I mentioned earlier has the same problem in reverse. The situation with that is that the monado-service binary compiled for OpenSUSE is asked to load a SteamVR library (~/.local/share/Steam/steamapps/common/SteamVR/drivers/lighthouse/bin/linux64/driver_lighthouse.so) that is presumably compiled for the Steam Runtime. It happens to work, but it would be nice to avoid it, but the tracking with libsurvive is so bad that there isn't a good alternative.

Just to make more progress, I made the change to compile the OpenSUSE monado package itself with the -DXRT_HAVE_SYSTEM_CJSON=OFF cmake flag only in my OBS repository's package. Later it turned out to not be necessary (keep reading), so I've since removed this change again.

In any case, at this point Beat Saber was still unable to connect to Monado and insisted that it could not find an OpenXR runtime. I eventually discovered there was more work required for Windows-native games like Beat Saber.

Monado for Steam Windows-native games

When SteamVR originally came out in 2015 for the HTC Vive headset, SteamVR came up with an API and called it OpenVR. It was expected to be used not just for Steam games but also applications like web browsers for showing 180°/360° video, for example. But Oculus came up with its own API for its hardware, and Windows came up with WMR, so there was no standardization. In 2017, the Khronos Group started working on OpenXR to create that standardization, and also handle both VR and AR hardware with the same runtime now that AR was becoming a thing.

So old games likely require an OpenVR runtime. Newer games might require OpenXR instead. In 2020, SteamVR implemented support for OpenXR applications in addition to OpenVR applications, so both kinds of games would work in SteamVR today. Monado however is only an OpenXR implementation. We just need an OpenVR compatibility layer that converts it to OpenXR, and the most popular open-source implementation for Linux is OpenOVR (née OpenComposite).

Beat Saber is actually an OpenXR application, so it would seem that having OpenOVR is not necessary. However because Beat Saber is a Windows game, it uses OpenXR through Proton, and Proton has a quirk that it uses OpenVR to "initialize OpenXR games" and fails if it can't find an OpenVR runtime. I'm not sure what it means for an OpenVR runtime to "initialize OpenXR games" in a way that they end up eventually using the OpenXR runtime anyway, but this is the wording used in this GH issue comment.

OpenOVR however does not have any stable releases, so I'm not sure it can be packaged for OpenSUSE in a way that will be accepted by any of the official repos. In any case, the vrclient.so library produced by OpenOVR would have the same cross-distro linkage problem I wrote about for libopenxr_monado.so above, so it's best to nip that problem in the bud and just compile it in a Steam Runtime SDK container. Furthermore, OpenVR has its own manifest file that tells the application the path of the library, ~/.config/openvr/openvrpaths.vrpath. In order for OpenVR to work on both the host (for any applications that need it) and with Steam games, it's best if the library path is under /home too so that it's the same path in both cases. This means that my setup is to build OpenOVR with this script:

#!/bin/bash

if ! [ -d ~/src/OpenOVR ]; then
    git clone --recursive gitlab.com:znixian/OpenOVR ~/src/OpenOVR
fi

podman container run \
    --rm \
    "--volume=$HOME/src/OpenOVR:/src" \
    registry.gitlab.steamos.cloud/steamrt/sniper/sdk \
    bash -c '
        set -euo pipefail

        apt update -y
        apt dist-upgrade -y --autoremove --purge


        rm -rf /src/build
        mkdir /src/build
        cd /src/build
        cmake -DCMAKE_BUILD_TYPE=Release ..
        make -j
    '

... such that I end up with ~/src/OpenOVR/build/linux64/bin/vrclient.so compiled for the Steam Runtime, then create the ~/.config/openvr/openvrpaths.vrpath.openovr manifest with this content:

{
    "config" : [
        "/home/arnavion/.local/share/Steam/config"
    ],
    "external_drivers" : null,
    "jsonid" : "vrpathreg",
    "log" : [
        "/home/arnavion/.local/share/Steam/logs"
    ],
    "runtime" : [
        "/home/arnavion/src/OpenOVR/build"
    ],
    "version" : 1
}

... and finally symlink ~/.config/openvr/openvrpaths.vrpath to openvrpaths.vrpath.openovr. (This strategy of symlinking openvrpaths.vrpath to the actual config allows for easily switching the symlink back to the original SteamVR config if needed; it is also described in the Monado setup doc mentioned above.)

Note that OpenOVR uses some C++ features that the old gcc in the Steam Runtime SDK container does not support, so I had to apply this patch to remove those features first:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 164fa89..b872224 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -62,7 +62,7 @@ else ()
    # There is also -Bsymbolic set when linking the final shared library, see the bottom of this file
    # FIXME the Vulkan functions still get exported, hopefully they won't fight with the application's ones.
     add_definitions(-fvisibility=hidden)
-   add_compile_options(-Wall -Wextra -Wpedantic -pedantic-errors -Wno-unused-parameter -Wno-missing-field-initializers -Wno-format-security)
+   add_compile_options(-Wall -Wextra -Wpedantic -Wno-unused-parameter -Wno-missing-field-initializers -Wno-format-security)
    set(ERROR_ON_WARNING_FLAG -Werror)
 endif ()
 if (ERROR_ON_WARNING)
diff --git a/OpenOVR/Misc/Input/InteractionProfile.h b/OpenOVR/Misc/Input/InteractionProfile.h
index 02fc121..d8f4e21 100644
--- a/OpenOVR/Misc/Input/InteractionProfile.h
+++ b/OpenOVR/Misc/Input/InteractionProfile.h
@@ -165,10 +165,9 @@ public:
    std::optional<T> GetProperty(vr::ETrackedDeviceProperty property, ITrackedDevice::TrackedDeviceType hand)
        const
    {
-       using enum ITrackedDevice::TrackedDeviceType;
-       if (hand != HAND_NONE && propertiesMap.contains(property)) {
+       if (hand != ITrackedDevice::TrackedDeviceType::HAND_NONE && propertiesMap.contains(property)) {
            hand_values_type ret = propertiesMap.at(property);
-           return std::get<T>((hand == HAND_RIGHT && ret.right.has_value()) ? ret.right.value() : ret.left);
+           return std::get<T>((hand == ITrackedDevice::TrackedDeviceType::HAND_RIGHT && ret.right.has_value()) ? ret.right.value() : ret.left);
        } else if (hmdPropertiesMap.contains(property)) {
            return std::get<T>(hmdPropertiesMap.at(property));
        }

With this, finally, I was able to get Beat Saber working with Monado without SteamVR running.

Base Station Power Management

The next problem is making the lighthouses go to sleep when they're not being used. SteamVR on Windows supports base station power management by communicating with the base stations via Bluetooth. It doesn't require the PC itself to have Bluetooth because it uses a Bluetooth adapter inside the VR headset. For the longest time SteamVR on Linux did not support power management, so if you search for this you will find many pages of people saying it doesn't work on Linux. SteamVR on Linux did start supporting it since 2023, though apparently only for the v2.0 lighthouses. It's not enabled by default though, so you will need to start SteamVR, open its menu, go to Devices -> Base Station Settings, and turn on power management there. This will make it so that closing SteamVR makes the base stations go to sleep.

The problem though is that the Monado setup works without launching SteamVR, so I needed an independent way to wake up the base stations and put them back to sleep. Ideally this would use the VR headset's Bluetooth adapter in the same way that SteamVR uses it, but there does not seem to be any information about how SteamVR does it. So the alternative is to use the PC's own Bluetooth adapter. My PC doesn't have one, but it does have an M.2 slot for one, and I was thinking of getting one anyway because I currently use a wired controller and headset and have been thinking of getting Bluetooth ones. So I purchased one such adapter based on the Intel AX210 chip (AX211 was not an option because I use an AMD CPU) from a noname Chinese brand on Amazon.

(If you have an Android phone, I saw mentions of an Android app to do so using the phone's Bluetooth instead. I use a Linux phone, and while it does have Bluetooth and I do run Waydroid on it, Waydroid can't make the phone's Bluetooth available to the inner LineageOS container, so it doesn't help.)

At first I was confused that I was unable to pair the base stations with my PC, but apparently this is by design and the SteamVR 2.0 base stations are not expected to be paired first. It is sufficient to just connect to them and send the power management commands that way. (This also seems to imply that anyone within range of your base stations can turn them on or off ?!)

So to send those commands, there were two options. One was the lh2ctrl script and the other was the monado-cli lighthouse <off|on> command using monado-cli from Monado. Obviously it would be easier to use the latter because I already had it, so I tried that first. However it seemed to not do anything and the lighthouses would remain off or on as they were. Under the hood, monado-cli talks to BlueZ over D-Bus to connect to the lighthouses and send them commands, and I was able to see using busctl --monitor that monado-cli was telling BlueZ's bluetoothd to connect to the lighthouses, but then it was not sending any commands to tell them to power off / on.

Just to be sure, I tried the lh2ctrl script, and that worked fine. I checked its code to see what it was doing, but it uses the bluepy Python library, which under the hood launches a C binary that uses BlueZ server code to send the Bluetooth commands itself, rather than talk to the BlueZ daemon as a D-Bus client. It did however give me enough information of how the whole thing works. The PC connects to the lighthouse, then enumerates its "characteristics" looking for one with a particular UUID, then "writes a value" to that characteristic that indicates whether it should wake up or go to sleep. Armed with this information, I was able to replicate what the monado-cli lighthouse command ought to be doing by manipulating the BlueZ D-Bus objects in d-feet by hand. That worked, so now I started debugging the monado-cli command code to see what it was doing differently.

It turned out that monado-cli was ignoring the characteristics for power management because it expected those characteristics to support notifications, but according to BlueZ those characteristics did not do so. So monado-cli did not find any characteristics to write to, and just silently exited without doing anything. I don't know if this fact that the characteristics don't support notifications was a problem with all base stations, or just all SteamVR 2.0 base stations, or just the base stations I have, or my distro's BlueZ library, or something else. In any case, the power management code did not seem to require that these characteristics supported notifications; the code to find the characteristics was just common code that was also used by other code that did require the characteristics to support notifications. I made an MR to Monado to fix this. I've also added that MR as a patch to the hardware:xr repository's monado package.

So now it's just a matter of wiring up the monado.service unit to run monado-cli lighthouse on when it starts and run monado-cli lighthouse off when it stops, via ExecStartPre and ExecStopPost, to get the same effect as SteamVR. I haven't done this yet because there seems to be one more problem - sometimes monado-cli lighthouse on successfully tells my lighthouses to turn on, and they flash their LED as they do when the start powering on, but then they seem to give up and remain in sleep. Running monado-cli lighthouse on a second time fixes it. This problem only happens about one in twenty times. I haven't experienced this problem with the lh2ctrl script, so there is possibly some more difference between it and monado-cli (see the MR comments for one such difference I identified), or it might just be some firmware bug with the lighthouses and a coincidence that I don't experience it with lh2ctrl. I'm still investigating this.

VR Video

Apart from playing games, I was also interested in getting 180°/360° videos working.

Since OpenVR never got standardized, browsers that implemented OpenVR via WebVR either removed it or plan to remove it or never started implementing it. There is a corresponding WebXR that uses OpenXR, but neither Firefox nor Chromium seem to support it on Linux, or at least I was not able to get any of the WebXR samples at immersive-web.github.io to work on either. These third-party patches apparently make them work but I didn't care to recompile browsers to try them.

What did end up working is two desktop players.

The first is vr-video-player that can use SteamVR directly and not need Monado, or use OpenOVR and forward to Monado. It can either capture any X11 window, or it can play any file that mpv can play (since it uses libmpv). It has a bunch of CLI flags that allow it to show not just 180°/360° projections but also stereoscopic projection (ie where the application / video renders the two viewpoints side-by-side) and monocular projection on a curved or flat surface. Picking the right flags for a particular video requires some trial and error because I don't always find it obvious by looking at the video on a monitor what projection it uses. vr-video-player does have keyboard shortcuts to change some aspects of the projection, which I can blindly operate while wearing the headset. But changing the projection itself requires stopping and restarting vr-video-player with different CLI flags, so I have to keep taking my headset off and on to do it.

The second is sphvr that uses OpenXR and thus requires Monado. It shows images and videos through gstreamer. Once I got Monado working, I tested this with images and videos and it works.

vr-video-player is not able to render all videos; sometimes it just renders a black screen and then locks up. I haven't yet debugged it to see why this happens, but it does seem to only happen with some videos and not others, so perhaps it's something related to the video format / codec. In these cases, there is the workaround of playing the video in regular mpv (with $WAYLAND_DISPLAY unset so that it renders an X11 window) and then using vr-video-player $mpv_window_id so that it captures and renders the mpv window instead. However this does have the downside that the playback is very choppy (~15fps).

sphvr seems to be able to handle all videos including those that break vr-video-player, but it has the disadvantage that it has no keyboard controls for seeking the video and so on. vr-video-player has the advantage of using standard mpv keybinds.

Neither vr-video-player nor sphr have any stable releases, so I'm not sure they can be packaged for OpenSUSE in a way that will be accepted by any of the official repos. For now I've packaged them here and here in my OBS repository. sphvr requires gulkan v0.16 and gxr v0.16. The packages in OpenSUSE repos were v0.15 so I sr'd v0.16 to them. gulkan is in X11:Wayland and gxr is in hardware:xr, so if you want to use the packages from my OBS repo, you'll need to add those repos too. (gulkan is in Factory too but so far the maintainer has not forwarded v0.16 to it.)

Camera

The Valve Index headset has a binocular camera, and SteamVR on Windows is apparently able to pipe the feed through to the headset screen so that you can see out into the world while still wearing the headset. There is also apparently some computer vision integration to detect objects and show outlines and such instead of a full color video. None of this works with SteamVR on Linux.

However the camera does appear as a v4l2 device showing a stereoscopic (side-by-side) combination of the two lenses. It should thus be possible to pipe this to vr-video-player or sphvr to be able to see out of the headset while wearing it. I did not try sphvr, but with vr-video-player it worked although it showed a very cross-eyed image. This GH issue comment told me how to patch vr-video-player to change the left eye offset to fix it, and indeed it works. However, rather than patch it that way (which would make the Index camera work but break regular video), I adapted it into a new CLI flag to specify the left eye offset so that both offsets can be used without having to recompile to switch.

vr-video-player --flat --no-stretch --eye-left-offset -0.5 --video 'av://v4l2:/dev/video0' --mpv-profile low-latency

I've added this patch to my OBS package already. I'll submit it upstream later when I have time to clean up the patch for submission.

sphvr can also render the v4l2 device via:

sphvr -o xr 'v4l2:///dev/video'

... and does not have the cross-eyed image issue.

SteamVR in a sandbox, revisited

Since SteamVR is no longer involved other than the lighthouse driver, I wondered if it might be possible to go back to running Steam in my "non-oss" container. The original hardship with exposing the headset's USB dev nodes should not be a problem because monado-service would still be running on the host, so it's only a matter of mounting the monado_comp_ipc socket into the container. To play it safe with ABI issues, I still wanted to build Monado and OpenOVR in a Steam Runtime container.

So I modified my non-oss container image build to have an initial build stage that runs an registry.gitlab.steamos.cloud/steamrt/sniper/sdk container to build Monado and OpenOVR. For Monado, since I only need to build the libopenxr_monado.so binary, the build only needs to do:

cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_DOC:BOOL=OFF \
    ..
make -j openxr_monado

And for OpenOVR, the same caveat as above applies, namely that the build script has to patch it to remove new C++ features that the Steam Runtime SDK's gcc does not support.

For Monado, the container build places $monado_dir/build/src/xrt/targets/openxr/libopenxr_monado.so at /usr/share/monado-steam/libopenxr_monado.so and generates /usr/share/monado-steam/openxr_monado_steam.json with the content:

{
    "file_format_version" : "1.0.0",
    "runtime" : {
        "library_path" : "./libopenxr_monado.so",
        "name" : "Monado"
    }
}

For OpenOVR, the container build places $openovr_dir/build/bin/linux64/vrclient.so at /usr/share/openovr-steam/bin/linux64/vrclient.so, and generates ~/non-oss-root/steam/.config/openvr/openvrpaths.vrpath with the content:

{
    "config" : [
        "/home/arnavion/.steam/debian-installation/config"
    ],
    "external_drivers" : null,
    "jsonid" : "vrpathreg",
    "log" : [
        "/home/arnavion/.steam/debian-installation/logs"
    ],
    "runtime" : [
        "/run/host/usr/share/openovr-steam"
    ],
    "version" : 1
}

Finally, the container build defines env vars:

ENV PRESSURE_VESSEL_FILESYSTEMS_RW "/run/user/$uid/monado_comp_ipc"
ENV XR_RUNTIME_JSON '/run/host/usr/share/monado-steam/openxr_monado_steam.json'
ENV IPC_IGNORE_VERSION 1

The first two are what I would normally need to configure each VR game with individually, but setting them on the whole container makes that unnecessary. The third is because libopenxr_monado has a check to make sure it's compatible with the monado-service on the other side of the IPC socket. Because we built the client and server on different OSes and build systems, those versions happen to not match, but we know they're the same version so this env var suppresses that check.

After all this, I was finally able to play Beat Saber inside my original sandboxed Steam. I'll still need the host Steam in case I need to re-run SteamVR Room Setup or update the leftover SteamVR install, but I can just install it then. Note that I do need the leftover SteamVR install to remain on the host, since that is where the SteamVR lighthouse driver loaded by monado-service lives.

Future

Apart from the few pending patches / packages mentioned above, there are a few other things I want to try:

  • Beat Saber running in Monado has very low haptic feedback on the Index controllers compared to Beat Saber running in SteamVR. I need to check if this is an issue with Monado or something else.

  • Monado requires the controllers to be powered on before starting Monado because it is unable to "hotplug" them. They acknowledge that this is a nice-to-have feature and would appreciate someone implementing it.

  • SteamVR also does power management for the controllers, specifically it turns them off when exiting SteamVR. Monado does not have a way to do this. It's not a big deal since I can just leave them plugged in to USB power, but I did forget a few times and had to skip the next day's VR session because the controllers had run out of battery. I assume this is also based on Bluetooth somehow, so I need to investigate it.

I will update this post as I do these things.