backend/x11: add SHM readback fallback when buffer import fails

On some NVIDIA systems, the DRI3 modifier list only advertises
DRM_FORMAT_MOD_INVALID while the renderer requires a specific
modifier (e.g. BLOCK_LINEAR). As a result, the X11 backend cannot
import renderer buffers via DRI3 and output commits fail.

Allow the renderer to allocate buffers with its preferred modifier
when DRI3 doesn't expose explicit modifiers. If buffer import fails,
fall back to CPU readback: read pixels from the rendered texture into
shared memory and upload the result to a pixmap via xcb_shm_put_image
before presenting it.

This ensures the X11 backend can still produce output even when the
X server cannot import the renderer's buffers.

While implementing this fallback, split the existing SHM capability
flag into two:

  have_shm: SHM extension available (required for readback fallback)
  have_shm_pixmaps: server supports shared pixmaps

Also fix the SHM version check which incorrectly used || instead of
a proper >= 1.2 comparison.

Tested with labwc 0.9.5 (WLR_BACKENDS=x11) on Ubuntu 24.04 with an
NVIDIA GeForce GTX 1650 (driver 580.126.18), where output commits
previously failed with "Failed to commit frame".
This commit is contained in:
Jonathan Marler 2026-03-10 18:10:51 -06:00
parent 7ccef7d9eb
commit 002659e6f4
3 changed files with 176 additions and 5 deletions

View file

@ -8,6 +8,7 @@
#include <wayland-server-core.h>
#include <xcb/xcb.h>
#include <xcb/present.h>
#include <xcb/shm.h>
#include <pixman.h>
#include <wlr/backend/x11.h>
@ -48,6 +49,15 @@ struct wlr_x11_output {
uint64_t last_msc;
struct {
xcb_shm_seg_t seg;
uint8_t *data;
size_t size;
int width, height;
xcb_pixmap_t pixmap;
xcb_gcontext_t gc;
} readback;
struct {
struct wlr_swapchain *swapchain;
xcb_render_picture_t pic;
@ -74,6 +84,7 @@ struct wlr_x11_backend {
xcb_render_pictformat_t argb32;
bool have_shm;
bool have_shm_pixmaps;
bool have_dri3;
uint32_t dri3_major_version, dri3_minor_version;