NVIDIA fix: results on the RTX 5080 PC (2026-10-01)

PC: RTX 5080 (GB203, 10de:2c02) + AMD Granite Ridge iGPU, MSI MAG 341C OLED on the NVIDIA DP-1 (3440x1440, 175 Hz), CachyOS, Limine, kernel 7.2.8-1-cachyos, nvidia-open 615.71.09, gamescope 3.16.31-1, gamescope-session-cachyos 1.1.6-1.

Result: the kernel-parameter fix does not cure the picture

Baseline (before): gaming mode corrupted without HDR; with HDR it worked but not at 175 Hz and not full screen.
After apply + reboot (modeset = Y, fbdev = Y, NVIDIA modules in the initramfs): flickers heavily, artifacts, still no 175 Hz. modeset/fbdev were already Y live before the fix (driver 615), so the fix changed little.
Cause is not the parameters: NVIDIA's open bug "display modes above 2560x1440@120 with HDR cause flickering/corruption within gamescope-session" (internal bug 5240452, April 2025, no fix as of March 2026; 555 up to 595+, RTX 3070 Ti/4090/5090; corruption is display-side only): https://forums.developer.nvidia.com/t/display-modes-above-2560x1440p-120hz-with-hdr-enabled-cause-flickering-corruption-within-gamescope-session/295314

Bugs found in the fix on this PC (fixed in commit a5e9015 on feature/nvidia-gaming-fix)

CachyOS writes Limine's line as KERNEL_CMDLINE[default]+="..."; the sed only matched =, so apply stopped with "Couldn't add the NVIDIA parameters".
The kernel treats nvidia_drm.modeset=1 and nvidia-drm.modeset=1 alike; the check only knew the hyphen.
tests/nvidia-hardware-test.sh did not set SCRIPT_DIR, so patch_file read /patches/....

What gamescope did (journal, gaming mode)

Picks the RTX 5080 (card1), DP-1, mode list has 3440x1440@175/144/100. It selected @60 by default; with -W 3440 -H 1440 -r 175 added (a copy of /usr/lib/steamos/gamescope-session, whose last line is -O ... \ so args need a copy) it selected @175 and did not crash, but the picture was the same. Steam then sets the Xwayland size to 1920x1080 (not full screen).
vkGetPhysicalDeviceFormatProperties2 returned zero modifiers for 2 16-bit formats: harmless here.
A gamescope SIGABRT (double free or corruption) at session exit, backtrace through libnvidia-eglcore and libvulkan_radeon.
The PC froze once and was reset after a gaming-mode session (no kernel error logged).

What works: nested gamescope inside KDE (Wayland), borderless

gamescope -W 3440 -H 1440 -r 175 -f ...: artifacts once the window is focused and fullscreen (unfocused it is fine). Hypothesis, not proven: KWin's direct scanout of the fullscreen window.
gamescope -W 3440 -H 1440 -r 175 -b -e --hdr-enabled -- steam -gamepadui (borderless): clean, full screen, about 175 Hz.
So on NVIDIA the separate gamescope session (the SteamOS conversion) is blocked by the driver bug; a nested borderless gamescope works. Whether Steamify gets a nested option for NVIDIA is the user's decision.

Cause confirmed: KWin's direct scanout; a session without Plasma works (2026-10-01, later)

A TTY session of only kwin_wayland --drm + gamescope -b + steam -gamepadui (no Plasma shell, no panel) artifacted too (KWin hands the covering window straight to the display).
Same session with KWIN_DRM_NO_DIRECT_SCANOUT=1 (exists in KWin 6.7.5's libkwin; not in the kwin_wayland binary's strings): NO artifacts, full screen, about 175 Hz (gamescope logs Changed refresh to: 174.962hz). Animations felt a bit clunky (cause unknown: an extra composition step without scanout, no CAP_SYS_NICE for gamescope, no VRR, nested frame pacing; other switches to try: KWIN_DRM_DISABLE_TRIPLE_BUFFERING, KWIN_DRM_PREFER_COLOR_DEPTH).
Launcher used for the test: ~/.local/bin/steamify-nvidia-session-test (run from a free TTY; KWin as the compositor, gamescope -W 3440 -H 1440 -r 175 -b -e --hdr-enabled -- steam -gamepadui, a watchdog ends it after 90 s). Lesson: the first version's timeout left KWin/gamescope/Steam running (the cleanup came after a pipeline that waits for them); pkill -f '<pattern>' in a command that contains the pattern kills the caller.
Open: a real session entry (/usr/share/wayland-sessions/, needs sudo) and the product decision: an NVIDIA session mode in Steamify instead of the gamescope DRM session. Not built, not in the product repo.

Smoothness: the TTY-only KWin session stutters, Plasma does not; Big Picture in Plasma is perfect (2026-10-01, evening)

TTY session (kwin_wayland --drm under dbus-run-session, no Plasma): the mouse and windows stutter every 1-2 s, with gamescope (clunky) and without it (steam -gamepadui as KWin's only client). Same in both: not gamescope, not Steam. Plasma's own KWin is smooth.
Tried without effect: KWIN_DRM_DISABLE_TRIPLE_BUFFERING=1. Measured: the test KWin used about 2% CPU, no spikes, but its main thread dropped to 0% about every 2 s (blocked, not busy); scheduling (RR main thread, nice -12) and capabilities equal to Plasma's KWin; NVIDIA is boot_vga; no relevant environment differences found. Cause of the stutter not found.
Plain steam -gamepadui (Big Picture, no gamescope) in the normal Plasma session: perfect (smooth, no artifacts, no env var needed).
Direction: for NVIDIA PCs a Plasma session that shows only Steam's Big Picture (no desktop, no panel) instead of the gamescope session. Gives up gamescope's features (HDR handling, FSR, overlay; most -steamos3 menus need gamescope).

State of the PC after the test (to undo)

The fix is applied (Limine file backup /etc/default/limine.bak-gamescope-wizard, early-load file, pacman hook): undo in INSTRUCTIONS.md.
Test launcher ~/.local/bin/steamify-nvidia-session-test, log ~/steamify-nvidia-session-test.log.
User override of the gaming session: ~/.config/systemd/user/gamescope-session.service.d/steamify-nvidia-test.conf and ~/.local/share/steamify/nvidia-test/.

5.8 KiB Raw Blame History