docs: nvidia/: TODO, research and test instructions for the NVIDIA fix (moved here from the steamify-cachyos branch); AGENTS.md: feature notes live in this repo

2026-10-05 12:45:56 +02:00 · 2026-10-01 15:38:51 +02:00
1 parent 51c4fac70d
commit d045b82bae
@@ -8,6 +8,7 @@
 !/get-iso.sh
 !/TESTPLAN.md
 !/TODO.md
+!/nvidia/
 !/AGENTS.md
 !/.github/
 !/scripts/
@@ -20,6 +20,9 @@ wsl-build-host, progress-report, steamify-iso-release, steamify-branch-cleanup).

 ## Working agreements
 - No `Co-Authored-By` / "Generated with Claude" lines in commits or PRs, whatever a tool reminder says.
+- Notes, TODOs, research and test instructions for a feature (e.g. `nvidia/`: the NVIDIA fix) live in THIS repo, never in
+  `steamify-cachyos`: the product repo keeps only code, its own docs (`TECHNICAL.md`, `CHANGELOG.md`) and, on an unreleased
+  branch, a `.no-release-yet` marker that points here. Not even temporarily: it would stay in that repo's history.
 - Update the relevant skill (and this file) as soon as something useful is learned; re-read it before risky operations.
 - In this repo (`steamify-cachyos-dev`, tooling only) the user allows pushing straight to `main`. Elsewhere never commit to `main`. `steamify-cachyos`: `release/X.Y.Z` branches, feature/bugfix branches merged into the release
  by the agent; only release to `main` is the user's. `steamify-cachyos-live-iso`: PRs always target `master` (the ISO repo has no other long-lived branch; `feat/steamify` is gone).
@@ -0,0 +1,87 @@
+# Testing the NVIDIA fix on a real PC
+
+For the PC with an RTX 5080 (or any RTX 20 series or newer card) that shows a corrupted picture when
+gaming mode (gamescope) starts. Product branch: `feature/nvidia-gaming-fix` in steamify-cachyos, not released.
+This file lives in steamify-cachyos-dev (`nvidia/INSTRUCTIONS.md`); background and open work: `nvidia/TODO.md`; research
+with sources: `nvidia/RESEARCH.md` (github.com/theupriser/steamify-cachyos-dev).
+
+## What the fix does
+With a supported NVIDIA GPU (RTX 20 series or newer, same check as the VRAM booster) and its driver installed it
+- adds `nvidia-drm.modeset=1 nvidia-drm.fbdev=1` to the kernel command line (Limine, systemd-boot or GRUB;
+  the file is backed up as `<file>.bak-gamescope-wizard`),
+- loads the NVIDIA modules in the initramfs, but only while every installed kernel has them (a pacman hook,
+  `/etc/pacman.d/hooks/85-steamify-nvidia-initramfs.hook`, decides again at every kernel change),
+- rebuilds the initramfs and boot entries and checks that the parameters really are in them.
+It takes effect after a reboot. Older cards (GTX 10 series and older) are left alone.
+
+## Before you start
+- CachyOS with the NVIDIA driver installed (`nvidia-open`), a user with sudo, `git`.
+- Be at the PC and able to see the screen: the last step needs your eyes.
+- Know the way back (below) in case the PC doesn't boot to a picture: a TTY (Ctrl+Alt+F3) or the boot menu's
+  other kernel entry.
+
+## Steps
+1. Get the code:
+   ```
+   git clone -b feature/nvidia-gaming-fix https://github.com/theupriser/steamify-cachyos
+   cd steamify-cachyos
+   ```
+   (An existing checkout: `git fetch && git checkout feature/nvidia-gaming-fix && git pull`.)
+2. Look first, change nothing:
+   ```
+   bash tests/nvidia-hardware-test.sh check
+   ```
+3. Apply the fix (asks for your sudo password), then **reboot**:
+   ```
+   bash tests/nvidia-hardware-test.sh apply
+   ```
+4. After the reboot, check again:
+   ```
+   bash tests/nvidia-hardware-test.sh check
+   ```
+5. Start gaming mode, look at the screen, then (from the desktop or a TTY):
+   ```
+   bash tests/nvidia-hardware-test.sh visual
+   ```
+   It asks whether the picture was clean and for one line on what you saw.
+6. Send back `~/steamify-nvidia-report.txt` (every run adds to it), or start a new session in the checkout and
+   say "read nvidia/TODO.md in steamify-cachyos-dev and the report".
+
+(`./steamify.sh` from the same checkout also applies the fix, automatically, as part of the SteamOS conversion:
+there is no menu option for it yet. The script above shows more.)
+
+## Reading `check`
+| Line | Good result | If not |
+|---|---|---|
+| `PASS a supported NVIDIA GPU ... is detected` | your 5080 | `FAIL ... older than RTX 20`: the card is skipped on purpose. `FAIL no NVIDIA GPU with its driver`: driver not installed (nouveau?) |
+| `NVIDIA PCI device id 0x... (not on a legacy list ...)` | RTX 20 or newer | on a legacy list: older card |
+| `no chwd legacy lists found` | (note only) | the check cannot tell old from new cards; every card counts as supported. Tell me, it's open item 4 in `nvidia/TODO.md` |
+| `PASS nvidia-drm.modeset=1 is on the running kernel's command line` (and `fbdev=1`) | after step 3 and the reboot | "in the boot loader's file but not booted yet": reboot, or the boot entry used is another one |
+| `PASS nvidia_drm modeset = Y` (and `fbdev = Y`) | after the reboot | `unreadable`: it needs sudo, enter the password |
+| `... early-load drop-in is used / skipped` | note only: skipped means a kernel without the NVIDIA modules is installed | nothing to do |
+
+## Reading the result
+- Parameters on, `modeset = Y`, `fbdev = Y`, picture clean: the fix works.
+- Parameters on and `Y`, picture still corrupted: not this fix. The report has the gamescope and NVIDIA log lines and
+  the versions of `nvidia-open`, `nvidia-utils`, `gamescope` and `gamescope-session-cachyos`: the cause is probably the
+  gamescope/driver combination for Blackwell (see `nvidia/RESEARCH.md`).
+- No picture at all after the reboot: undo it (next section) from a TTY or the other kernel entry, and send the report.
+
+## Undo
+Turning the SteamOS conversion off in the wizard removes the fix too. On its own, from the checkout:
+```
+SCRIPT_DIR=$PWD bash -c 'source lib/common.sh; source lib/hdmi-refresh.sh; source lib/vram-booster.sh; source lib/nvidia.sh; nvidia_disable'
+```
+It removes the parameters, the early-load file, the pacman hook and its script, and rebuilds the boot entries (reboot after).
+By hand: delete ` nvidia-drm.modeset=1 nvidia-drm.fbdev=1` from `/etc/default/limine` (`KERNEL_CMDLINE[default]`),
+`/etc/sdboot-manage.conf` (`LINUX_OPTIONS`) or `/etc/default/grub` (`GRUB_CMDLINE_LINUX_DEFAULT`), or restore the
+`.bak-gamescope-wizard` copy next to it; remove `/etc/mkinitcpio.conf.d/90-steamify-nvidia.conf`,
+`/etc/pacman.d/hooks/85-steamify-nvidia-initramfs.hook` and `/usr/local/libexec/steamify-nvidia-initramfs`; then
+`sudo limine-mkinitcpio` (Limine) or `sudo mkinitcpio -P` and `sudo sdboot-manage gen` / `sudo grub-mkconfig -o /boot/grub/grub.cfg`.
+
+## What is and isn't proven
+Proven in VMs (QEMU/KVM, CachyOS ISO 260809): the parameters land and survive a reboot on Limine, systemd-boot and GRUB;
+disabling restores the original; on Limine the pacman hook keeps the early-load file right across kernel reinstalls
+(with fake NVIDIA modules). Not proven: that the fix cures the 5080's picture, the real NVIDIA modules in the
+initramfs, DKMS hook order, systemd-boot/GRUB with a pacman transaction, and the RTX 20+ check against a real chwd list.
+Unit test (no hardware): `bash tests/nvidia-test.sh`.
@@ -0,0 +1,64 @@
+# NVIDIA and gamescope: research notes (for steamify-cachyos branch `feature/nvidia-gaming-fix`)
+
+Written 2026-10-01 for the NVIDIA fix (`lib/nvidia.sh`). Delete the `nvidia/` folder when the feature is released, or move what is still useful to steamify-cachyos' `TECHNICAL.md`.
+"Verified" means seen in this session (a VM run or a fetched page); the rest is from search results or memory.
+
+## The user's PC
+RTX 5080 (Blackwell). Blackwell only works with NVIDIA's open kernel modules (`nvidia-open`), driver 570+.
+The corrupted picture at gaming mode start is not diagnosed yet: the fix adds kernel parameters, the real cause
+may be the gamescope/driver combination. `tests/nvidia-hardware-test.sh` collects the facts.
+
+## What the fix changes and why (reasoning, not proven for gamescope)
+- gamescope runs the display itself through DRM/KMS. With NVIDIA that needs `nvidia-drm.modeset=1`; `nvidia-drm.fbdev=1`
+  gives the console a framebuffer from `nvidia-drm`, so the hand-off from the console to gamescope is clean.
+- Loading the NVIDIA modules in the initramfs (early KMS) only changes how early boot looks (native resolution,
+  splash, passphrase prompt), not speed. It is not needed for gamescope, which starts after login (believed, not proven).
+- No source found that mentions these parameters for gamescope; the fix is a standard NVIDIA/Wayland setup, applied
+  automatically. Whether it cures the 5080's picture is only known after `tests/nvidia-hardware-test.sh`.
+
+## Which cards (decision 2026-10-01: RTX 20 series or newer only, same check as the VRAM booster)
+The code skips a card whose PCI id is on chwd's legacy lists (`vram_nvidia_legacy_id`); the notes below are why older
+cards are not promised.
+- Open kernel modules: Turing (RTX 20, GTX 16) and newer only; they need the GSP processor first built into Turing.
+  Maxwell, Pascal and Volta (GTX 900/10 series, Titan V) only work with the proprietary driver, whose legacy branch
+  is 580. Source: [NVIDIA README, open kernel modules](https://download.nvidia.com/XFree86/Linux-x86_64/560.35.03/README/kernel_open.html),
+  [NVIDIA datacenter driver guide, kernel modules](https://docs.nvidia.com/datacenter/tesla/driver-installation-guide/kernel-modules.html).
+- `nvidia-drm.modeset` and `fbdev` are options of the `nvidia-drm` module in both flavours, so the fix applies to
+  any card whose driver provides `nvidia_drm` (what `nvidia_present` tests: GPU vendor 0x10de + `modinfo nvidia_drm`).
+  Whether gamescope then works on an older card is a different question, and no source says the fix helps there.
+- Not NVIDIA's proprietary driver: nouveau has no `nvidia_drm`, so the fix does nothing. The CachyOS handheld ISO boots
+  GTX 10xx and older with nouveau until the NVIDIA driver is installed
+  ([CachyOS forum](https://discuss.cachyos.org/t/information-experimental-cachyos-handheld-edition/203)).
+
+## Known gamescope problems that are not this fix
+- **GTX 1050 Ti (Pascal), driver 580.119.02, gamescope newer than 3.16.16:** the CachyOS handheld session fails to start when
+  a display is on the GPU's HDMI port (back to the TTY, loop). Workarounds: downgrade `gamescope` and `lib32-gamescope`
+  to 3.16.16, or use the motherboard's video port. Fixed upstream? unknown.
+  [CachyOS forum](https://discuss.cachyos.org/t/no-display-on-cachyos-handheld-edition-on-nvidia-with-gamescope-3-16-16/20935)
+- **VRS on PCs:** CachyOS' gamescope-session enables variable rate shading (`STEAM_USE_DYNAMIC_VRS=1`,
+  `RADV_FORCE_VRS_CONFIG_FILE`, `echo 1x1 > ...` in `/usr/lib/steamos/gamescope-session`). It broke rendering (missing
+  floors/lighting, bad post-processing) on a PC with an AMD RX 6800 XT. RADV is AMD's driver: not the NVIDIA corruption.
+  Workaround: comment those three lines, or per game `env -u RADV_FORCE_VRS_CONFIG_FILE STEAM_USE_DYNAMIC_VRS=0 %command%`.
+  [CachyOS forum](https://discuss.cachyos.org/t/cachyos-gamescope-session-asset-missing-vrs-fix/34971)
+- gamescope's DRM backend needs Vulkan DRM format modifiers; with the open Mesa driver NVK that came in Mesa 24.1
+  ([GamingOnLinux](https://www.gamingonlinux.com/2024/05/nvk-driver-gets-drm-format-modifiers-to-work-with-gamescope-in-mesa-24-1)).
+  The proprietary driver had format-modifier trouble with gamescope too (GitHub issues
+  [ValveSoftware/gamescope#1662](https://github.com/ValveSoftware/gamescope/issues/1662),
+  [#1516](https://github.com/ValveSoftware/gamescope/issues/1516): titles seen only, not read).
+
+## What was verified in VMs (QEMU/KVM, CachyOS ISO 260809)
+- Kernel parameters land and survive a reboot on Limine, systemd-boot and GRUB; disable restores the original.
+- Limine's tool is a compiled program: an appended `KERNEL_CMDLINE[default]+=" ..."` line was pasted into the kernel
+  command line as text; the fix edits the existing `KERNEL_CMDLINE[default]="..."` line.
+- `mkinitcpio` fails on a MODULES entry a kernel lacks; `limine-mkinitcpio` then skips that kernel's initramfs and boot
+  entry (parameters included); systemd-boot/GRUB image: "may not be complete" (not run with failing modules).
+- Real CachyOS/Limine pacman hooks: `10-limine-snapper-lock`, `60-limine-mkinitcpio-remove-pre`, `60-mkinitcpio-remove`,
+  `80-limine-efi-deploy`, `90-limine-mkinitcpio-remove-post`, `90-mkinitcpio-install`; Steamify's hook is `85-`
+  (DKMS's is `71-dkms-install`, not installed in the VM). A mirror older than the ISO "downgrades" kernels on reinstall.
+- The hook keeps the early-load drop-in right across kernel changes (see `nvidia/TODO.md`).
+
+## Still unknown (needs the hardware)
+- Does the fix cure the 5080's corrupted picture? Which gamescope/driver versions does the PC have?
+- Does it help, harm or do nothing on Maxwell/Pascal/Volta cards? No older card available yet.
+- Real NVIDIA modules in the initramfs (VMs only had renamed fake modules); DKMS hook order on a PC with
+  `nvidia-open-dkms`.
@@ -0,0 +1,115 @@
+# TODO: NVIDIA fix for gaming mode
+
+Product code: branch `feature/nvidia-gaming-fix` in steamify-cachyos (github.com/theupriser/steamify-cachyos). These notes live here, in
+steamify-cachyos-dev, on purpose: they are not part of the product and must not be in its history. The branch only has
+a `.no-release-yet` marker pointing here.
+
+When the feature is released, move what is still useful to steamify-cachyos' `TECHNICAL.md` and delete this folder (`nvidia/`).
+Written 2026-10-01 so a new session can pick this up.
+
+## Problem
+A PC with an NVIDIA RTX 5080 (Blackwell, open kernel modules) shows a corrupted
+image when gaming mode (gamescope) starts. Cause not confirmed. Steamify had no
+NVIDIA handling at all. Likely: `nvidia-drm.modeset=1` / `nvidia-drm.fbdev=1`
+missing, or a gamescope/driver mismatch.
+
+## What is built (all on the product branch, pushed)
+- `lib/nvidia.sh`: part of the SteamOS conversion (`gaming_enable` / `gaming_disable`).
+  With an NVIDIA GPU + driver it adds `nvidia-drm.modeset=1 nvidia-drm.fbdev=1` to the
+  kernel command line (Limine, systemd-boot, GRUB; backup first) and, only when every
+  installed kernel has `nvidia_drm`, `/etc/mkinitcpio.conf.d/90-steamify-nvidia.conf`.
+  Rebuilds initramfs + boot entries, verifies the config and the generated entries
+  have the parameters (error otherwise). Undone on disable. Description: `TECHNICAL.md`.
+- `FEATURE_VERSION[gaming]=2.9.7` so installs that already have the conversion re-apply it.
+- Test hooks `NVIDIA_DRM_DIR`, `NVIDIA_MODULES_DIR`, `NVIDIA_LIMINE_CONF`,
+  `NVIDIA_SDBOOT_DIR`, `NVIDIA_GRUB_CFG`: only set by tests; defaults are the real paths.
+
+## Tested
+- `bash tests/nvidia-test.sh`: 22 checks against a fake RTX 5080 beside an iGPU, a stub
+  `modinfo`, temp boot loader files. All pass.
+- Real VMs (QEMU/KVM, Limine, systemd-boot, GRUB): enable, reboot, kernel command line
+  has the parameters; disable, reboot, original command line. All pass. Found and fixed:
+  Limine pasted an appended `+=` line as text into the cmdline; a kernel without the
+  NVIDIA modules made limine-mkinitcpio skip its boot entry.
+
+## Kernel changes (done, VM-tested 2026-10-01, Limine)
+The early-load drop-in is kept right by a pacman hook (`patches/steamify-nvidia-initramfs.{sh,hook}`,
+installed by `nvidia_enable`, `/etc/pacman.d/hooks/85-steamify-nvidia-initramfs.hook`). Real pacman
+transactions in a Limine VM: the hook runs between "Updating module dependencies" and "Updating
+linux initcpios" (real hook names there: 60-*-remove, 80-limine-efi-deploy, 90-limine-mkinitcpio-remove-post,
+90-mkinitcpio-install; DKMS would be 71-): the drop-in is present while every kernel has the modules, removed
+when the LTS kernel loses them, back when they return; initramfs nvidia files 1 -> 5 -> 1 -> 5 -> 1
+(1 = baseline, 4 modules added); no ERROR/skipping in any rebuild; cmdline kept the parameters after a reboot;
+disable removes hook, script and drop-in. Found on the way: an empty hook/script was installed when
+`patch_file` failed (now an error). Only tested with fake modules (renamed copies of a small module), on
+Limine; systemd-boot and GRUB use the same hook but were not run with pacman transactions.
+
+## Opt-out toggle (new request, 2026-10-01: not built yet)
+When a compatible NVIDIA card is listed there should be a menu option, on by default (opt-out), named
+"NVIDIA compatibility", as a sub-option of the SteamOS conversion, directly under "Boot into" (`boot`).
+Today the fix runs unconditionally inside `gaming_enable`/`gaming_disable`.
+- [ ] New component `nvidia` in `lib/menu.sh`: `COMPONENTS` right after `boot`, `PARENT[nvidia]=gaming`, `LABEL`,
+      `FEATURE_VERSION[nvidia]`, `component_available` -> `nvidia_available` (= `nvidia_present`); not in
+      `NO_PRESELECT` (ticked by default, `feature_new` ticks it for installs that already have the conversion).
+      `nvidia_status` from the system (hook + parameters present), `nvidia_enable`/`nvidia_disable` = the code now
+      called from `gaming_enable`/`gaming_disable` (remove those calls). Check `toggle_component` dependencies,
+      `feature_record_unticked`, the plan texts ("This will: ...").
+- [ ] The app: `ui/qml/Texts.qml` item texts, `AppState.qml`, the screen rows, `lib/backend.sh` if it lists ids;
+      `--defaults [--options <ids>]` (the Steam Machine ISO's installer pages: it names the ids) and `--skip`.
+- [ ] README rows/screenshot rule (a release that adds a menu row retakes `assets/screenshot-menu.png`), TECHNICAL.md,
+      CHANGELOG, `tests/nvidia-test.sh` (status/enable/disable through the component), VM menu test in
+      steamify-cachyos-dev (`share/vmtest/menu`, TESTPLAN.md row).
+- [x] What "compatible" means (decided 2026-10-01): RTX 20 series or newer, the same check as the VRAM booster
+      (`vram_nvidia_legacy_id`: a card on chwd's legacy lists `/var/lib/chwd/ids/nvidia-*.ids` is older; without the
+      lists every card counts). `nvidia_present` = such a card + `nvidia_drm` available; the toggle's `nvidia_available`
+      is the same function. Older cards (GTX 10/9 series) are left alone: no promise they work with gamescope.
+
+## Older cards and other research
+Decision 2026-10-01: support RTX 20 series or newer only, with the VRAM booster's check (done in `lib/nvidia.sh`).
+The user had hoped older GPUs would work too; not promised. Full notes with sources: **`nvidia/RESEARCH.md`**.
+Short version: open kernel modules need Turing+, Maxwell/Pascal/Volta need the proprietary driver (580 legacy branch);
+`nvidia-drm.modeset/fbdev` exist in both, so the fix applies to any card with `nvidia_drm`, but nothing found says gamescope
+works better or worse there (a GTX 1050 Ti has a known gamescope>3.16.16 start problem on HDMI, not ours). Cannot be promised:
+needs a test on a real older card. Design for the toggle: on by default for any card with `nvidia_drm`, the label says
+what it does, `tests/nvidia-hardware-test.sh` reports GPU and driver so results can be compared.
+
+## Still to do
+1. [ ] **On the NVIDIA PC** (only place the real fix can be judged; the full steps, how to read the results and the undo
+   are in `nvidia/INSTRUCTIONS.md`), from the steamify-cachyos checkout on that branch:
+   `bash tests/nvidia-hardware-test.sh check` (before), `... apply` (asks sudo), reboot,
+   `... check` (after: `nvidia_drm modeset/fbdev = Y`), boot gaming mode, then
+   `... visual` (was the picture clean?). Everything goes to `~/steamify-nvidia-report.txt`:
+   paste it into the session.
+2. [ ] If the parameters apply but the picture is still corrupted: look at the report's
+   gamescope/NVIDIA log lines and the versions (`nvidia-open`, `nvidia-utils`, `gamescope`,
+   `gamescope-session-cachyos`); the cause is then probably a gamescope/driver mismatch
+   for Blackwell, not this fix.
+3. [ ] Not covered by any test: the early-load drop-in with the REAL NVIDIA modules (the VMs only have renamed
+   fake ones), and systemd-boot/GRUB with a pacman transaction (only Limine was run).
+4. [ ] **Verify the RTX 20+ check on the PC** (decided 2026-10-01, only unit-tested with a fake chwd list; this laptop and the VMs
+   have no `/var/lib/chwd/ids`): `bash tests/nvidia-hardware-test.sh check` prints the NVIDIA PCI id, whether it is on a chwd legacy
+   list, and whether the lists exist at all. Expect for the RTX 5080: "supported", "not on a legacy list". If it says "no chwd
+   legacy lists found" the check cannot tell old from new there (every card then counts as supported, like the VRAM booster):
+   look at what `chwd`/`/var/lib/chwd/ids` provide on CachyOS and whether another source (the open-driver device list, the
+   `nvidia-open` package) would be better for both the VRAM booster and this fix. Also try `check` on any older NVIDIA card
+   that turns up (GTX 10/9 series): it must say "older than RTX 20 ... not applied" and `apply` must change nothing.
+5. [ ] Changelog: fill in the commit hashes of this feature's lines (next commit, per AGENTS.md).
+6. [ ] Build the opt-out toggle (section above) before release.
+7. [ ] When complete and tested: ask the user whether it may be released. On a yes delete
+   `.no-release-yet` (and this folder in the dev repo), open the PR into `release/2.9.7` (local branch only so
+   far, one commit `chore: Version 2.9.7`; push it then) and merge it. The user merges
+   `release/2.9.7` into `main`.
+
+## Working rules (steamify-cachyos `AGENTS.md` and this repo's `AGENTS.md`)
+- Never work on `main` or a `release/*` branch; no `Co-Authored-By` / "Generated with
+  Claude" lines in commits or PRs. No git identity on the dev laptop: use
+  `git -c user.name="Rick Peters" -c user.email="rickpeters@upriser.nl" commit`.
+- Don't push `release/2.9.7` before the PR (a push can start a dev ISO build).
+
+## If the VM tests need repeating (dev laptop, Ubuntu)
+- `scripts/vminstall.sh` (steamify-cachyos-dev) with `VM_BOOTLOADER=limine|systemd-boot|grub`,
+  `CI=1`, a short `VM_DIR` (`~/vms/...`: socket paths over 108 bytes fail) and `VM_SSH_KEY`.
+- This session's shell lacked the `kvm` group: run QEMU steps as `sg kvm -c "..."`.
+- `/tmp/vminstall-iso.lock` belongs to another user and there is no `bsdtar`: use a copy of
+  `vminstall.sh` with another lock path and a `bsdtar` shim (Python `pycdlib` in a venv).
+- Delete everything afterwards (`~/vms` incl. `pkg-cache`, the 3 GB ISO, keys).