docs: the first full run on the PC (328 pass, 18 min): TESTPLAN, TODO, AGENTS, wsl-build-host; vmsuite.sh writes its summary into its log

This commit is contained in:
theupriser committed 2026-09-29 18:05:10 +02:00
1 parent 37d65fc79d
commit a82053d6bc
5 files changed
+52 -11

No files matched your search

+19 -3
View File
@@ -9,7 +9,8 @@ The laptop is an arm64 Mac: x86_64 QEMU or containers there run emulated,
far too slow for Plasma/Calamares or an ISO build. The PC is the x86 host:
same workflows as `steam-machine-iso`, `vm-install` and `vmtest.sh`, with
the differences below. Found and tried on 2026-09-28 (ISO build, ISO VM with
a window, the loop-mount fallback); `vmtest.sh` itself not yet run here.
a window, the loop-mount fallback). 2026-09-29: the whole `vmtest.sh` ran here, 328 checks pass in
18 minutes (3 VMs at a time).
## Access
@@ -33,8 +34,12 @@ a window, the loop-mount fallback); `vmtest.sh` itself not yet run here.
- User is **root**: no `sudo`, `systemd-run` without `--user`, paths under
`/root/projects/` and `/root/vms/`. Units have no `$HOME`: set `HOME=/root`
for scripts started with `systemd-run` (vminstall.sh needs it).
- No udisks: `vminstall.sh` falls back to `losetup -P` + `mount` when
`udisksctl` is missing and it runs as root. Don't install udisks2 here.
- udisks: `vminstall.sh` uses `udisksctl` when it's there (udisks2 is
installed on this box now, and works) and falls back to `losetup -P` +
`mount` as root without it. The ISO's loop device is set up under a lock
(parallel installs collided: "Device or resource busy").
- Root is the host user: `vminstall.sh` then makes `theupriser` the guest
user (never root).
- `pkill -x qemu-system-x86_64` never matches (names over 15 characters):
`pkill -f '^[q]emu-system'`.
@@ -89,6 +94,17 @@ Windows desktop. SSH sessions and units lack the variables, so set them:
run.sh's default virgl (`virtio-vga-gl`, `gl=on`) renders through WSLg's
d3d12 Mesa on it, but QMP screenshots then give "no surface".
## Running the tests here
`scripts/vmtest.sh --screen` from `~/projects/steamify-cachyos-dev` (as root,
`HOME=/root`), then read `~/vms/last-test.txt`; follow it as the
`progress-report` skill says. Needs `~/vms/steamify-vm` (`VM_STEAMIFY=skip
VM_DIR=/root/vms/steamify-vm scripts/vminstall.sh --iso <CachyOS ISO>`, the
user keeps one in `/root/`) and `~/vms/bl-<loader>` (`vmbootloadertest.sh
<loader> --install`, or `vmtest.sh --install`). Memory: WSL sees 30 GB, fine
for 3 VMs at 4 GB (and the 8 GB install VMs); raise `memory=` in
`.wslconfig` before `MAX_PARALLEL=4` or `VM_MEM=8G`.
## Showing output in the user's WSL terminal
Their visible terminal is the `-bash` whose parent is WSL's `/init`
+8 -3
View File
@@ -31,10 +31,11 @@ wsl-build-host, progress-report).
- `scripts/vmtest.sh` runs the whole unattended test: boot loaders limine/systemd-boot/grub (`vmbootloadertest.sh`) and the
suites cli, menu, hw, installer, toggles (`vmsuite.sh`, checks in `share/vmtest/`). Each passed in its own run: limine 41,
systemd-boot 40, grub 40, cli 55, menu 36, hw 70, installer 4, toggles 42 checks.
- Written but not proven as a whole: the speed-ups (base image + qcow2 overlays, own ssh port per suite, package cache,
`VM_MEM=4G`, `MAX_PARALLEL=3`); a complete parallel run has never finished. First job on a new machine: one full
- 2026-09-29, on the PC (WSL2): the first complete parallel run, **328 pass, 0 fail in 18 minutes** (base image +
qcow2 overlays, own ssh port per job, package cache, `VM_MEM=4G`, `MAX_PARALLEL=3`). On a new machine: one full
`scripts/vmtest.sh --screen`, read `~/vms/last-test.txt`, fix what fails.
- Open work: `TODO.md` (WSL2 setup R1b, the GitHub Actions workflow, speed P1-P3).
- Open work: `TODO.md` (the B1 boot entry check, the ISO's hidden `@log` logs, WSL start with Windows R1d, the
GitHub Actions workflow, P3).
## Where the tests run
The tests run on the user's PC (Ryzen 9800X3D, 64 GB, **WSL2**), started from a laptop over ssh (the user's own skill, or
@@ -44,6 +45,10 @@ down, syncing from the laptop, the podman ISO build, VM windows through WSLg, ou
passthrough): `.claude/skills/wsl-build-host/SKILL.md`. `vminstall.sh` works there without udisks (losetup as root).
## Problems found in the Steamify ISO (fix in `steammachine-cachyos-live-iso`, PRs into `feat/steamify`)
- Fixed (PR #2): cachyos-installer's `bootctl install` runs in a chroot and writes no EFI boot entry, so systemd-boot
only booted through the disk's fallback path (in OVMF after 4-5 minutes of network boot, every boot).
`steamify-install` now registers it with efibootmgr from the live system.
- `/var/log` (`@log`) isn't mounted when `steamify-install` runs: its logs are hidden under the mount after boot.
- cachyos-installer leaves systemd-boot with `#timeout 3` and no default entry: a real machine waits in the menu for ever
(the test setup papers over it in `share/vminstall-post.sh`).
- It enables ufw, which drops ssh from the host (the test setup runs `ufw allow 22/tcp`).
+9 -1
View File
@@ -102,7 +102,7 @@ and started with `--fremont`. Automated: `scripts/vmtest.sh` (see the vm-install
| # | Step | Expect |
|---|---|---|
| B1 | first boot after the unattended install | boots by itself (systemd-boot: `timeout`/`default` set by the test setup), SSH answers |
| B1 | first boot after the unattended install | boots by itself through the loader's own EFI boot entry, no network boot first (systemd-boot: the entry comes from the ISO's `steamify-install` since PR #2; `timeout`/`default` set by the test setup), SSH answers |
| B2 | Steamify's state, as its own `*_status` functions | gaming, theme, glyphs, single, launcher, notify, poweroff, cec all on |
| B3 | OS name | `os-release` NAME/PRETTY_NAME are CachyOS's, `lsb-release` has no "with Steamify"; Limine has `TARGET_OS_NAME` |
| B4 | loader identity, firmware setup | the expected loader; `Boot into FW: supported` (BIOS item) |
@@ -125,6 +125,13 @@ and started with `--fremont`. Automated: `scripts/vmtest.sh` (see the vm-install
a release it shows the old version; start the branch's app by hand
(steam-machine-iso skill).
- B1: no check yet that the loader's own EFI entry exists, points to a real partition and was used
(`efibootmgr` `BootCurrent`): systemd-boot booted through the disk's fallback path unnoticed until
2026-09-29 (a 4-5 minute network boot per boot in OVMF). To add to `share/bootloader-test/`.
- ISO: `/var/log` is its own btrfs subvolume (`@log`), not mounted at `$ROOT/var/log` when
`steamify-install` runs: its logs (`steamify-install.log`, `steamify-bootentry.log`) end up hidden
under the mount after boot.
## Results log
| Date | Branch / version | Blocks | Result |
@@ -135,3 +142,4 @@ and started with `--fremont`. Automated: `scripts/vmtest.sh` (see the vm-install
| 2026-09-28 | `release/2.7.0` | H-Real | counted as passed: power-off fix, VRAM booster, LEDs, gamescope and Steam Machine support are unchanged since 2.6.0, which passed on the real Steam Machine. HDMI-CEC's change (enabling `cec-audio-control.socket`) passed with the fake TV in the VM; check the TV remote's volume keys on the real TV after updating (HDMI-CEC shows as update) |
| 2026-09-29 | `feature/vminstall-bootloader` (Steamify ISO 2026.09.28, `--fremont`) | B1-B8 | Limine 41 / systemd-boot 40 / GRUB 40 checks pass. Found on the way (test setup, not Steamify): ufw blocks ssh, systemd-boot has no timeout/default after cachyos-installer |
| 2026-09-28 | `release/2.8.0` (`feature/defaults-list`) | G1-G3, R1.1, R1.2 | all pass (VM `--fremont`, fake Steam Machine). Bundle shellcheck clean |
| 2026-09-29 | Steamify 2.9.0 (`steamify-cachyos` main), ISO `feat/steamify` with PR #2 (2026.09.29), dev `main` 5b867f8; on the PC (WSL2) | everything automated: B1-B8 (3 loaders), cli, menu, hw, installer, toggles (`scripts/vmtest.sh`, parallel, `MAX_PARALLEL=3`) | **328 pass, 0 fail** (toggles 42 +1 expected skip, hw 70, cli 55, menu 36, installer 4, Limine 41, systemd-boot 40, GRUB 40) in 18 minutes: the first complete parallel run. Found on the way: systemd-boot had no EFI boot entry (installer's `bootctl` in a chroot), fixed in the ISO (PR #2); `vminstall.sh` fixes for parallel installs and a root host user |
+15 -3
View File
@@ -52,9 +52,9 @@ R2.2 (first desktop login look), H-Real (real Steam Machine).
Where the time goes: boot loader VMs ~3 min each, every suite block restores a snapshot and reboots
(~1 min), and every block that applies Steamify downloads Steam and the rest again from the mirrors.
- [~] P1 (implemented, one clean full run still to prove). Package cache for the suite VM: `VM_CACHE` shared as 9p and bound over `/var/cache/pacman/pkg`
- [x] P1 (proven 2026-09-29: full run in 18 min on the PC). Package cache for the suite VM: `VM_CACHE` shared as 9p and bound over `/var/cache/pacman/pkg`
in `vmreset.sh`/the prelude (as vminstall-live.sh does); downloads only once
- [~] P2 (implemented: base image + overlays, own port per suite, MAX_PARALLEL=3, VM_MEM=4G; a whole parallel run has not finished yet). Parallel: each suite (and each boot loader) on its own copy of the VM (`cp --reflink` of the disk and vars),
- [x] P2 (base image + overlays, own port per suite, MAX_PARALLEL=3, VM_MEM=4G; first complete parallel run 2026-09-29: 328 pass, 18 min). Parallel: each suite (and each boot loader) on its own copy of the VM (`cp --reflink` of the disk and vars),
own `VM_PORT` and `VM_DIR`; `vmtest.sh` starts them together and waits; watch host CPU (16 cores, 6 vCPUs each)
- [ ] P3. Fewer resets: merge blocks that only read state (cli 10 + the menu R1.1)
@@ -63,15 +63,27 @@ Where the time goes: boot loader VMs ~3 min each, every suite block restores a s
- [ ] R1. `scripts/vmtest-remote.sh <ssh-host>` written; try it once: the PC needs the three repos, qemu + OVMF + screen,
KVM (on Windows: WSL2 with nested virtualization), the VMs in `~/vms` (`vmtest.sh --install` builds the boot loader
VMs, `vminstall.sh --iso <CachyOS ISO>` with `VM_STEAMIFY=skip` the plain one), and the Steamify ISO
- [ ] R1b. The PC is WSL2 (Windows). To check there: `/dev/kvm` exists (Windows 11, virtualization on in the BIOS,
- [x] R1b (done 2026-09-29: full vmtest.sh run there, 328 pass in 18 min; setup in the wsl-build-host skill). The PC is WSL2 (Windows). To check there: `/dev/kvm` exists (Windows 11, virtualization on in the BIOS,
nested virtualization on); `%UserProfile%\.wslconfig` `memory=` is raised (WSL2 defaults to half the RAM = 32 GB;
3-4 VMs at 4-8 GB want ~48 GB) and `processors=16`; keep `~/vms` and the repos on WSL's own ext4, not under `/mnt/c`;
the distro is Arch (root): `pacman -S qemu-full edk2-ovmf screen`; automated runs are headless with no Konsole
(`view_start` returns early; watch with `screen -r vmtest` or `tail -f ~/vms/test.log`), a VM for the user gets a
window through WSLg (see the wsl-build-host skill). Still to do: a first full `vmtest.sh` run there
- [ ] R1d. Start WSL with Windows (Task Scheduler at logon: `wsl.exe -d <distro> --exec /bin/sleep infinity`, hidden),
so the PC runs tests without an open Arch window
- [x] R1c. `vminstall.sh` without udisks (WSL): `losetup -P` + `mount` when `udisksctl` is missing and it runs as root
- [ ] R2. With 64 GB and 8c/16t: try `MAX_PARALLEL=4` and 8 GB VMs (`VM_MEM=8G`)
## Found 2026-09-29 (first full run on the PC)
- [ ] B1 check: the loader's own EFI boot entry exists, points to a real partition (not `HD(0,GPT,0000...)`)
and was used (`efibootmgr` `BootCurrent`); systemd-boot booted through the fallback path unnoticed
- [ ] ISO: mount `@log` at `$ROOT/var/log` before `steamify-install` writes its logs (they end up hidden
under the mount); PR into `feat/steamify`
- [x] ISO: systemd-boot's EFI boot entry (efibootmgr from the live system), PR #2 merged
- [x] vminstall.sh: a poweroff that drops ssh is no failure, wait only for its own VM, ISO loop device under
a lock, never root as the guest user
## Boot loader test (local)
- [x] `scripts/vmtest.sh [--install] [loader...]` runs all advertised loaders, `vmbootloadertest.sh` one
+1 -1
View File
@@ -95,6 +95,6 @@ done
vm_stop
} 2>&1 | tee "$log" | sed -u "s/^/[$suite] /" >> "$TEST_LOG"
fails="$(grep -c '^FAIL' "$log")"
echo "== $suite: $(grep -c '^PASS' "$log") passed, $fails failed, $(grep -c '^SKIP' "$log") skipped (log: $log)"
echo "== $suite: $(grep -c '^PASS' "$log") passed, $fails failed, $(grep -c '^SKIP' "$log") skipped (log: $log)" | tee -a "$log"
grep '^FAIL' "$log"
exit "$fails"