mirror of
https://github.com/theupriser/steamify-cachyos-dev.git
synced 2026-10-05 12:45:56 +02:00
docs: skills and AGENTS.md after 2026-09-29: the ISO release pipeline (new skill), boot entry check and install gotchas, monitor rules, freeing WSL disk, deleting merged branches
This commit is contained in:
1 parent
195518f0da
commit
0d0279bd20
7 files changed
+160
-3
No files matched your search
@@ -49,6 +49,16 @@ against a VM that gets reinstalled saves the live ISO's host key in `$VM_DIR/kno
|
||||
install then hangs at its first-boot ssh check ("REMOTE HOST IDENTIFICATION HAS CHANGED"). Never
|
||||
ssh to a VM with its `VM_DIR` settings while it installs; fix it with `ssh-keygen -R "[localhost]:<port>" -f $VM_DIR/known_hosts`.
|
||||
|
||||
Monitor rules learned on 2026-09-29: **one monitor per job, stop the old one** (`TaskStop`) when a new run
|
||||
starts, or every run is reported twice; a job that ends by itself (the ISO build on Gitea, `vmtest.sh`) is
|
||||
best watched with a loop that exits on its terminal states and prints only *changes* (a new step, a finished
|
||||
job, a FAIL, a new HTTP status), never on a percentage or a PASS count alone. For a Gitea Actions run, poll its
|
||||
public log (`.../actions/runs/<run>/jobs/<job>/logs`, see `steamify-iso-release`) and grep for the step names.
|
||||
A chain across systems (Steamify release -> ISO tag -> mirror sync -> Gitea build -> download link) is one
|
||||
monitor that says each link when it happens and ends on the real check (the download URL answers 200).
|
||||
When the tool that runs commands stops answering (auto-mode classifier "no verdict"), retry once, then say so
|
||||
and carry on with reading/local work: nothing running on the PC is affected.
|
||||
|
||||
## What the user sees on the PC
|
||||
|
||||
- **Their terminal**: stream the running job's log into it (the `showlog` unit, `wsl-build-host`
|
||||
|
||||
@@ -5,6 +5,9 @@ description: Use when working on the Steam Machine CachyOS ISO (repo steammachin
|
||||
|
||||
# Steam Machine ISO and Steamify's install-time mode
|
||||
|
||||
Releasing the ISO (GitHub tag, built and attached on the Gitea mirror, naming, runner and server settings) is
|
||||
the `steamify-iso-release` skill; this one is for building and testing by hand.
|
||||
|
||||
A CachyOS live ISO for the Steam Machine that installs Steamify's default
|
||||
setup during the install. Two repos:
|
||||
|
||||
|
||||
@@ -12,6 +12,12 @@ Rules:
|
||||
- Never touch unmerged branches; list them and report instead.
|
||||
- Never commit or merge to main.
|
||||
|
||||
Merging a feature/bugfix PR yourself (into a release branch, or `feat/steamify` in the ISO repo): delete the
|
||||
branch at once, remote and local (`gh pr merge N --merge --delete-branch`, then `git branch -D`, `git fetch
|
||||
--prune`); the user asked for this on 2026-09-29. Release branches are never deleted (kept as history next to
|
||||
their tags). Unmerged old branches (`feat/vram-booster`, `refactor/qml-screens`, `backup/*`) are reported, not
|
||||
touched. The GitHub-side `release delete <tag> --cleanup-tag` is how test ISO releases go.
|
||||
|
||||
Steps:
|
||||
1. `git fetch --all --tags --prune`
|
||||
2. `git checkout main && git merge --ff-only origin/main`
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: steamify-iso-release
|
||||
description: Use when releasing, debugging or changing how the Steamify ISO is built and published - the GitHub tag that names it, the Gitea mirror (git.upriser.nl) that builds and attaches the 3.2 GB ISO, the Steamify workflows that start it, and every trap met while setting it up (413, privileged runner, sudo dropping variables, mirror syncs deleting tags, upload-artifact@v3, GitHub 429). Also for the Gitea side of steamify-cachyos' own releases.
|
||||
---
|
||||
|
||||
# Steamify ISO release pipeline (set up 2026-09-29, first ISO out the same night)
|
||||
|
||||
Tested end to end: release `v2.9.3-dev.2026.09.29-2044` -> `steamify-cachyos-2.9.3-dev.2026.09.29-2044-x86_64.iso`
|
||||
on git.upriser.nl, GitHub's link to it answers 200.
|
||||
|
||||
## The flow (one tag names everything)
|
||||
|
||||
1. **GitHub, `iso-release.yml`** (ISO repo `theupriser/steammachine-cachyos-live-iso`, `workflow_dispatch` only):
|
||||
takes Steamify's newest release (`X.Y.Z`), the time **now in UTC** and makes an **annotated tag**
|
||||
`vX.Y.Z-[dev.]YYYY.MM.DD-HHMM` plus a GitHub release with the changelog notes and the *direct* download link
|
||||
on Gitea. `dev.` and a pre-release on `feat/steamify` (test build), none on `master` (a release).
|
||||
GitHub is the only place that reads a clock.
|
||||
2. **The mirror syncs** (git.upriser.nl is a pull mirror of GitHub, every 10 minutes): the tag arrives.
|
||||
3. **Gitea, `steamify-iso.yml`** (`on: push: tags: ['v*']`, skipped on GitHub by `github.server_url`): parses the
|
||||
tag, builds the ISO on the runner with exactly that Steamify (`STEAMIFY_VERSION`), the tag's time (label) and
|
||||
the tag without its `v` (`STEAMIFY_ISO_VERSION`: file name, boot menu, `/etc/steammachine-iso-build`), then
|
||||
attaches ISO + `.sha256` + `.sha1` + `.pkgs.txt` to the mirror's release for that tag
|
||||
(`akkuman/gitea-release-action`, the run's own token). Never overwrites a release that already has its ISO.
|
||||
4. **Trigger:** by hand (*Actions -> Steamify ISO release (tag) -> Run workflow*, or
|
||||
`gh workflow run iso-release.yml -R theupriser/steammachine-cachyos-live-iso --ref feat/steamify`), or by a
|
||||
new Steamify release: `steamify-cachyos`' `bundle.yml` step "Start the Steamify ISO's release" (secret
|
||||
`ISO_DISPATCH_TOKEN` there: fine-grained token, only the ISO repo, *Actions: read and write*; without it the
|
||||
step is skipped). A push does **not** release (a build is 20 minutes and 3.2 GB).
|
||||
5. Names, for tag `v2.9.3-dev.2026.09.29-2044`: file `steamify-cachyos-2.9.3-dev.2026.09.29-2044-x86_64.iso`,
|
||||
label `STEAMIFY_2_9_3_20260929_2044` (ISO 9660: 32 chars, `A-Z 0-9 _`). By hand, without a tag: `local`
|
||||
(`steamify-cachyos-local-x86_64.iso`, `STEAMIFY_<version>_LOCAL`). The build reads no clock.
|
||||
|
||||
Total from a Steamify release to the ISO on Gitea: about 30 minutes (release, tag, sync <=10 min, build 15-20).
|
||||
|
||||
## Why it is split like this (learned the hard way)
|
||||
|
||||
- **A release made only on the mirror disappears** at the next sync: the pull mirror removes tags GitHub doesn't
|
||||
have, and the release goes with it (one that has files may linger, tagless, and sorts oddly: delete it). A
|
||||
release on a tag that **comes from GitHub survives** every sync, files included (tested with a 0-byte asset).
|
||||
Hence: GitHub makes the tag, Gitea only attaches the file. You can't delete a tag on a mirror by hand either.
|
||||
- GitHub releases take **2 GB per file**; the ISO is 3.2 GB. So GitHub gets notes and a link, Gitea the file.
|
||||
- **Annotated tags**: a lightweight tag has only its commit's date, so tags on the same commit sort randomly
|
||||
on Gitea. UTC is fine (the release name says "UTC"; it is 2 h behind Dutch summer time, and nobody cared).
|
||||
- The GitHub release's link is built from the tag alone (`.../releases/download/<tag>/steamify-cachyos-<tag
|
||||
without v>-x86_64.iso`), so the Gitea file name must be that: check it with `curl -I -L` (the monitor in the
|
||||
session did: HTTP 200 = the whole chain is right).
|
||||
|
||||
## Gitea side: what the runner and the server need
|
||||
|
||||
- **Runner** (act_runner in Docker Compose, network `gitea_gitea`): `config.yaml` -> `container: privileged: true`
|
||||
(the workflow's `options: --privileged` is **ignored** otherwise; symptom: `mount: .../airootfs/proc: permission
|
||||
denied`, `failed to setup chroot`; the job log shows `Privileged:false`). ~20 GB free disk.
|
||||
- **Upload limit: `[repository.release] FILE_MAX_SIZE = 10240`** in `app.ini` (MB; default 2048), restart Gitea.
|
||||
`[attachment] MAX_SIZE` is for issues, **not** releases: symptom `status: 413` on `.../releases/<id>/assets`.
|
||||
The release is created *before* the upload, so a failed run leaves an empty release: delete it.
|
||||
- Logs of a public repo's run are readable without login: `.../actions/runs/<run>/jobs/<job>/logs` (job numbers
|
||||
count up; re-runs get a new job number). Good for a monitor; no `gh` for Gitea.
|
||||
- Mirror sync is set to 10 min by the user with their own token; "Synchronize Now" or the API
|
||||
(`POST /api/v1/repos/<repo>/mirror-sync`) for now. `GITEA_TOKEN` (secret in the ISO repo) would make GitHub
|
||||
trigger it; not needed.
|
||||
- **Build traps in the container:** `sudo mkarchiso` in `util-iso.sh` **drops environment variables**:
|
||||
it needs `sudo --preserve-env=STEAMIFY_ISO_VERSION,STEAMIFY_BUILD_STAMP` (the first Gitea ISO came out as
|
||||
`-local-`). Step names are evaluated **before** earlier steps set env: don't put `${{ env.X }}` in a name.
|
||||
Lots of `error: command failed to execute correctly` (pacman hooks wanting systemd) are harmless.
|
||||
The bash test of the naming must go through `sudo` too, not only call `profiledef.sh` directly.
|
||||
- Kernel: the ISO is built from the generic `x86_64` repo (must boot on any PC), which lags the `v3/v4/znver4`
|
||||
repos by a day or so: 7.2.7 on the ISO while installed systems get 7.2.8. Normal, not a bug: the installer
|
||||
switches to the CPU's optimised repo (znver4 on the 9800X3D) and the first update brings the newer kernel.
|
||||
|
||||
## Steamify's own workflow (`steamify-cachyos/.github/workflows/bundle.yml`), one file for both servers
|
||||
|
||||
- Build + check run everywhere. shellcheck through `ludeeus/action-shellcheck@2.0.0` (brings its own binary:
|
||||
act_runner's image has none; GitHub's has). The artifact: `actions/upload-artifact@v4` on GitHub,
|
||||
`christopherhx/gitea-upload-artifact@v4` elsewhere. **Never write `upload-artifact@v3`**: GitHub fails the
|
||||
whole workflow for merely mentioning a deprecated version, even in a skipped step.
|
||||
- Publishing: GitHub release (`gh`, tag + `latest`) only on GitHub; on Gitea the release is `akkuman/gitea-release-action`
|
||||
with a Gitea-API version check (`RELEASE_TOKEN` secret optional, else the run's token). Steps are guarded by
|
||||
`github.server_url == 'https://github.com'` (or `!=`). YAML check without PyYAML on the Mac:
|
||||
`ruby -ryaml -e 'YAML.load_file(...)'`.
|
||||
- Steamify releases still follow the AGENTS.md flow (`release/X.Y.Z`, `bugfix/`/`feature/` branch, PR into the
|
||||
release branch, merged and **deleted** by the agent, the release PR into `main` is the user's). Even a CI-only
|
||||
change is a release (2.9.2, 2.9.3).
|
||||
|
||||
## GitHub rate limit that started this (HTTP 429)
|
||||
|
||||
`raw.githubusercontent.com` refused Valve's CEC driver source after a day of test installs (each CEC turn-on
|
||||
downloaded it again; the limit is per address, ~1 per 5 minutes seen). Fixed in Steamify 2.9.1: the driver is
|
||||
cached in `/var/cache/steamify/cros-ec-cec-<sha256:12>.c` (named after its checksum: a new pin downloads once and
|
||||
deletes the old file; a damaged file is fetched again). The test VMs share the host's `~/vms/pkg-cache/steamify`
|
||||
(bound over `/var/cache/steamify` by `vmsuite.sh`, `vmbootloadertest.sh`, `vminstall-live.sh`); hw block 50 tests
|
||||
the cache offline (`file://`). The source repo `evlaV/linux-integration` is an unofficial mirror without releases:
|
||||
if it ever vanishes, ship the file with Steamify.
|
||||
|
||||
## Checklist for a release
|
||||
|
||||
1. Steamify released? (`gh release list -R theupriser/steamify-cachyos`). The ISO gets the newest.
|
||||
2. `gh workflow run iso-release.yml ... --ref feat/steamify` (test) or `--ref master` (release; master must have
|
||||
the workflows: it doesn't yet, `feat/steamify` is the ISO repo's working branch).
|
||||
3. Watch: tag on Gitea (`.../api/v1/repos/theupriser/steamify-cachyos-live-iso/tags`), the run's log (above),
|
||||
then `curl -I -L` the GitHub link. Old test releases: delete on GitHub (`gh release delete <tag>
|
||||
--cleanup-tag -y`); the sync removes the tag on Gitea; delete a leftover tagless Gitea release by hand.
|
||||
@@ -138,6 +138,14 @@ lose the executable bit (`chmod --reference` or call scripts with `bash`); a sui
|
||||
be a `.host.sh` block; ssh has no Plasma session, the prelude exports one; a one-off network failure can fail
|
||||
an install step (LED driver from the AUR), rerun the block before suspecting Steamify.
|
||||
|
||||
Gotchas found on 2026-09-29: never ssh to a VM with its `VM_DIR` settings while it installs (a leftover loop, also
|
||||
one started from a laptop command that was stopped, saves the *live ISO's* host key in `$VM_DIR/known_hosts` and
|
||||
the install then hangs at its first-boot ssh check: `ssh-keygen -R "[localhost]:<port>" -f $VM_DIR/known_hosts`);
|
||||
`vminstall.sh` now never uses root as the guest user (WSL runs as root: `theupriser`), takes the ISO's loop
|
||||
device under a lock (parallel installs collided), tolerates the poweroff dropping ssh, waits only for its own
|
||||
qemu and stops at once when the live system reports `== failed: ...`. A failed install used to look like
|
||||
"still installing" for 90 minutes: `vmprogress.sh` shows `INSTALL FAILED: <why>`.
|
||||
|
||||
Following a run for the user (a table per VM with a bar and a total, only on a change; their terminal and
|
||||
the VM's screen): the `progress-report` skill, with `scripts/vmprogress.sh` as the monitor.
|
||||
|
||||
@@ -149,7 +157,16 @@ loader has no test). One VM per loader in `~/vms/bl-<loader>` (`--install` build
|
||||
Steamify ISO, ~8 min each, ~3 min per test), headless with a Konsole on the logs (`--window` shows the VMs),
|
||||
one summary line per loader, `~/vms/bl-<loader>.test.log`, exit status = failed checks. Checks live in
|
||||
`share/bootloader-test/*.sh` (one PASS/FAIL line each); the driver is `scripts/vmbootloadertest.sh`.
|
||||
Last result: Limine 41, systemd-boot 40, GRUB 40 checks, 0 failed.
|
||||
Last result (2026-09-29, ISO with Steamify 2.9.1): Limine 43, systemd-boot 42, GRUB 42 checks, 0 failed.
|
||||
The boot entry check (B1, `boot-check.sh`, after every reboot): the system booted through the loader's **own**
|
||||
EFI entry (`BootCurrent`, not `auto_created_boot_option`, a real partition GUID not `HD(0,GPT,0000...)`). It
|
||||
found that the installer never registered systemd-boot (bootctl skips the EFI variables in a chroot): OVMF then
|
||||
tried PXE/HTTP boot/EFI shell first, 4-5 minutes per boot, or "no bootable device". The ISO's `steamify-install`
|
||||
now runs `efibootmgr` from the live system (a real partition, first in the order) and mounts the new system's
|
||||
`/var/log` (`@log` subvolume) first so its logs (`steamify-install.log`, `steamify-bootentry.log`) survive.
|
||||
To read a VM's boot entries without booting it: copy `vars.fd`, `virt-fw-vars -i <copy> --print` (package
|
||||
`virt-firmware`). The power-off check accepts "loaded" or "tried at boot: No such device" (journal, not dmesg:
|
||||
the VM has no AMD GPIO controller, the module refuses on purpose).
|
||||
What only shows up per loader: Limine keeps its images under `/boot/<machine-id>/<kernel>/` and copies one
|
||||
only when its content changed (same version = untouched), and `remember_last_entry: yes` overrides
|
||||
`default_entry` (the test turns it off and restores it); GRUB's other kernel is picked with
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: wsl-build-host
|
||||
description: Use when building or testing the Steam Machine ISO, or running the VM tests (vmtest.sh), on the user's Windows PC - Arch Linux in WSL2 (`ssh wsl`), with podman, QEMU + KVM and WSLg windows on the Windows desktop. Covers access, syncing working trees from the laptop, the podman ISO build as root, VM windows vs headless, showing output in the user's WSL terminal, and the WSL pitfalls (closing the window kills it, no udisks, no GPU passthrough).
|
||||
description: Use when building or testing the Steam Machine ISO, or running the VM tests (vmtest.sh), on the user's Windows PC - Arch Linux in WSL2 (`ssh wsl`), with podman, QEMU + KVM and WSLg windows on the Windows desktop. Covers access, syncing working trees from the laptop, the podman ISO build as root, VM windows vs headless, showing output in the user's WSL terminal, freeing disk space, and the WSL pitfalls (closing the window kills it, no GPU passthrough).
|
||||
---
|
||||
|
||||
# WSL build host (Arch on WSL2, the Windows PC)
|
||||
@@ -96,6 +96,18 @@ Windows desktop. SSH sessions and units lack the variables, so set them:
|
||||
run.sh's default virgl (`virtio-vga-gl`, `gl=on`) renders through WSLg's
|
||||
d3d12 Mesa on it, but QMP screenshots then give "no surface".
|
||||
|
||||
## Freeing disk space
|
||||
|
||||
Everything here is re-creatable (VMs, `~/vms/pkg-cache`, `~/projects/iso-cache`, `build/`, the podman image):
|
||||
`rm -rf` them when the VMs aren't running (`pgrep -f "^[q]emu"`), `podman system prune -a -f`,
|
||||
`pacman -Scc --noconfirm`, `fstrim -av`. That freed 68 GB inside WSL (81 -> 13 GB), but Windows only gets it
|
||||
back when the vhdx is sparse: in an admin PowerShell `wsl --shutdown`, `wsl --manage archlinux --set-sparse
|
||||
true` (seconds), reopen the Arch window, `fstrim -av` again; else `compact vdisk` in diskpart. A rebuild costs
|
||||
the caches: a full `vmtest.sh --install` (three loader VMs ~12 min in parallel, the suite VM ~12 min from a
|
||||
CachyOS ISO the user keeps in `/root/`), and an ISO build ~6-8 min. Don't `wsl --shutdown` while a job runs.
|
||||
Hand-built ISOs are called `steamify-cachyos-local-x86_64.iso`; releases are built on the Gitea mirror
|
||||
(`steamify-iso-release` skill). Windows starting WSL at logon is the user's own business, not Steamify's.
|
||||
|
||||
## Running the tests here
|
||||
|
||||
`scripts/vmtest.sh --screen` from `~/projects/steamify-cachyos-dev` (as root,
|
||||
|
||||
@@ -4,7 +4,7 @@ Test tooling for Steamify (`../steamify-cachyos`) and the Steam Machine ISO (`..
|
||||
VM scripts (`scripts/`, `run.sh`), the automated test (`scripts/vmtest.sh`, checks in `share/vmtest/` and
|
||||
`share/bootloader-test/`), `TESTPLAN.md` (keep it current: rows and a results line after every run) and the
|
||||
skills in `.claude/skills/` (vm-install, cachyos-vm-testing, steam-machine-testing, steam-machine-iso,
|
||||
wsl-build-host, progress-report).
|
||||
wsl-build-host, progress-report, steamify-iso-release, steamify-branch-cleanup).
|
||||
|
||||
## Running the tests (do this, don't re-derive it)
|
||||
- Everything: `scripts/vmtest.sh --screen` (detached `screen` session `vmtest`), then read only
|
||||
@@ -36,6 +36,14 @@ wsl-build-host, progress-report).
|
||||
`scripts/vmtest.sh --screen`, read `~/vms/last-test.txt`, fix what fails.
|
||||
- Open work: `TODO.md` (the CEC driver download, the GitHub Actions workflow, P3).
|
||||
|
||||
## Releases (2026-09-29; details in the `steamify-iso-release` skill)
|
||||
- Steamify: `release/X.Y.Z` -> PR into `main` (the user's) -> GitHub release, and the same workflow on the Gitea
|
||||
mirror. A new Steamify release starts the ISO's release (`ISO_DISPATCH_TOKEN`); a CEC driver download is cached.
|
||||
- The ISO: GitHub makes the tag `vX.Y.Z-[dev.]YYYY.MM.DD-HHMM` (UTC) and a release with the notes; the Gitea
|
||||
mirror (git.upriser.nl, pull mirror, 10 min) builds on that tag and attaches the 3.2 GB ISO, named after the tag.
|
||||
A release that exists only on the mirror is deleted by its next sync: the tag must come from GitHub.
|
||||
- Merged feature/bugfix branches are deleted at once (`gh pr merge --delete-branch`). Push every commit.
|
||||
|
||||
## Where the tests run
|
||||
The tests run on the user's PC (Ryzen 9800X3D, 64 GB, **WSL2**), started from a laptop over ssh (the user's own skill, or
|
||||
`scripts/vmtest-remote.sh <ssh-host> [args]`: pulls the pushed branch there, starts `vmtest.sh --screen`, waits for
|
||||
|
||||
Reference in new issue
Block a user