188 lines
21 KiB
Markdown
188 lines
21 KiB
Markdown
# QA: Updates part 3 (restart, helper, rollback)
|
|
|
|
## Automated Go (`task test`, package `test`, black-box, no live systemd, no real SMTP, no network)
|
|
|
|
Files: `test/update_restart_http_test.go` (HTTP), `test/update_restart_usecase_test.go` (request and status use cases),
|
|
`test/update_helper_test.go` (helper logic over fakes, including rollback and hardening), `test/update_helper_install_test.go`
|
|
(enable and disable over fakes), `test/update_helper_files_test.go` (real file adapters in `t.TempDir()`),
|
|
`test/update_dispatch_test.go` (built binary).
|
|
|
|
Fakes with unique names: `fakeRestartRequestStore`, `fakeResultStore`, `fakeRunningInfoStore` (scripted per poll: absent, corrupt, stale,
|
|
new), `fakeHelperFS`, `fakeServiceControl` (records restart calls with the clock instant; can fail), `fakeHealthChecker` (scripted per poll),
|
|
`fakeClock` (advances without sleeping), `fakeUnitWriter` (records unit contents, can fail the Nth write). Reuse `fakeJobStore`,
|
|
`fakeHelperDetector`, `fakeUpdateStore`, `mockUserRepo` and the throwaway ed25519 signer from slices 1 and 2. Never read the committed `release.pub`.
|
|
|
|
`fakeHelperFS`: an in-memory tree of files, directories and symlinks with owners and modes; `lstat`, no-follow open, create-exclusive, copy,
|
|
rename, remove and a fresh private temp directory are the only operations; every call is recorded in order with its path; an operation that
|
|
would traverse a symlink without the no-follow flag follows it (so a test fails if the code forgets to refuse); it can fail the Nth
|
|
call of an operation, and it has a hook `after(op, n, mutate)` that mutates the tree after a chosen call (used to swap the original
|
|
staged file after the copy step). Tests assert the paths the helper touched stay inside the data directory, the binary directory and
|
|
its own temp directory.
|
|
|
|
HTTP setup: `validator.InitValidator()`; `jwtauth.New("HS256", []byte("test-secret"), nil)`; a fresh `chi.NewRouter()` at `/v1/system`
|
|
mirroring `NewSystemRouter` (Verifier, Authenticator, RequireCurrentToken, RequireVerifiedUser; the restart route also RequireOwner);
|
|
cookies as in `test/stale_token_invalidation_http_test.go`. Marker, result and job files go in `t.TempDir()` through the real JSON adapters,
|
|
plus the fakes where noted. Helper time is driven by `fakeClock`: no `time.Sleep`, so the 60 s window runs instantly.
|
|
|
|
### Tests the coder must update (they exist from slices 1 and 2 and will break)
|
|
- Slice 1 tests that assert GET /v1/system/update has exactly 12 keys: now 13, with `finished_at` (null when no result file).
|
|
- Slice 2 tests and Vitest specs that assert the Restart button in "Ready to install" is disabled and has no handler, that no restart
|
|
endpoint, dialog or overlay exists, and that plain Restart is not rendered: replace them with the enabled behaviour below.
|
|
- Slice 2 "Pending states for slice 3" visual entries: they become real setups.
|
|
- Any slice 2 test that builds the status use case with a result store must accept the new `ResultStore` dependency.
|
|
|
|
### HTTP and use case (Go-testable)
|
|
1. Restart, update: ready job, can_apply on. POST with no body and POST `{"kind":"update"}` give 202 with state "restarting" and
|
|
`started_at` equal to the marker's `requested_at`; the marker has kind "update", the tag, requested_by equal to the owner id; the result file
|
|
was deleted first; no other file changed; GET as an admin also reads "restarting".
|
|
2. Plain restart: idle job. No body and `{"kind":"restart"}` give 202, marker kind "restart", empty tag. Also after a failed job and after a
|
|
rolled-back result.
|
|
3. Kind mismatch: `{"kind":"update"}` with no ready job (idle, failed) and `{"kind":"restart"}` with a ready job give 409; no marker, result and job untouched.
|
|
"Restart site never installs": owner A's body `{"kind":"restart"}` after the fake job store turned "ready" gives 409, no marker, staged files untouched.
|
|
4. Bad body: `{"kind":"upgrade"}`, `{"kind":""}`, `{"kind":7}`, truncated JSON, `not json` give 400 plain text, no marker. The struct uses
|
|
`validate:"omitempty,oneof=update restart"` and goes through `validator.Validate.Struct`; an empty body and `{}` are accepted.
|
|
5. Guards: admin 403 with `apperr.OwnerRequiredErrorMessage+"\n"`; no cookie 401; stale token 401; unverified 403; none writes a marker or deletes the result.
|
|
Order: an admin with a bad body gets 403, an owner with a bad body and can_apply off gets 400, can_apply off and a kind mismatch gives 409.
|
|
6. 409 table: can_apply off, downloading, verifying, marker pending, result "running": no second marker, existing files untouched.
|
|
7. Concurrency (`-race`): two simultaneous POSTs give one 202, one 409, one marker.
|
|
8. 500: a directory where the marker goes, and a read-only temp directory (`t.Skip` that one on Windows): 500, plain text without the path, no marker,
|
|
job record unchanged.
|
|
9. State derivation, one subtest per row of the feature table. Each row calls GET twice: the two bodies are byte-identical, `state`, `error` and
|
|
`finished_at` match, and a snapshot of every file in the data directory (names, bytes, mtimes) is unchanged by the GETs (GET has no side effects).
|
|
The 5-minute boundary uses `fakeClock`. Assert exactly the 12 slice 1 keys plus `finished_at`.
|
|
10. The result lives until the next job: a "rolled_back" result, then POST download and a job reaching "ready": the result is gone,
|
|
state "ready", error null, `finished_at` null. POST restart also deletes it before writing the marker.
|
|
11. After success: result "succeeded" for the running tag gives `available` false, `current` the new version, `finished_at` set.
|
|
12. Server startup file: with the real running-info adapter in `t.TempDir()` write twice; the file has `version`, `pid` and `started_at`, the second
|
|
write replaces the first, it is written by a temp file plus rename, and a write into a missing directory returns an error that the startup code logs
|
|
(use case or startup function returns it; it does not panic). The built binary (item 15) writes it before it answers `/healthz`.
|
|
13. `/healthz` is unchanged: body "ok", no `X-RenewCMS-Version` header, public.
|
|
14. A new download after "rolled_back" gives 202 and later a "ready" with error null.
|
|
15. Dispatcher: build with `go build -tags nofrontend -ldflags "-X main.version=v9.9.9" ./cmd/server` into `t.TempDir()`; with an empty environment
|
|
`--version` prints `v9.9.9`; an unknown subcommand exits non-zero with usage and does not listen; `apply-update` and `enable-browser-updates`
|
|
run without `.env` and create no `data/` (on a non-Linux machine the latter exits non-zero with the Linux/systemd message, which covers the Windows refusal).
|
|
|
|
### Helper logic over fakes (Go-testable, no systemd)
|
|
Fixture: fake FS with the binary `/opt/renewcms/renewcms` ("old"), staged `updates/v2.3.0-beta/` binary, `checksums.txt` and `checksums.txt.sig`
|
|
signed with a throwaway key injected through the verifier constructor, a marker, running version v2.2.0-beta, `update-running.json` with pid 100 and
|
|
started_at T0.
|
|
|
|
16. Plain restart: the marker is opened no-follow and deleted first, result "running" then "succeeded", one service restart, no write to the binary.
|
|
Success needs `/healthz` 200 AND a running file that is NEW.
|
|
17. Stale running info, one subtest per row: old pid and old started_at; target version but old pid and old started_at; absent; corrupt; old version with a new pid
|
|
(for an update). Health answers 200 throughout. The helper keeps polling and is not declared succeeded or rolled back until the file turns new or the
|
|
fake clock passes 60 s. Also: a file whose started_at is later than the restart instant counts as new even with the same pid.
|
|
18. Update happy path, asserting the ORDER of fake calls: no-follow read and delete of the marker, "running", fresh 0700 temp directory, no-follow copy of the
|
|
three staged files into it, signature check on the copy, SHA-256 of the copied binary, copy of the current binary to `.previous`, temp file in the
|
|
binary's directory written from the verified copy, re-hash of that temp file, atomic rename, mode 0755, record running info, restart, polls,
|
|
"succeeded", removal of the temp directory and `updates/<tag>/`. Final FS: binary equals the new bytes, `.previous` equals the old bytes.
|
|
Success is declared on the first matching poll, not at 60 s.
|
|
19. Re-verification, one subtest each: tampered binary, tampered checksums.txt, other-key signature, missing signature, no line for the asset, missing
|
|
binary. Each: binary unchanged, no `.previous`, zero restarts, result "failed", staged dir and temp dir removed, reason in the injected logger only.
|
|
20. Symlinks, one subtest each: marker; `updates`; `updates/<tag>`; the binary; checksums.txt; checksums.txt.sig; a link at the result path; a link at a
|
|
predictable temp file name. Each: the FS records no read, write, copy or removal through the link (the targets, placed outside the allowed
|
|
directories with sentinel bytes, are unchanged), binary untouched, no `.previous`, zero restarts, result "failed", the log names the link path.
|
|
The symlinked marker case: the link itself is unlinked and the target is intact.
|
|
21. TOCTOU: with the `after` hook, replace the original staged binary with other bytes immediately after the copy step; the installed bytes equal the
|
|
verified copy's bytes. Repeat with the swap after the signature check, after the hash check, and after the `.previous` copy. A swap of the temp file
|
|
in the binary's directory between write and rename is caught by the re-hash: result "failed", binary untouched.
|
|
22. Result and temp writes: create-exclusive and no-follow, then rename; a pre-planted symlink at the result path or a temp name leaves the target intact.
|
|
23. Hostile marker tags (traversal, traversal with valid prefix, empty, older version, malformed): result "failed", zero restarts, every touched path
|
|
under the allowed directories.
|
|
24. Rollback table: health never answers (60 s of fake clock, about 60 polls at 1 s), healthy but running file stays old, other version in the running file,
|
|
restart error. Each: binary restored to "old" through an atomic rename, second restart issued, polled until a NEW running file holds the old version,
|
|
result "rolled_back" with from_version and tag, temp directory and staged dir removed.
|
|
25. Swap failures: `.previous` copy fails (binary untouched, no restart, "failed"); atomic install fails (old binary and `.previous` intact, "failed");
|
|
restore fails ("failed", error returned, loud log); old version never returns after rollback ("failed", not "rolled_back").
|
|
26. Result always written: a panic in the fake service control still leaves outcome "failed" with `finished_at`; the "running" record exists before the
|
|
first service call; writes are atomic.
|
|
27. Timestamps come from `fakeClock`; finished is not before started.
|
|
28. Enable, over fake FS, unit writer and service control: refuses non-Linux, no systemd, not root, no unit and no flags, no data directory (exact
|
|
messages); on success writes both units containing the service, binary and data directory from the parsed unit, the `PathExists=` marker line and
|
|
the port flag (`--port` over `PORT` from the env file over 8080); the units never contain `JWT_SECRET` or any value from the env file (seed the
|
|
fixture env file with `JWT_SECRET=sentinel-secret` and search both units); flags override the unit; the status file is written LAST; running it
|
|
twice gives identical files and one unit pair; each failing step leaves no status file and removes the first unit.
|
|
29. Disable: removes units and the status file, keeps marker, result, staged files and backups; idempotent; a no-op where it was never enabled;
|
|
`can_apply` reads false afterwards through the real file detector.
|
|
30. Real adapters on `t.TempDir()`: the JSON marker, result, status and running-info stores round trip; the atomic write replaces an existing file.
|
|
The Linux-only no-follow adapter has a test file with `//go:build linux` (it runs on Linux CI machines and the user's WSL2, and is skipped on Windows).
|
|
31. Review items, not Go tests: the app (handlers, use cases, `api/`) execs no `systemctl` and writes only inside the data directory;
|
|
`internal/domain/update` imports only the stdlib; use cases import no `os/exec`, `net/http`, `gorm` or adapter; the handler imports no adapter;
|
|
the real filesystem adapter is the only code that calls `syscall.O_NOFOLLOW`; no secret in the repo; no new env var; `go vet -tags nofrontend`
|
|
and `gofmt` clean; no comments.
|
|
|
|
## Automated SPA (Vitest + Vue Test Utils, `task test:web`)
|
|
|
|
Mock `useApi.request` for `GET /v1/system/update`, `POST .../restart` and `POST .../download`; mock `fetch` for `GET /healthz`. Fake timers drive the
|
|
1 s polls and the 60 s switch. Stub `window.location.reload` and use a fresh `sessionStorage` per test. No test freezes the system date and none asserts a
|
|
relative date or "09:51". Sample data: running `v2.2.0-beta`, latest `v2.3.0-beta`, `started_at` and `finished_at` as plain strings.
|
|
|
|
One test per state:
|
|
1. Restart button: ready gives primary enabled; idle gives secondary enabled; downloading, verifying, restarting give disabled; can_apply false and admin give none.
|
|
2. Install dialog: exact title, text, buttons; focus on the first button; Escape and "Restart later" close it; focus returns to Restart; no request until
|
|
"Restart now"; the confirm is neutral; the request body is `{"kind":"update"}`.
|
|
3. Plain restart dialog: exact copy, "Cancel" / "Restart site", same behaviour; the request body is `{"kind":"restart"}`.
|
|
4. Restart request failed: 500 and `NetworkError`, in both dialogs: dialog stays, the exact error text, confirm enabled again, no overlay, nothing in sessionStorage. A 400 gives the same.
|
|
5. After 202, sessionStorage holds one entry with the kind and the response's `started_at`; no localStorage write.
|
|
6. Restarting to install: overlay `role="status"`, title and text, announced once; stays mounted when `fetch` rejects; the offline banner state is ignored; polls
|
|
/healthz and GET every second.
|
|
7. Plain restart: restarting: "Restarting your site" copy.
|
|
8. Taking longer: at 60 s "Still restarting", the text, "Reload page" present; pressing it calls reload; the overlay stays.
|
|
9. Overlay end: reload once when /healthz is ok and GET gives a non-"restarting" state with `finished_at` later than the stored `started_at`; no reload on
|
|
"restarting", on `finished_at` null, on `finished_at` not later, on a rejected GET, or without a GET answer; reload is not called twice.
|
|
10. Update done (entry present, state "succeeded", later `finished_at`): entry cleared, focus on `#updates-h`, success toast "Update installed" / "RenewCMS is now on version 2.3.0-beta.", card text and the "Read the release notes" link.
|
|
11. Rolled back (error "rolled_back" and "install"): entry cleared, focus on `#updates-h`, sticky error toast, card copy, "Try again" posts download.
|
|
12. Plain restart done (state "idle", later `finished_at`, kind "restart"): toast "Site restarted" / "Your site is running again.", focus, Up to date card, entry cleared.
|
|
13. No entry, or an entry not earlier than `finished_at`: for each of succeeded, rolled_back, install and plain-restart-idle results the card renders with no toast, no
|
|
focus move and no sessionStorage write.
|
|
14. Another owner's tab: no entry, GET "restarting" then "succeeded": overlay then card, no toast, no focus move. An owner tab with an entry still gets the toast. An admin sees the
|
|
admin view, no overlay and no poll.
|
|
15. Stale entry: `finished_at` null or earlier than the entry's `started_at` gives no toast and keeps the entry; a new restart overwrites it.
|
|
16. 409: reload downloading gives the info toast and progress; reload restarting gives the toast and overlay; reload can_apply false gives Not prepared and no toast; reload "ready"
|
|
or "idle" after a kind mismatch gives that card, no toast, nothing in sessionStorage.
|
|
17. 403: error toast and the admin view; 401 shows no local copy (the global handler is mocked).
|
|
18. Home: no banner or poll while "restarting".
|
|
19. Regression: slice 2's disabled-Restart assertions are replaced; Not prepared and the admin view still have no Restart.
|
|
|
|
## Visual (`task visual:check -- settings`; desktop and mobile, light and dark; every capture within 10 %)
|
|
|
|
`web/visual/states.ts` setups are added later by the coder, with the board's sample data (v2.2.0-beta, v2.3.0-beta). Exact values come from
|
|
`docs/design/source/AppFrame.dc.html` and the "Settings" group of the Screens board; describe layout as drawn (open the frame markup for the dialog, the overlay card
|
|
and its 440 px width before judging).
|
|
- [ ] Restart: confirm (install dialog)
|
|
- [ ] Plain restart: confirm
|
|
- [ ] Restart: request failed
|
|
- [ ] Restarting to install
|
|
- [ ] Plain restart: restarting
|
|
- [ ] Update failed, rolled back (the toast only if the frame draws it)
|
|
- [ ] `skip: 'clock-sensitive'`: Update done and Plain restart: done (meta "Updated today at 09:52"), Restarting, taking longer (meta "Started at 09:51 · 1 minute ago"); their copy is covered by Vitest
|
|
- [ ] Slice 2 pending states are now real: re-run the other Settings states and confirm Ready to install with an enabled Restart is within budget
|
|
- [ ] If a capture exceeds 10 %, stop and report; do not widen the budget or add an exception without the user's approval
|
|
- [ ] Not covered by screenshots: keyboard and focus order in both dialogs, the ring on `#updates-h` after the result states, the one-time overlay announcement, the motion of the bar, the Offline banner being ignored
|
|
|
|
## Manual only (real systemd Linux: WSL2 with systemd enabled, or a VPS): CANNOT be verified automatically
|
|
The user will run these. The helper's decision logic, ordering, symlink refusal, TOCTOU defence and rollback are unit-tested over fakes (items 16 to 27), but none of the following is covered:
|
|
- [ ] On Windows or macOS `renewcms enable-browser-updates` refuses with the clear message.
|
|
- [ ] On a VPS following docs/deployment.md (service user `renewcms`, `ProtectSystem=strict`, only `/var/lib/renewcms` writable): `sudo renewcms enable-browser-updates` reads the real unit, writes
|
|
`renewcms-update.path` and `renewcms-update.service`, enables the path unit (`systemctl status` active) and writes `update-helper.json`; a second run changes nothing.
|
|
- [ ] The Settings page shows the Update button (can_apply true) after a reload.
|
|
- [ ] The path unit fires when the sandboxed app writes the marker, and the app (as `renewcms`) can write the marker and cannot write the binary, the units or anything outside the data directory.
|
|
- [ ] The server writes `update-running.json` at startup (`cat` it: version, pid, started_at) and a restart replaces it.
|
|
- [ ] Real update from a real signed release (real `release.pub` committed, `task release` signing): Update, Ready, Restart, overlay, reload on the new version, toast on the tab that started it,
|
|
`.previous` next to the binary, `renewcms --version` equals the new tag, `journalctl -u renewcms-update` shows the helper log.
|
|
- [ ] Real rollback: stage a signed release whose binary prints the right `--version` but crashes at boot; the old binary is restored within about 60 s, the card reads "The update didn't install", the toast stays.
|
|
- [ ] Plain restart: overlay, about 10 s of downtime, reload with "Site restarted".
|
|
- [ ] The real no-follow adapter on Linux: as the service user, replace `updates/<tag>` with a symlink to `/etc`, replace the staged binary with a symlink, replace the marker with a symlink, and swap the staged file during
|
|
an update (a loop that rewrites it); the helper refuses or installs only verified bytes, and nothing outside the data directory is read or changed.
|
|
- [ ] Planted symlink at `update-result.json` pointing at a root-only file: the helper does not write through it.
|
|
- [ ] Binary directory ownership, SELinux and AppArmor do not block the atomic rename, and the temp file is on the same filesystem as the binary; the helper's private temp directory is root-owned 0700 and is removed.
|
|
- [ ] A hand-written marker with another tag, as the service user: refused, binary untouched.
|
|
- [ ] `sudo renewcms disable-browser-updates` removes the units and status file; the card returns to "Not prepared"; nothing else is deleted.
|
|
- [ ] A reverse proxy (nginx or Caddy) returns 502 during the restart and the overlay survives it; `/healthz` is not cached.
|
|
- [ ] Two owners in two browsers during a restart: both show the overlay; only the tab that started it gets the toast and focus.
|
|
- [ ] Embedded SPA build (no `-tags nofrontend`) serves `/settings` with the new dialogs and overlay.
|
|
- [ ] Journal and files contain no secret, and technical reasons are only in the journal.
|
|
|
|
## Final note
|
|
Not verifiable automatically: the whole privileged path on a real systemd Linux machine (generated units, path-unit trigger from the sandboxed app, real service restart, atomic swap on a real filesystem, real rollback, O_NOFOLLOW behaviour, permissions, SELinux or AppArmor, reverse-proxy behaviour). It needs the manual test on WSL2 with systemd or a VPS, which the user will do.
|