fsylvain/RenewCMS
1
1 Fork 0
Code Issues Pull Requests Packages Projects Releases 19 Wiki Activity

fix: technical debts left by the Sites, Authors and SEO slices, plus backend support for the new Sites and Authors pagination

Browse Source
This commit is contained in:
Florian Sylvain committed 2026-10-19 00:25:31 +02:00
1 parent 9f4ce426a8
commit daf3594316
86 files changed
+4223 -1485
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

features/articles_pagination_api.feature
+73
View File
@@ -189,3 +189,76 @@ Feature: Paginated article listing
Given the repository fails on its next call
When a client sends GET /v1/article/
Then the status is 500 and the body is "Failed to retrieve articles"
Rule: counts.suggestions
Background:
Given counts holds all, online, draft and suggestions, and the rest of the body shape ({"items","total","page","size","counts"}) is unchanged
And suggestions is the number of visible articles whose seo_status is "suggestions"
And it ignores q, status, site, seo, sort, page and size, like the other counts
And an article whose seo_status is "hidden" or "ready" is not counted
And a suggestion code listed in the article's seo_dismissed is never counted
And it equals the total of GET /v1/article/?seo=suggestions sent with no other filter by the same caller
Scenario: counts carries the suggestions count
Given 4 online and 2 draft articles of which 3 have suggestions
When a verified account sends GET /v1/article/
Then counts is {"all": 6, "online": 4, "draft": 2, "suggestions": 3}
Scenario: It is zero when nothing needs attention
Given 5 articles that all have a clean SEO check
When a verified account sends GET /v1/article/
Then counts.suggestions is 0
Scenario: It equals the total of the suggestions filter
Given 30 articles of which 12 have suggestions
When a verified account sends GET /v1/article/?seo=suggestions
Then total is 12 and counts.suggestions is 12
Scenario Outline: It ignores every other query parameter
Given 6 articles of which 3 have suggestions, on the sites "renew" and "journal"
When a verified account sends GET /v1/article/?<params>
Then counts.suggestions is 3
Examples:
| params |
| q=tile |
| status=draft |
| site=journal |
| seo=ready |
| seo=hidden |
| sort=title:asc |
| size=5&page=99 |
Scenario: Hidden articles are not counted
Given an article with suggestions that is noindex
When a verified account sends GET /v1/article/
Then counts.suggestions is 0
Scenario: Dismissed suggestions are not counted
Given an article whose only suggestion is dismissed
When a verified account sends GET /v1/article/
Then counts.suggestions is 0
Scenario: A verified caller counts offline articles too
Given an online and a draft article, both with suggestions
When a verified account sends GET /v1/article/
Then counts.suggestions is 2
Scenario: A public caller counts online articles only
Given an online and an offline article, both with suggestions
When a caller without a verified account sends GET /v1/article/
Then counts is {"all": 1, "online": 1, "draft": 0, "suggestions": 1}
Scenario: An empty library
When a verified account sends GET /v1/article/ with no article stored
Then counts is {"all": 0, "online": 0, "draft": 0, "suggestions": 0}
Scenario: A failure to compute the count fails the request
Given the repository fails while the suggestions are evaluated
When a verified account sends GET /v1/article/
Then the status is 500 and the body is "Failed to retrieve articles"
Scenario: Existing fields are unchanged
Then total, page, size, ordering and every item field are as before
And counts.all, counts.online and counts.draft keep their values
features/articles_pagination_api.qa.md
+15
View File
@@ -30,6 +30,21 @@ Helper: decode the body into {items []map, total, page, size int, counts {all, o
18. Use-case tests: the paged list method returns items, total and counts, propagates a repo error (errors.Is on a sentinel), and the unpaged `Filters{}` call used by `purgePlan.go` and `ListMedia.go` still returns every article (existing tests keep passing).
19. Existing tests that decode `GET /v1/article/` as an array (`test/public_article_listing_test.go`, `test/article_test.go`, `test/use_case_baseline_test.go`, features/articles_list.qa.md item 1) are amended to the envelope. List them in the diff.
## counts.suggestions (Go, `test/`, `task test`)
Build `article.NewHandler(...)` over the real use cases and the extended `mockArticleRepo`, as in the section above. The fixture articles carry bodies that yield suggestions, hidden (noindex) or ready SEO status and a `seo_dismissed` list. Mount on a fresh `chi.NewRouter()` that mirrors `NewArticleRouter` and call `validator.InitValidator()`.
20. Counts table: 4 online and 2 draft with 3 suggestions gives `counts` exactly `{all:6, online:4, draft:2, suggestions:3}`. Decode into a map and assert the key set is exactly those four.
21. Zero case: 5 clean articles gives `suggestions` 0. Empty library gives all four at 0.
22. Equality: 30 articles, 12 with suggestions; `GET /?seo=suggestions` gives `total` 12 and `counts.suggestions` 12.
23. Independence table: `q=tile`, `status=draft`, `site=journal`, `seo=ready`, `seo=hidden`, `sort=title:asc`, `size=5&page=99` all give 3.
24. Exclusions: a noindex article with suggestions and an article whose only suggestion is dismissed are not counted; one with two suggestions of which one is dismissed is counted once.
25. Visibility: a verified caller counts an online and a draft article (2); an anonymous caller and an unverified caller count the online one only, with `draft` 0.
26. Repository failure while evaluating gives 500 "Failed to retrieve articles" and no partial body.
27. Regression: the existing assertions on `total`, `page`, `size`, `counts.all/online/draft` and items stay green without edits.
28. Docs, read by hand: `docs/api/articles.md` (example JSON gets `"suggestions"`, and the `counts` bullet says global, visibility-scoped, noindex and dismissed excluded, equal to the unfiltered `seo=suggestions` total); `web/src/data/apiCatalogue.ts` list response fields with its spec and `IntegrationView.spec.ts` counts; `docs/release-notes.md` line.
29. Scoped mutation through `task mutate` only: `MUTATE_FILES=ListArticles.go MUTATE_RUN='TestArticle.*Count' MUTATE_WORKERS=2 task mutate -- ./internal/infrastructure/useCases`, plus any other file touched by the change. Re-check LIVED or NOT COVERED suspects without `MUTATE_RUN`. Then `task lint:go` and `task fmt`.
## Docs (verified by reading the diff)
- `docs/api/articles.md`: new "Pagination, search, sort and counts" section (parameters including `sort` values, default and id tie-break, envelope, ordering, errors) and a "Breaking change" note, like the existing format note: GET /v1/article/ no longer returns a bare array; read `items`.
features/authors_backend.feature
+57 -1
View File
@@ -865,8 +865,64 @@ Feature: Authors backend, public bylines that articles credit in order
When it is read back from the repository
Then the authors come back as 7, 5, 6
Rule: Total in X-Total-Count
Background:
Given GET /v1/author/ keeps its {"items","total","page","size"} body and also sends total in the X-Total-Count header on every 200
And the header equals the body total
And X-Total-Count is already in the CORS ExposedHeaders
And a refused request sends no X-Total-Count
Scenario: The total is also sent as a header
Given 30 authors
When a verified account sends GET /v1/author/?size=5
Then the body total is 30 and the header X-Total-Count is "30"
Scenario: The header follows the search
Given 30 authors of which 7 match "mar"
When a verified account sends GET /v1/author/?q=mar
Then X-Total-Count is "7" and the body total is 7
Scenario: A page past the end keeps the total in the header
Given 30 authors
When a verified account sends GET /v1/author/?page=3&size=100
Then items is [], the body total is 30 and X-Total-Count is "30"
Scenario: No match
When a verified account sends GET /v1/author/?q=nobody
Then items is [], the body total is 0 and X-Total-Count is "0"
Scenario Outline: Every reader gets the header
When GET /v1/author/ is sent with <credentials>
Then the status is 200 and X-Total-Count equals the body total
Examples:
| credentials |
| the owner's jwt cookie |
| a verified admin's jwt cookie |
| a verified account with another role |
| a valid API key |
Scenario Outline: Refused requests send no header
When GET /v1/author/?<params> is sent with <credentials>
Then the status is <status> and no X-Total-Count is sent
Examples:
| params | credentials | status |
| size=30 | a verified account | 400 |
| page=0 | a verified account | 400 |
| page=abc | a verified account | 400 |
| | an unverified account's jwt cookie | 403 |
| | no credentials | 401 |
Scenario: CORS exposes the header
Given CORS_ALLOWED_ORIGINS includes "https://app.example.com"
When a preflighted GET /v1/author/ comes from that origin
Then Access-Control-Expose-Headers lists X-Total-Count
Rule: Documentation
Scenario: The docs describe the new surface
Then docs/api.md documents the /v1/author routes with auth and roles, the error tokens, the author shape, article author_ids and authors, the author field of /v1/auth/me and the authors_updated media field
Then docs/api.md names X-Total-Count on the GET /v1/author/ row
And docs/api.md documents the /v1/author routes with auth and roles, the error tokens, the author shape, article author_ids and authors, the author field of /v1/auth/me and the authors_updated media field
And docs/architecture.md mentions the author aggregate, its repository and its use cases
features/authors_backend.qa.md
+7
View File
@@ -102,6 +102,13 @@ Spec: `features/authors_backend.feature`. Go tests only, in `test/` (package `te
57. Scoped mutation run through `task mutate` only, for example `MUTATE_FILES=<author and article use case files> MUTATE_RUN='TestAuthor' MUTATE_WORKERS=2 task mutate -- <use case package path>`; report LIVED and NOT COVERED mutants; re-check filtered suspects without `MUTATE_RUN`. Also run `task lint:go` and `task fmt`.
58. Docs, checked by reading the files: docs/api.md documents the five /v1/author routes (GET, POST, POST /me, PATCH, DELETE) with auth and roles, error tokens, the author and article-author shapes, `author_ids`, the `/auth/me` `author` field and `authors_updated`; docs/architecture.md names the author aggregate, repository and use cases.
## M. X-Total-Count (Go, `test/`, `task test`)
59. On the same `mockAuthorRepo` and router as item 4, assert `X-Total-Count` equals the decoded body `total` for: size=5 of 30, `q=mar` with 7 matches, `page=3&size=100` (items empty, total 30), `q=nobody` (0), and for owner, admin, "editor" and API key. Assert the header is absent for `size=30`, `page=0`, `page=abc`, unverified (403) and none (401). The body assertions of item 4 stay unchanged.
60. On the SQLite contract harness, the author list total (and so the header) follows `q` and does not depend on `page`/`size`; `task test:mysql` and `task test:postgres` run the same case.
61. CORS preflight from an allowed origin exposes `X-Total-Count`.
62. Scoped `task mutate` on the author list use case and handler files only. `docs/api.md` author row and `web/src/data/apiCatalogue.ts` (`X-Total-Count` response header on `GET /v1/author/`) with its spec, read by hand. No comment added.
## Manual
- MySQL and Postgres: run the contract suite with the opt-in env vars against real servers.
features/seo_sites_articles_integration.feature
+20 -13
View File
@@ -8,8 +8,8 @@ Feature: SEO round, slice 4: Sites defaults, Articles search status, Integration
Given the screens are SitesView (/settings/sites), ArticleView (/article, also reached as /articles), IntegrationView (/integration) and the Editor SEO panel (/article/:id/edit), inside DashboardShell, per docs/design/routes.md
And the visual spec of record is docs/design/screens/sites.md ("Sites: SEO defaults", 5 states), articles.md ("Articles: search status", 3 states), integration.md ("Search and sharing"), editor.md (SuggestionList), docs/design/components.md (SeoStatusChip, SuggestionList), docs/design/seo-slices.md and docs/design/states.md ("SEO round"). Exact copy is read from those files and is not repeated here unless a scenario needs it
And the board frames of docs/design/source/DirectoryFrame.dc.html (Sites form, SuggestionList), AppFrame.dc.html (Articles rows, cards and toolbar) and IntegrationPage.dc.html (group and index) are the layout tiebreaker
And this slice is SPA only and uses the API of features/seo_data_backend.feature and features/seo_output_backend.feature without change. No Go file changes
And the SPA calls, with relative URLs through useApi only: GET /v1/site/, POST /v1/site/ and PATCH /v1/site/{id} (site_name, title_template, default_description), GET /v1/article/ with the optional seo=suggestions parameter, and PATCH /v1/article/{id} with seo_dismissed. The Integration page makes no request
And this slice is SPA only and uses the API of features/seo_data_backend.feature and features/seo_output_backend.feature, plus counts.suggestions added by features/articles_pagination_api.feature. No other Go file changes
And the SPA calls, with relative URLs through useApi only: GET /v1/site/, POST /v1/site/ and PATCH /v1/site/{id} (site_name, title_template, default_description), GET /v1/article/ with the optional seo=suggestions parameter, whose response also carries counts.suggestions, and PATCH /v1/article/{id} with seo_dismissed. The Integration page makes no request
And an article list item carries seo_status ("ready", "suggestions" or "hidden") and seo_suggestions (a count)
And the SPA never re-implements a suggestion rule or an SEO status. It draws the status and the count the list returned
And the API refuses a title_template without the exact, case-sensitive token %title% with 400 invalid_title_template, and refuses site_name above 100, title_template above 200 and default_description above 320 characters
@@ -306,23 +306,30 @@ Feature: SEO round, slice 4: Sites defaults, Articles search status, Integration
And the switch, the chips and ?seo=suggestions behave exactly as with several sites
Scenario Outline: The count on the switch
Given the count request answers <answer>
Given the switch is on or off
When the list response <answer>
Then the switch reads <label>
Examples:
| answer | label |
| total 3 | "Needs attention" with the count "3" in the mono face |
| total 0 | "Needs attention" with the count "0" |
| still pending | "Needs attention" with no count |
| status 500 or a network error | "Needs attention" with no count and no notice |
| answer | label |
| holds counts.suggestions 3 | "Needs attention" with the count "3" in the mono face |
| holds counts.suggestions 0 | "Needs attention" with the count "0" |
| has no counts.suggestions | "Needs attention" with no count |
| has not arrived yet (first load) | "Needs attention" with no count |
| failed with status 500 or a network error | "Needs attention" with no count and no extra notice |
@not-in-design
Scenario: Where the count comes from
Given the switch is off
Then one extra GET /v1/article/?seo=suggestions&page=1&size=5 is sent with the current q, status and site, and its total is the count
And the count is read from the main response total when the switch is on, with no extra request
And the count request is refreshed whenever q, tab or site change, and never when only the page, the size or the sort change
And a failing count request never blocks the list, never shows a notice and never changes the URL
Given the switch is on or off
Then the count is counts.suggestions of the latest successful GET /v1/article/ response
And no request with seo=suggestions and size=5 is sent for the count
And the count does not change when only q, tab, site, sort, page or size change, because counts.suggestions is global like the tab counts
@not-in-design
Scenario: The count survives a page or search change that is still loading
Given the count "3" was shown from an earlier response
When a new list request is in flight
Then the switch keeps showing "3"
Scenario: Needs attention: none
Given ?seo=suggestions, no search, the tab "All", and the response is empty
features/seo_sites_articles_integration.qa.md
+7 -7
View File
@@ -1,6 +1,6 @@
# QA: SEO round, slice 4 (Sites defaults, Articles search status, Integration, suggestion dismissal)
SPA slice only. No Go change: the API of `features/seo_data_backend.feature` and `features/seo_output_backend.feature` already exists, so this slice has no httptest part. A wrong contract is reported as a backend defect.
SPA slice only. No Go change here: the API of `features/seo_data_backend.feature` and `features/seo_output_backend.feature` already exists, and `counts.suggestions` is covered by the httptest part of `features/articles_pagination_api.qa.md`, so this file has no httptest part. A wrong contract is reported as a backend defect.
Roles: the domain defines only `owner` and `admin`. "verified" is a verified account without either role.
@@ -8,7 +8,7 @@ Roles: the domain defines only `owner` and `admin`. "verified" is a verified acc
Mock `useApi`'s `request` per case: GET/POST/PATCH /v1/site/..., GET /v1/article/ (items with `seo_status` and `seo_suggestions`, `counts`, `total`), PATCH /v1/article/{id}, POST /v1/article/{id}/seo/check, and the media routes of the shared picker. Mount SitesView, ArticleView, IntegrationView and EditorView inside DashboardShell with a resolved session and a test router. Mock matchMedia for desktop and mobile. Use fake timers for the search debounce. Do not freeze the system date. The rendering of "Updated …" is not asserted. Report the web tests as run or NOT RUN, never as passing when not run. One spec file per numbered group, under `web/src/views/__tests__/` (`SeoSites*.spec.ts`, `SeoArticles*.spec.ts`, `SeoDismiss*.spec.ts`, `SeoIntegration*.spec.ts`).
Regression: every existing Sites, Articles (ArticlesPagination, ArticlesPageSizeMemory, ArticlesSitesAuthors), Integration and EditorSeo* spec now sees the toolbar count request, the new fields and the dismiss buttons. Give them default mocks (count request answers total 3 on the request carrying `seo=suggestions`; sites with the three SEO fields) rather than loosening unmocked-call checks, and show that they still pass.
Regression: every existing Sites, Articles (ArticlesPagination, ArticlesPageSizeMemory, ArticlesSitesAuthors), Integration and EditorSeo* spec now sees the toolbar count request, the new fields and the dismiss buttons. Give them default mocks (every mocked `GET /v1/article/` envelope carries `counts.suggestions: 3`, and no spec mocks an extra count request any more; sites with the three SEO fields) rather than loosening unmocked-call checks, and show that they still pass.
1. Sites form fields: order, copy of states 1 and 2 (edit and add), placeholders, helpers, the counter, no "(optional)" on Site name, the "How articles use these defaults" block and its placement under the preview block.
2. Live example: the Examples table (site name, name fallback, placeholder "Renew" fallback, `%title%` only, two `%site%`), updated on each keystroke, no request sent; global replace, not first occurrence.
@@ -17,8 +17,8 @@ Regression: every existing Sites, Articles (ArticlesPagination, ArticlesPageSize
5. Save payloads: PATCH holds only changed fields (assert exact JSON) for template only and for name + description + robots; POST omits the three fields when untouched and sends them when changed; the discard dialog on a changed SEO field; a site without the fields opens with defaults and "Save changes" sends nothing.
6. Save failure and sessions: 500 and network failure keep the panel and the notice; 403 `owner_only` gives the existing toast; 401 goes to `/login?redirect=/settings/sites&reason=expired`; admin and verified users get the 403 page and no form.
7. SeoStatusChip: the status table (ready, suggestions 1 and 2, hidden with and without a count), placement in the Status cell and in the card meta row, tooltip/accessible name (singular and plural), the unusable items (no seo_status, suggestions with 0, unknown status), the row action that keeps the chip.
8. Toggle and URL: parse and write `seo` (`parseArticleQuery`, `toRouteQuery`, `sameArticleQuery`), page reset, q/status/sort/site/size kept, other changes keep seo, reset-to-1 table, opened with `?seo=suggestions&page=2`, unknown values dropped silently via replace (the six values), the request carries `seo=suggestions` only when on, tab counts unchanged.
9. Count: total 3, total 0, pending, failure (no count, no notice, no URL change); the extra request (`seo=suggestions&page=1&size=5`, with q, status and site) is made only while off, only when q/tab/site change, and the main response total is used while on.
8. Toggle and URL: parse and write `seo` (`parseArticleQuery`, `toRouteQuery`, `sameArticleQuery`), page reset, q/status/sort/site/size kept, other changes keep seo, reset-to-1 table, opened with `?seo=suggestions&page=2`, unknown values dropped silently via replace (the six values), the request carries `seo=suggestions` only when on, tab counts unchanged and `counts.suggestions` unchanged by the toggle.
9. Count: `counts.suggestions` 3, 0, absent (no count shown), list pending on first load (no count), list failure 500 and network error (no count, no extra notice, URL unchanged); the count keeps its last value while a new request is in flight; no request carrying `seo=suggestions&page=1&size=5` is ever sent (assert the mocked `useApi` is never called with it, with the toggle off, with q, status and site set, and after a change of each); the count is the same on and off, and unchanged by q, tab, site, sort, page and size.
10. Empty states: "Nothing needs attention" with its action (clears seo only), the not-in-design combinations (search, Draft, Online, a site), paginator hidden, Retry notice still works.
11. Single-site install: the Site column and filter are absent and everything above behaves the same.
12. Dismiss ×, rendering: one button per row of the panel list, none in the compact section list; the footnote; accessible name with the message interpolated; Tab order (after "Go to …"); Enter and Space; visible focus class.
@@ -32,7 +32,7 @@ Regression: every existing Sites, Articles (ArticlesPagination, ArticlesPageSize
Setups go in `web/visual/states.ts`, one per state, with mocked /v1 and the board's own sample data. Hard gate: every capture within 10 %, no per-state exception. Reference frames: `DirectoryFrame.dc.html` (Sites form, SuggestionList), `AppFrame.dc.html` (Articles) and `IntegrationPage.dc.html`.
- Sites: SEO defaults (5): Edit: SEO defaults (Renew, `f.sname`, template `%title% | %site%`, example "How we price a bathroom renovation | Renew"), Add: defaults, Template: custom (site name Journal, template `%title% – %site%`), Template: not valid, Save failed. The existing Sites form frames (7) now contain the three fields and the new block: update their setups with the board values.
- Articles: search status (3): Status chips (rows: 2 suggestions, hidden, 3 suggestions, 1 suggestion), Needs attention on, Needs attention: none. Every existing Articles and pagination frame now shows the toggle with the count 3 and the chips of the board rows, so each setup mocks the count request with total 3. States whose "Updated" column or text is clock-sensitive follow what the existing setups already do (`skip: clock-sensitive`), and state 8 "Published" stays skipped.
- Articles: search status (3): Status chips (rows: 2 suggestions, hidden, 3 suggestions, 1 suggestion), Needs attention on, Needs attention: none. Every existing Articles and pagination frame now shows the toggle with the count 3 and the chips of the board rows, so each setup in `web/visual/states.ts` puts `counts.suggestions: 3` in the list mock and mocks no extra request. States whose "Updated" column or text is clock-sensitive follow what the existing setups already do (`skip: clock-sensitive`), and state 8 "Published" stays skipped.
- Integration (8 existing states): the index and the mobile select now contain "Search and sharing", and the page has the new group (collapsed). Mock nothing new. The Navigation state's open select must list the new option. If the 403 note makes a capture exceed 10 %, report it to the user; do not add an exception.
- Editor: the dismiss column is now implemented. Remove the five `withMobileException(..., dismissColumnReason)` wrappers (and the `dismissColumnReason` constant, `withMobileException` if nothing else uses it) in `web/visual/states.ts`, covering 'Panel: defaults', 'Panel: outside the soft ranges', 'Panel: cover without alt', 'Panel: choose cover' and the fifth state at line ~5828. Those states must pass `task visual:check -- editor` at the default 10 % in both viewports. If one still fails, report it; do not restore an exception.
- Re-run Settings, Users, Profile and Home: report any capture above 10 %. Any failure needs the user's decision (an `approvedBy` exception or a design update).
@@ -65,8 +65,8 @@ Setups go in `web/visual/states.ts`, one per state, with mocked /v1 and the boar
Each has a French entry in `web/src/i18n/messages.ts`.
1. Source of the "Needs attention" count (extra request, `total`).
2. Zero count shown as "0", no count while pending or on failure.
1. Source of the count (`counts.suggestions` of the list response).
2. Zero shown as "0"; no count while pending, absent or on failure.
3. Empty-state precedence when `seo=suggestions` is combined with a search, a tab or a site.
4. Chips refresh only with the next list load after a row action.
5. Items without a usable status show no chip.
features/sites_backend.feature
+144
View File
@@ -632,9 +632,153 @@ Feature: Sites backend, the owner manages sites and every article belongs to exa
When a verified account sends GET /v1/activity/?category=sites
Then the site events are returned with category "sites"
Rule: Pagination and search of the site list
Background:
Given GET /v1/site/ accepts q, page and size, keeps the body a JSON array, and sends the number of matching sites in the X-Total-Count header on every 200
And when neither page nor size is sent, every matching site is returned
And when page or size is sent, size is one of 5, 25, 50 or 100 (default 25 if only page is sent) and page is a positive integer (default 1 if only size is sent)
And q is at most 200 characters and matches name, key or base_url as a case- and accent-insensitive substring
And the order is unchanged and applied before paging: the default site first, then by name, then by id
And X-Total-Count is already in the CORS ExposedHeaders, so the router needs no change
And an invalid page, size or q answers 400 with apperr.BodyErrorMessage and sends no X-Total-Count, like GET /v1/article/
And access is unchanged: a verified session or an API key reads, an unverified account gets 403, no credentials get 401
And the body is [] and never null when nothing matches
Scenario: With no paging parameter every site is returned
Given 61 sites of which "Renew" is the default
When a verified account sends GET /v1/site/
Then the status is 200
And the body is an array of 61 sites starting with "Renew"
And X-Total-Count is "61"
Scenario: q alone returns every match
Given 61 sites of which 40 match "shop"
When a verified account sends GET /v1/site/?q=shop
Then the body holds 40 sites and X-Total-Count is "40"
Scenario Outline: Paging and sizes
Given 61 sites
When a verified account sends GET /v1/site/?<params>
Then the body holds <count> sites
And X-Total-Count is "61"
Examples:
| params | count |
| page=1 | 25 |
| page=2 | 25 |
| page=3 | 11 |
| size=5 | 5 |
| size=25 | 25 |
| size=50&page=2 | 11 |
| size=100 | 61 |
Scenario: Pages never overlap and keep the order
Given 61 sites
When pages 1 to 3 are read with the default size
Then the ids are all distinct, the default site is first on page 1, and the concatenation equals the unpaged list
Scenario: A page past the end is an empty array with the total
Given 61 sites
When a verified account sends GET /v1/site/?page=99
Then the status is 200 and the body is []
And X-Total-Count is "61"
Scenario: No match is an empty array
When a verified account sends GET /v1/site/?q=nomatch
Then the status is 200 and the body is [] and not null
And X-Total-Count is "0"
Scenario Outline: Search matches name, key and base URL
Given the sites "Renew" (key "renew", https://renew.example.com), "Journal" (key "blog", https://journal.example.org) and "Archive" (key "archive", https://old.example.net)
When a verified account sends GET /v1/site/?q=<q>
Then the names are <names>
And X-Total-Count is "<total>"
Examples:
| q | names | total |
| renew | "Renew" | 1 |
| JOURNAL | "Journal" | 1 |
| blog | "Journal" | 1 |
| example.org | "Journal" | 1 |
| example | "Renew", "Journal", "Archive" | 3 |
| nobody | none | 0 |
Scenario: Search ignores accents
Given the site "Café" with key "cafe"
When a verified account sends GET /v1/site/?q=cafe and GET /v1/site/?q=CAFÉ
Then both answer "Café" with X-Total-Count "1"
Scenario: Search and paging compose and the total follows the search
Given 61 sites of which 27 match "shop"
When a verified account sends GET /v1/site/?q=shop&page=2
Then the body holds 2 sites and X-Total-Count is "27"
Scenario: The default site is not forced into the results
Given the default site "Renew" and the site "Journal"
When a verified account sends GET /v1/site/?q=journal
Then the body is "Journal" only
Scenario: A wildcard in q is literal
Given the site "100% Shop" and the site "Main"
When a verified account sends GET /v1/site/?q=%25
Then the body is "100% Shop" only
Scenario Outline: Invalid page, size or q
When a verified account sends GET /v1/site/?<params>
Then the status is 400
And the body is apperr.BodyErrorMessage
And no X-Total-Count header is sent
Examples:
| params |
| page=0 |
| page=-1 |
| page=abc |
| page= |
| size=0 |
| size=30 |
| size=101 |
| size=abc |
| size= |
| q=(201 characters) |
Scenario: q at the limit is accepted
When a verified account sends GET /v1/site/?q=(200 characters)
Then the status is 200 and the body is [] and X-Total-Count is "0"
Scenario: The other fields are unchanged
When a verified account sends GET /v1/site/?size=5
Then every site carries the same fields as before, article_count included, and the owner-only writes are unchanged
Scenario Outline: Every reader gets the header
When GET /v1/site/?size=5 is sent with <credentials>
Then the status is 200 and X-Total-Count is present
Examples:
| credentials |
| the owner's jwt cookie |
| a verified admin's jwt cookie |
| a valid API key |
Scenario Outline: Refused readers get no header
When GET /v1/site/?size=5 is sent with <credentials>
Then the status is <status> and no X-Total-Count is sent
Examples:
| credentials | status |
| an unverified account's jwt cookie | 403 |
| no credentials | 401 |
Scenario: CORS exposes the header
Given CORS_ALLOWED_ORIGINS includes "https://app.example.com"
When a preflighted GET /v1/site/?page=2 comes from that origin
Then Access-Control-Expose-Headers lists X-Total-Count and Link
Rule: Documentation
Scenario: The docs describe the new surface
Then docs/api.md documents the /v1/site routes, their errors, article site_id, the ?site= filter and the media sites_updated field
And docs/api.md documents q, page, size, the all-sites default without paging parameters, the 400, the past-the-end page and X-Total-Count on GET /v1/site/
And docs/configuration.md states that PUBLIC_URL seeds the first site once at migration
And docs/architecture.md mentions the site aggregate, its repository and its use cases
features/sites_backend.qa.md
+17
View File
@@ -88,8 +88,25 @@ Spec: `features/sites_backend.feature`. Go tests only, in `test/` (package `test
47. Scoped mutation run through `task mutate` only, for example `MUTATE_FILES=<site and article use case files> MUTATE_RUN='TestSite' MUTATE_WORKERS=2 task mutate -- <use case package path>`; report LIVED and NOT COVERED mutants; re-check filtered suspects without `MUTATE_RUN`. Also run `task lint:go` and `task fmt`.
48. Docs (Rule: Documentation), checked by reading the files: docs/api.md documents the five /v1/site routes with auth, error tokens, article `site_id`, the `?site=` filter, `sites_updated` on media merge and purge; docs/configuration.md says PUBLIC_URL seeds the first site once at migration; docs/architecture.md names the site aggregate, repository and use cases. No comment is added to any file.
## L. Pagination and search of the list (Go, `test/`, `task test`)
Build `site.NewHandler(...)` over `mockSiteRepo` extended with `q`, paging and a total, mounted on a `chi.NewRouter()` mirroring `NewSiteRouter`, with the same jwtauth setup and cookies as above. Call `validator.InitValidator()`. Add the CORS options of `enableCorsIfConfigured` to one test router to check the exposed header (pattern: `test/seo_output_sitemap_http_test.go`).
49. 61 sites, no parameter: 61 items, default first, `X-Total-Count` "61". `q=shop` alone returns every match with the matching count.
50. Page/size table (page=1/2/3, size=5/25, size=50&page=2, size=100): count of items and `X-Total-Count` "61" each time.
51. Pages 1-3 have distinct ids; their concatenation equals the unpaged list; the default site is first on page 1.
52. `page=99` gives 200, body exactly `[]` (raw-bytes check, not `null`) and the real total. `q=nomatch` gives `[]` and "0".
53. The `q` table (name, key, base URL, uppercase, accent folding, partial host, no match), search plus paging, the default site not forced into a non-matching result, and `%` and `_` in `q` matched literally.
54. The invalid table (page 0, -1, abc, empty; size 0, 30, 101, abc, empty; q of 201 characters): 400 `apperr.BodyErrorMessage` and no `X-Total-Count`. q of 200 characters is 200 with `[]`.
55. Credentials table: owner, admin, API key give 200 with the header; unverified 403 and none 401 give no header. The existing section F owner-only write table is rerun and stays green.
56. CORS preflight from an allowed origin exposes `X-Total-Count`.
57. Repository (real SQLite in the temp-dir harness, same file as the contract suite): ordering across the page boundary, `q` over the three fields, total equals matches, `article_count` correct on each page. MySQL and Postgres run the same cases through `task test:mysql` and `task test:postgres`, which use the opt-in switches `RENEWCMS_TEST_MYSQL_DSN` and `RENEWCMS_TEST_POSTGRES_DSN`.
58. Docs, read by hand: `docs/api.md` (the `GET /v1/site/` row), `web/src/data/apiCatalogue.ts` (query parameters q/page/size and the `X-Total-Count` response header, no-paging default) with `apiCatalogue.spec.ts`, and the Integration reference counts in `IntegrationView.spec.ts`. No comment added anywhere.
59. Scoped `task mutate` only: `MUTATE_FILES=<site list use case and repository files> MUTATE_RUN='TestSite' MUTATE_WORKERS=2 task mutate -- ./internal/infrastructure/useCases`. Report LIVED and NOT COVERED, and re-check filtered suspects without `MUTATE_RUN`. Then `task lint:go` and `task fmt`.
## Manual
- `curl -i -b <cookie> 'http://localhost:8080/v1/site/?q=ren&size=5&page=2'` against `task dev`; a browser cross-origin read with `CORS_ALLOWED_ORIGINS` set; the editor's Site select and the Articles Site filter still list all sites (they send no paging parameter).
- MySQL and Postgres: run the contract suite with the opt-in env vars against real servers.
- Real boot with an old database copy and PUBLIC_URL set: one "Default" site appears, articles backfilled, second boot changes nothing.
- Docker with the `/data` volume: boot an existing database and confirm the same.
features/tech_debt_hardening.feature
+216
View File
@@ -0,0 +1,216 @@
Feature: Hardening after the Sites, Authors and SEO slices
As the maintainer of a RenewCMS installation
I want the SEO panel, the SEO check cache, the chunked lookups and the slug backfill to hold on large data
So that behaviour stays correct beyond the first page of media, on every database, and on every boot
Background:
Given this file covers four items; items 4, 8 and 9 are behaviour-preserving and item 3 adds one read endpoint, GET /v1/media/{id}
And every existing scenario stays green without edits to its assertions
And no comment is added to any file
Rule: The SEO panel names its images by id (item 3)
Background:
Given GET /v1/media/{id} is mounted in NewMediaRouter behind the same middleware as GET /v1/media/: jwtauth.Verifier, the API key authentication, AuditContext, RequireSignedIn, jwtauth.Authenticator, RequireCurrentToken and RequireVerifiedUser
And it reads one image and writes nothing, so it records no activity event
And it answers 200 with the same JSON item shape as one element of items in GET /v1/media/
And a missing image answers 404 with apperr.ImageNotFoundErrorMessage ("Image not found")
And a malformed id answers 400 with apperr.IdUint32ErrorMessage
And the SPA useMediaNames resolves the wanted ids with GET /v1/media/{id} and no longer reads the first 100 items of the list
Scenario Outline: Who can read one image
Given the image 5 exists
When GET /v1/media/5 is sent with <credentials>
Then the status is <status>
Examples:
| credentials | status |
| the owner's jwt cookie | 200 |
| a verified admin's jwt cookie | 200 |
| a verified account with another role | 200 |
| a valid API key | 200 |
| an unverified account's jwt cookie | 403 |
| no credentials | 401 |
Scenario: The item has the shape of the list
Given the image 5 exists with a name, a hash, a size, dimensions, a used_in count and variants information
When a verified account sends GET /v1/media/5
Then the body is the JSON object equal to the element of items for the image 5 in GET /v1/media/
And no other field is added or removed
Scenario: A missing image is not found
When a verified account sends GET /v1/media/999
Then the status is 404 and the body is "Image not found"
Scenario Outline: A malformed id is refused
When a verified account sends GET /v1/media/<id>
Then the status is 400 and the body is apperr.IdUint32ErrorMessage
Examples:
| id |
| abc |
| -1 |
| 4294967296 |
| 1.5 |
Scenario: The route does not shadow the existing media routes
When GET /v1/media/5/usage and GET /v1/media/5/variants are sent as before
Then they answer exactly as before this change
Scenario: A non-GET method keeps its meaning
When PATCH /v1/media/5 and DELETE /v1/media/5 are sent as before
Then they behave exactly as before this change
Scenario: Reading one image records nothing
When a verified account sends GET /v1/media/5
Then no activity event is recorded
Scenario: A selected image outside the first 100 media items is named
Given the article's cover image has id 150 and is not among the first 100 media items
When the SEO panel opens
Then the panel requests GET /v1/media/150 and shows the name and thumbnail of the image 150
And the same holds for the social image
Scenario: Images already known are not fetched again
Given the panel has resolved the images 150 and 151
When the panel re-renders with the same selection
Then no lookup request is sent
Scenario: An image that no longer exists
Given GET /v1/media/150 answers 404
Then the panel shows its existing "unknown image" fallback and no error notice
Scenario: A failing lookup never blocks the panel
Given GET /v1/media/150 fails with a 500 or a network error
Then the panel stays usable, shows the existing fallback and raises no notice
Scenario: Behaviour with fewer than 100 media items is unchanged
Given the library holds 12 media items
Then the names shown are the same as before
Scenario: The list is no longer requested for names
When the SEO panel resolves any selected image
Then no GET /v1/media/?page=1&size=100&sort=newest request is sent for names
Scenario: The docs describe the route
Then docs/api.md documents GET /v1/media/{id} with its auth, the 200 shape, the 404 and the 400
And web/src/data/apiCatalogue.ts lists GET /v1/media/{id} in the Media group, with its spec and the Integration reference counts updated
Rule: The SEO check answer cache (item 4)
Scenario: The same effective input is checked once
Given a check was answered for an article body and fields
When the same fields are checked again with a different but irrelevant difference in the request (key order or whitespace of the JSON)
Then no second POST /v1/article/{id}/seo/check is sent and the cached answer is shown
Scenario Outline: A field that affects the result invalidates the cache
Given a check was answered
When <field> changes and the check runs again
Then POST /v1/article/{id}/seo/check is sent again
Examples:
| field |
| the title |
| the body |
| the description |
| the excerpt |
| the slug |
| the cover image |
| the social image |
| the noindex switch |
| the site |
| the canonical URL |
Scenario: A field that does not affect the result does not invalidate it
Given a check was answered
When only a field the check ignores changes
Then the cached answer is reused
Scenario: A failed check is not cached
Given a check failed with a 500
When the same input is checked again
Then the request is sent again
Scenario: The cache stays bounded
Given 200 distinct inputs were checked
Then the cache holds no more than its existing limit and a repeated recent input is still a hit
Rule: Chunked IN lookups on a real database (item 8)
Background:
Given the article repository looks ids up in batches of 500 (articles by id, article_authors by article id, authors by id)
Scenario: A page-sized lookup keeps every row once and in order
Given 1,200 articles, so the lookup spans 3 batches
When the repository loads all 1,200 ids
Then 1,200 articles come back, each once, in the requested order
Scenario Outline: Batch boundaries
Given <n> ids are requested and all exist
Then exactly <n> rows come back
Examples:
| n |
| 0 |
| 1 |
| 499 |
| 500 |
| 501 |
| 1000 |
| 1001 |
Scenario: Missing ids are absent
Given 600 ids of which 100 do not exist
Then 500 rows come back and none is invented
Scenario: Credits keep their order across batches
Given 700 articles each crediting 3 authors in a chosen order
When the credits are loaded
Then every article lists its authors in its stored order (position, then id)
Scenario: Soft-deleted rows follow the existing rule
Given 3 of the 1,200 articles are soft-deleted
Then the lookup returns the same rows as the single-query path would
Scenario: It works on every supported database
When the tests run against SQLite, MySQL/MariaDB and PostgreSQL
Then the results are identical
Rule: The article slug backfill (item 9)
Scenario: A large database is filled in batches
Given a database from before slugs holding 5,000 articles with empty slugs
When the backfill runs
Then every article has a non-empty slug unique within its site
And the writes were made in several transactions of bounded size and not in one
And the batch size is a named constant chosen by the implementation
Scenario: The second boot changes nothing
Given the backfill already ran
When it runs again
Then no row is written, no slug changes and updated_at is untouched
Scenario: A crashed backfill resumes
Given the backfill stopped after its first batch
When it runs again
Then the remaining articles are filled and the already filled ones keep their slugs
Scenario: Duplicates get the first free suffix
Given 3 articles with the same title in one site and 1 in another site
When the backfill runs
Then the first keeps the plain slug, the others get -2 and -3, and the other site's article keeps the plain slug
Scenario: Soft-deleted rows get their tombstone once
Given 2 soft-deleted articles
When the backfill runs twice
Then each has the tombstone slug after the first run and is not rewritten by the second
Scenario: Existing slugs are kept
Given an article whose slug was set by the owner
When the backfill runs
Then its slug is unchanged
Scenario: The unique index exists afterwards and a repeat does not fail
When the backfill and index creation run on a second boot
Then idx_articles_site_slug exists and no error is returned
features/tech_debt_hardening.qa.md
+63
View File
@@ -0,0 +1,63 @@
# QA procedure: Hardening after the Sites, Authors and SEO slices
Spec: `features/tech_debt_hardening.feature`. The Go part is in `test/` (package `test`, `task test`); the web part is Vitest (`task test:web`). Constraints: no commits; no comments in code; scoped `task mutate` only for the Go files of items 3, 8 and 9 (and of items 5 and 6 per their own QA files); `task visual:check` only on `articles`, `sites` and `integration` to confirm no regression, with no new `approvedBy` exception and the waived screens untouched; `docs/design` read-only.
## Item 3: GET /v1/media/{id} (Go, `test/`, `task test`)
Build `media.NewHandler(...)` over the real media use cases and the existing fake image repository and storage fakes used by the media tests (unique fake type names across files). Mount on a fresh `chi.NewRouter()` that mirrors `NewMediaRouter` in `api/router.go`, behind `jwtauth.Verifier`, the API key authentication path, `middleware.RequireSignedIn`, `jwtauth.Authenticator` and `RequireVerifiedUser`, with a test `jwtauth.New("HS256", ...)`. Requests carry signed `jwt` cookies with the `user_id` claim for an owner, an admin, a verified account with another role and an unverified account. Call `validator.InitValidator()`.
1. Credentials table on an existing image 5: owner, admin, other verified role and a valid API key give 200; an unverified account gives 403; no credentials give 401.
2. Shape: decode `GET /v1/media/5` and the matching element of `items` from `GET /v1/media/` into maps; assert the key sets and values are equal.
3. `GET /v1/media/999` gives 404 with the exact plain-text body "Image not found" (`apperr.ImageNotFoundErrorMessage`).
4. Malformed ids `abc`, `-1`, `4294967296`, `1.5` give 400 with `apperr.IdUint32ErrorMessage`.
5. Regression: `GET /{id}/usage`, `GET /{id}/variants`, `PATCH /{id}` and `DELETE /{id}` still answer as before (the existing tests stay green without edits), and the variant routes keep their `RejectAPIKey` and owner/admin rule.
6. No activity event is recorded by the read (the fake audit repository stays empty).
7. Docs, read by hand: `docs/api.md` documents `GET /v1/media/{id}` (auth, 200 shape, 404, 400); `web/src/data/apiCatalogue.ts` lists it in the Media group, and its spec and `IntegrationView.spec.ts` counts are updated; `docs/release-notes.md` line.
8. Scoped mutation through `task mutate` only, for the media files touched by the change: `MUTATE_FILES=<the GetMedia use case file and any media file touched> MUTATE_RUN='TestMedia' MUTATE_WORKERS=2 task mutate -- ./internal/infrastructure/useCases`. Report LIVED and NOT COVERED; re-check filtered suspects without `MUTATE_RUN`. Then `task lint:go` and `task fmt`.
## Item 3: media names (Vitest, `web/`)
Mock `useApi`'s `request` for `GET /v1/media/{id}` per case.
9. Open the SEO panel with cover id 150 and social id 151 (neither in any first page of the library). Mock `GET /v1/media/150` and `GET /v1/media/151` to return items. Assert the panel shows their names and thumbnails, and that no `GET /v1/media/?page=1&size=100&sort=newest` request is made.
10. Re-render with the same selection: assert no new request (the mocked call count stays).
11. `GET /v1/media/150` answers 404: the fallback is shown, no notice. A 500 and a network error: panel usable, fallback, no notice.
12. A library of 12 items: the same names as before.
13. The existing `useMediaNames` and SEO-panel specs pass; the assertions that depended on the list lookup are amended only where they mock the old request, and that diff is listed in the report.
## Item 4: SEO check cache (Vitest)
14. Same fields, different key order: one POST (assert the mocked call count).
15. Table, one case per result-affecting field (title, body, description, excerpt, slug, cover image, social image, noindex, site, canonical URL): a second POST is sent.
16. A field the check ignores changes: no second POST.
17. A 500 is not cached: the retry POSTs.
18. 200 distinct inputs: the cache size stays at its existing limit and the most recent input is a hit.
19. The existing editor SEO specs pass unchanged.
## Item 8: chunked lookups (Go, `test/`)
20. Temp-dir SQLite (never the real database file): seed 1,200 articles; load all ids; assert 1,200 rows, no duplicates, requested order.
21. Boundary table 0, 1, 499, 500, 501, 1000, 1001 ids.
22. 600 ids with 100 missing gives 500 rows. 700 articles with 3 ordered credits each come back in stored order across batches (including the `position ASC, id ASC` rule). Soft-deleted rows match the single-query path.
23. Name the MySQL and Postgres variants so the existing filters match them: `TestMySQLChunkedLookups` (run by `-run MySQL`) and `TestPostgresChunkedLookups` (run by `-run Postgres`). They skip when `RENEWCMS_TEST_MYSQL_DSN` or `RENEWCMS_TEST_POSTGRES_DSN` is unset, like the existing contract tests.
## Item 9: slug backfill (Go, `test/`, temp-dir SQLite)
24. Seed 5,000 articles with empty slugs across two sites; run `database.BackfillArticleSlugs`; assert unique non-empty slugs within each site; assert the writes ran in several bounded transactions (the batch size is a named constant chosen by the coder; assert through a counting hook or a GORM callback counter in the test, not a log).
25. Second boot: record every row's `slug` and `updated_at` and the DB write count (callback), run again, and assert the values are identical and the write count is 0. Also call `database.Migrate` twice.
26. Interrupted run: stop after the first batch (failure injection on the second batch), assert the first batch is committed, rerun, and assert the rest are filled and the first batch's slugs unchanged.
27. Suffix table (3 same-title articles in one site plus 1 in another), tombstones for soft-deleted rows written once, owner-set slugs kept, index `idx_articles_site_slug` present and a repeated `ensureArticleSlugIndex` returns nil.
28. Existing slug migration tests (`planSlugAssignments`) stay green.
## Quality gates
29. Layering guard: use cases import no persistence, mailer, models, gorm, net/http or chi; the domain package imports no GORM or HTTP; handlers import no persistence; no comment added.
30. Scoped mutation for items 3, 8 and 9 only, through `task mutate`: `MUTATE_FILES=articleRepository.go,articleSlugBackfill.go,<item 3 media files> MUTATE_RUN='TestChunked|TestBackfill|TestMedia' MUTATE_WORKERS=2 task mutate -- ./internal/infrastructure/persistence/repositories ./internal/infrastructure/persistence/database ./internal/infrastructure/useCases`. Report LIVED and NOT COVERED; re-check filtered suspects without `MUTATE_RUN`. Then `task lint:go`, `task fmt`, `task test` and `task test:web`.
31. `task visual:check -- articles`, `-- sites` and `-- integration`: all captures pass at 10 %; no exception added.
## Manual
- MySQL and PostgreSQL (item 8, and the repository cases of items 5 and 6): run `task test:mysql` (mysql:8.4 then mariadb:11) and `task test:postgres` (postgres:16 then 17). Docker is required; the tasks start and remove the containers. Extra arguments go after `--`, for example `task test:postgres -- -run Postgres.*Chunked -v`. To use your own server instead: `RENEWCMS_TEST_MYSQL_DSN='user:pass@tcp(host:3306)/db' go test -tags nofrontend ./test/... -run MySQL -count=1` and `RENEWCMS_TEST_POSTGRES_DSN='postgres://user:pass@host:5432/db?sslmode=disable' go test -tags nofrontend ./test/... -run Postgres -count=1`. Use a scratch database. No CI exists, so this is run by hand.
- Item 3 on a live app: with more than 100 media items, open an article whose cover is an old image; the SEO panel shows its name. `curl -i -b <cookie> http://localhost:8080/v1/media/<id>` returns the item, 404 for a missing id and 400 for `abc`.
- Slug backfill on a real file: copy an old database, boot with `task dev`, confirm slugs and that a second boot changes no row (compare `updated_at` before and after).
- Item 7 (the `?seo=` measurement) is a measurement with its result recorded in `docs/`, so it has no Gherkin.