refac: article reads, story delete, no leaks, stored SEO status, rate limit, cleanup (%site%, article repo split, old lang repos, one default per site rule to domain)
This commit is contained in:
Florian Sylvain committed
1 parent
9d284e65a9
commit
7c15206170
85 files changed
+1788
-593
No files matched your search
+1
-1
@@ -178,7 +178,7 @@ RenewCMS renders no page and injects no meta tag: it returns the data a frontend
|
||||
| `POST /v1/article/{id}/seo/check` | verified session or API key (401 without credentials, 403 unverified) | Evaluates the article without saving. The body is `{}` or any of the twelve SEO fields of `PATCH` plus `body`; each replaces the saved value for this check only. `site_id`, `author_ids`, unknown fields and `null` (except on `cover_image_id` and `social_image_id`) are 400 with the body-mismatch message. The other validation is the one of `PATCH` (`invalid_slug`, `invalid_url` as 422, `image_not_found`, validator text). Answers `{ status, suggestions, slug }` where `suggestions` is `[{ code, field, params }]` and `slug` is `{ available, suggestion }`. A taken slug is a 200 with `available: false` and the first free `<slug>-N` in `suggestion`, never a 409. |
|
||||
| `GET /v1/site/{key}/sitemap` | public | A JSON array `[{ url, modified_at, noindex, lang }]` of the eligible versions (published, language enabled) of the site with that key, by id ascending. `url` is the localized address when the site has a Public URL pattern, otherwise `base_url/slug` (never the canonical override). `page` (default 1) and `size` (25, 50, 100 or 500, default 100); an invalid value is 400 with the body-mismatch message. The total is in the `X-Total-Count` header. An unknown key, an id included, is 404 `site_not_found`. |
|
||||
|
||||
`GET /v1/article/` also takes `seo=ready|suggestions|hidden` (any other value, an empty one included, is 400 with the body-mismatch message, after the `format` check). It is applied before paging and composes with `q`, `status`, `site`, `sort`, `page` and `size`; `total` follows it and `counts` ignore it. Every item carries `seo_status` (`ready`, `suggestions` or `hidden`) and `seo_suggestions` (the number of suggestions that are not dismissed). The filter is evaluated in memory over every matching article, so its cost grows with the number of articles on the site; without it only the returned page is assessed.
|
||||
`GET /v1/article/` also takes `seo=ready|suggestions|hidden` (any other value, an empty one included, is 400 with the body-mismatch message, after the `format` check). It is applied before paging and composes with `q`, `status`, `site`, `sort`, `page` and `size`; `total` follows it and `counts` ignore it. Every item carries `seo_status` (`ready`, `suggestions` or `hidden`) and `seo_suggestions` (the number of suggestions that are not dismissed). The filter reads the status stored on each article (see [SEO status and suggestions](api/articles.md#seo-status-and-suggestions)), in SQL, before paging.
|
||||
|
||||
`X-Total-Count` is added to `Access-Control-Expose-Headers` next to `Link`, so a browser on an allowed origin can read it.
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ The headers are set on those responses only. Same-origin SPA code reads them dir
|
||||
| 400 | `Article title must be between 3 and 200 characters.` | a `PATCH` title outside 3 to 200 runes |
|
||||
| 400 | `invalid_slug` | a `PATCH` slug that is not a slug |
|
||||
| 401 | `You must be logged in to access this resource.` | no valid session or key |
|
||||
| 403 | `missing_permission` | the caller lacks the permission (`X-Missing-Permission`) |
|
||||
| 403 | `missing_permission` | the caller lacks the permission (`X-Missing-Permission`); on `DELETE /v1/story/{story_id}`, a `content.delete` that is not at scope `all` |
|
||||
| 404 | `Article not found` | the version, or the story, does not exist or the caller cannot read it |
|
||||
| 404 | `source_unavailable` | the chosen source does not exist or is not readable; the same answer for both |
|
||||
| 404 | `site_not_found` | `POST /v1/story/{story_id}/move`, or `POST /v1/article/` with `site`, naming an unknown site key |
|
||||
@@ -43,7 +43,7 @@ The headers are set on those responses only. Same-origin SPA code reads them dir
|
||||
| 409 | `slug_taken` | a `PATCH` slug used by another article of the same site and language (`X-Slug-Suggestion`) |
|
||||
| 409 | `slug_conflict` | a move would put an address that already exists at the destination (`X-Conflicting-Articles`) |
|
||||
| 409 | `story_move_required` | `PATCH site_id` on a version whose story holds several versions |
|
||||
| 409 | `target_site_deleted` | `POST /v1/article/`: the target site was deleted after it was looked up and before the article was stored; choose another site |
|
||||
| 409 | `target_site_deleted` | `POST /v1/article/`: the target site was deleted after it was looked up and before the article was stored; nothing is stored, choose another site and retry. The SPA has no copy for it yet |
|
||||
| 409 | `story_busy` | the database could not lock the rows in time (lock timeout, deadlock or serialization failure); retry |
|
||||
| 422 | `invalid_language` | the tag is not a language tag |
|
||||
| 422 | `unsupported_language` | a valid tag outside the supported list |
|
||||
@@ -63,11 +63,15 @@ Added to the article object of `GET /v1/article/{id}`, `POST`, `PATCH`, `publish
|
||||
"revision": 3,
|
||||
"capabilities": ["read", "edit", "publish", "delete", "create_translation"],
|
||||
"translations": [
|
||||
{"id": 10, "lang": "en", "title": "Renovation costs", "is_online": true, "language_enabled": true, "capabilities": ["read", "edit", "publish", "delete", "create_translation"]},
|
||||
{"id": 13, "lang": "ar", "title": "Takalif", "is_online": true, "language_enabled": true, "capabilities": ["read", "create_translation"]},
|
||||
{"id": 12, "lang": "fr", "title": "Couts", "is_online": false, "language_enabled": false, "capabilities": ["read", "edit", "publish", "delete", "create_translation"]}
|
||||
{"id": 10, "lang": "en", "title": "Renovation costs", "is_online": true, "language_enabled": true, "excluded_from_public": false, "source_article_id": null, "capabilities": ["read", "edit", "publish", "delete", "create_translation"]},
|
||||
{"id": 13, "lang": "ar", "title": "Takalif", "is_online": true, "language_enabled": true, "excluded_from_public": false, "source_article_id": 10, "capabilities": ["read", "create_translation"]},
|
||||
{"id": 12, "lang": "fr", "title": "Couts", "is_online": false, "language_enabled": false, "excluded_from_public": true, "source_article_id": 10, "capabilities": ["read", "edit", "publish", "delete", "create_translation"]}
|
||||
],
|
||||
"source": {"article_id": 10, "revision": 4}
|
||||
"source": {"article_id": 10, "revision": 4},
|
||||
"source_unavailable": false,
|
||||
"missing_langs": ["de"],
|
||||
"excluded_from_public": false,
|
||||
"partial_view": false
|
||||
}
|
||||
```
|
||||
|
||||
@@ -75,6 +79,28 @@ Added to the article object of `GET /v1/article/{id}`, `POST`, `PATCH`, `publish
|
||||
- `capabilities` lists what the caller may do with this version, always in this order and always starting with `read`: `edit` (`content.update` covers it), `publish` (`content.publish`), `delete` (`content.delete`), `create_translation` (the caller holds `content.create`). An anonymous caller gets `["read"]`.
|
||||
- `translations` lists the readable versions of the story, the version being read included, with `id`, `lang`, `title`, `is_online`, `language_enabled` and their own `capabilities`. A version without language (`lang` `""`) reports `language_enabled: true`. The order is the default language of the site first, then the other tags in ascending order (`en`, `ar`, `de`, `fr`); a disabled language is still listed with `language_enabled: false`. A single-version article lists itself. An anonymous caller gets `[]`.
|
||||
- `source` is `{"article_id", "revision"}` when the version was created by copying another version and that source still exists and is readable by the caller, otherwise `null` (a version created empty, a deleted source, an unreadable source, an anonymous caller). `source.revision` is the content revision of the source at the moment of the copy.
|
||||
- `translations[].excluded_from_public` and `excluded_from_public` (the version being read) are `true` when the language keeps the version off the public language surfaces even once published: the language is disabled on the site, or the version has no language yet (`lang` `""`, the site default is not confirmed). The SPA reads this flag instead of re-deriving it from `language_enabled` and the site languages. A `false` flag says nothing about publication: drafts are off the public surfaces anyway.
|
||||
- `translations[].source_article_id` is the id of the version this entry tracks when that source is itself listed in `translations` (so readable), otherwise `null`.
|
||||
- The four keys below are sent to signed-in callers only; an anonymous read has none of them (its shape is unchanged).
|
||||
- `source_unavailable`: see [Versions the caller cannot read](#versions-the-caller-cannot-read).
|
||||
- `missing_langs`: the enabled languages of the site without a readable version in the story, default first then by tag, the same rule as the editorial list (`[]` when the caller cannot create, and `[]` for a story whose only versions have no language).
|
||||
- `excluded_from_public`: the flag above, for the version being read.
|
||||
- `partial_view`: see [Versions the caller cannot read](#versions-the-caller-cannot-read).
|
||||
|
||||
## Versions the caller cannot read
|
||||
|
||||
No answer reveals the existence, id, title or language of a version the caller cannot read. Where the editor needs to explain an absence, the API gives one neutral answer that is the same whether the version was deleted or is hidden from the caller, or a flag that depends on the caller's own permissions only.
|
||||
|
||||
| Need | Answer | Same for |
|
||||
| --- | --- | --- |
|
||||
| The tracked source of a version is gone | `"source": null` and `"source_unavailable": true`. The editor shows "The source isn't available to you" and asks to choose a new source | a deleted source and a source the caller cannot read |
|
||||
| Creating from a source that is gone | `404 source_unavailable` | a deleted, unknown or unreadable source |
|
||||
| Deleting a version that others track ("Review tracking will pause") | the dialog names the entries of `translations` whose `source_article_id` is the id being deleted | trackers the caller cannot read are never listed or counted, because `translations` lists readable versions only and their `source_article_id` is never sent |
|
||||
| Telling the caller that some versions may be out of view ("Versions you can't read aren't listed or counted.") | `partial_view` | `true` whenever the caller's `content.read` is not at scope `all`, whether or not the story holds a version the caller cannot read |
|
||||
| Deleting the whole article while some versions are out of view | `DELETE /v1/story/{story_id}` needs `content.delete` at scope `all` (`403 missing_permission` otherwise, decided before anything is read), deletes every version, readable or not, and answers `204` | a story with or without hidden versions |
|
||||
| Moving or editing a story with hidden versions | `article_ids` and the `X-Missing-Languages` and `X-Conflicting-Articles` headers name readable versions only (see [Moving a story](#moving-a-story-post-v1storystory_idmove)) | |
|
||||
|
||||
`partial_view` never depends on the stored data, so the note "Versions you can't read aren't listed or counted." shows for such a caller on every story, also when nothing is hidden. Showing it only when a hidden version exists would reveal that version.
|
||||
|
||||
## What raises revision
|
||||
|
||||
@@ -129,7 +155,9 @@ The new version joins the story of `{id}`, in the same site, as an offline draft
|
||||
|
||||
Answers `201` with the created article object. Order of checks: permission, path id, body, version readable, `copy` rules, language tag, `setup_required`, source, language configuration. A story deleted in the meantime answers `404 Article not found`. Both entry points leave the stored `lang` normalized.
|
||||
|
||||
### `POST /v1/article/` with `story_id`
|
||||
### `POST /v1/article/` with `story_id` (deprecated)
|
||||
|
||||
Deprecated: use `POST /v1/article/{id}/translations`. Both entry points run the same use case and keep working; new clients should not send `story_id`.
|
||||
|
||||
```json
|
||||
{"title": "Couts", "lang": "fr", "story_id": 5, "copy": true, "source_article_id": 10}
|
||||
@@ -145,7 +173,7 @@ It creates a new story with one offline version, on the default site unless the
|
||||
{"title": "Neuf", "lang": "fr", "site": "atelier"}
|
||||
```
|
||||
|
||||
- `site` is optional: the site key. Absent or `null`, the default site is used exactly as before. A present `site` is validated like any site key (`400` with the plain-text body `invalid_site_key` for `""` or `Bad Key!`), then looked up (`404 site_not_found` for an unknown key, a plain text body, nothing stored). A site deleted after that lookup and before the article is stored answers `409 target_site_deleted` (plain text, nothing stored). Every caller who may create articles may create on every site: `content.create` is not scoped to a site.
|
||||
- `site` is optional: the site key. Absent or `null`, the default site is used exactly as before. A present `site` is validated like any site key (`400` with the plain-text body `invalid_site_key` for `""` or `Bad Key!`), then looked up (`404 site_not_found` for an unknown key, a plain text body, nothing stored). A site deleted after that lookup and before the article is stored answers `409 target_site_deleted` (plain text, nothing stored). Every caller who may create articles may create on every site: `content.create` is not scoped to a site, because a role grants each permission at a scope of `none`, `own` or `all` and has no per-site scope (see [Per-site permissions](#per-site-permissions)).
|
||||
- `lang` is optional. Absent, the version gets the starting language of the chosen site; present, it is normalized and must be an enabled language of the chosen site (`422 invalid_language`, `unsupported_language`, `language_not_enabled`; `409 language_unavailable` when disabled on that site; `409 setup_required` before that site's default language is confirmed). The same tag can therefore be accepted on one site and refused on another.
|
||||
- The slug is generated from the title and made free within the chosen site and language only.
|
||||
- `site` with `story_id`, `source_article_id` or `copy` answers `400` with the body mismatch message and stores nothing. `POST /v1/article/{id}/translations` has no `site` field: the translation keeps the site of its source and an unknown `site` key in its body is ignored.
|
||||
@@ -156,6 +184,20 @@ It creates a new story with one offline version, on the default site unless the
|
||||
|
||||
`DELETE /v1/article/{id}` removes one version (`204`, empty body) and checks `content.delete` per version (`403 missing_permission`). Other versions and their story remain. Deleting the last version of a story removes the story. Nothing in the answer mentions a sibling. A deleted language can be created again in the same story; a language with a deleted version still counts as in use for removing the language from the site.
|
||||
|
||||
### Deleting the whole article: `DELETE /v1/story/{story_id}`
|
||||
|
||||
Verified session or API key with `content.delete` at scope `all` ("delete permission on every version"). Order of checks: path id (`400` with the uint32 message), `content.delete` held (`403 missing_permission`, `X-Missing-Permission: content.delete`), `content.delete` at scope `all` (`403 missing_permission`, same header; decided from the caller's role only, before any read), at least one readable version (`404 Article not found`, the same answer for an unknown story). Then every version of the story, readable or not, is deleted in one transaction under the locks of its sites, and the story row is removed: `204`, empty body.
|
||||
|
||||
- Atomic: a failure answers `500 Failed to delete article` and removes nothing; every version stays.
|
||||
- Each version is deleted like `DELETE /v1/article/{id}` does it (slug tombstone, soft delete, credits removed), so its slug is free again and its language still counts as in use for removing the language from the site.
|
||||
- Versions that tracked a deleted version read `source_unavailable: true` afterwards.
|
||||
- Activity: one `article.delete` per deleted version, oldest first, with `params.lang` and `params.story_id`. A failure records one failed `article.delete` on the oldest readable version; `403` and `404` record nothing.
|
||||
- A caller whose `content.delete` is at scope `own` deletes versions one by one; deleting the last one removes the story.
|
||||
|
||||
### Per-site permissions
|
||||
|
||||
Roles grant each permission at scope `none`, `own` or `all` and have no site dimension, so `content.create`, `content.delete` and the other content permissions apply to every site. A per-site permission (for example "may create on Journal only") is not supported; it needs a site scope in the role model first.
|
||||
|
||||
## Moving a story: `POST /v1/story/{story_id}/move`
|
||||
|
||||
```json
|
||||
@@ -186,7 +228,7 @@ An opt-in shape of the list for a signed-in, verified caller whose role resolves
|
||||
|
||||
Only versions the caller can read count: they are titled, credited, listed in `coverage`, searched and counted. A hidden version never appears, and a story whose versions are all hidden has no row.
|
||||
|
||||
**Rows.** With no `lang` a row is a story: `total` counts stories. With `lang` a row is the version of that language (`tab=all`), and the whole story's `coverage`, `story_authors` and `missing_langs` still ride on the row. With `tab=missing` rows are stories again: `lang=fr` lists the stories on a site that enables `fr` and that have a readable version but none in `fr`; without `lang` it lists the stories that lack at least one enabled language, and a caller who cannot create (no `content.create`) gets `total` 0. Articles without language (`lang` `""`) are one story each and are never listed as missing a language. A version of the story must satisfy `status`, `author`, `q` and `seo` together to make the story match; `q` ignores case and accents.
|
||||
**Rows.** With no `lang` a row is a story: `total` counts stories. With `lang` a row is the version of that language (`tab=all`), and the whole story's `coverage`, `story_authors` and `missing_langs` still ride on the row. With `tab=missing` rows are stories again: `lang=fr` lists the stories on a site that enables `fr` and that have a readable version with a language but none in `fr`; without `lang` it lists the stories that lack at least one enabled language, and a caller who cannot create (no `content.create`) gets `total` 0. A story whose readable versions all lack a language (`lang` `""`) is never listed as missing a language. A mixed story (a version without language plus at least one version with a language) is listed under Missing like any other: the version without language fills no language. A version of the story must satisfy `status`, `author`, `q` and `seo` together to make the story match; `q` ignores case and accents.
|
||||
|
||||
**Item.** The article list item of the representative version (the one in `lang`, otherwise the site default language version, otherwise the first readable one by language tag), plus:
|
||||
|
||||
@@ -194,14 +236,17 @@ Only versions the caller can read count: they are titled, credited, listed in `c
|
||||
| --- | --- | --- |
|
||||
| `story_updated_at` | RFC 3339 | The latest `updated_at` among the readable versions of the story |
|
||||
| `story_authors` | array of bylines | The union of the bylines of the readable versions, without duplicates |
|
||||
| `coverage` | array | One entry per readable version, in the order of `translations` (default language, other enabled tags, disabled tags): `{ id, lang, title, is_online, language_enabled, capabilities }` |
|
||||
| `missing_langs` | array of tags | Enabled languages without a readable version, default first then by tag; `[]` when the caller cannot create |
|
||||
| `coverage` | array | One entry per readable version, in the order of `translations` (default language, other enabled tags, disabled tags), with the same keys as a `translations` entry: `{ id, lang, title, is_online, language_enabled, excluded_from_public, source_article_id, capabilities }` |
|
||||
| `missing_langs` | array of tags | Enabled languages without a readable version, default first then by tag; `[]` when the caller cannot create or when no readable version has a language |
|
||||
| `excluded_from_public` | boolean | The flag of the representative version (see [Fields added to an article read](#fields-added-to-an-article-read)) |
|
||||
| `capabilities` | array | The capabilities of the representative version |
|
||||
|
||||
**Sort.** The whitelist is the flat one. `updated_at` uses the latest readable version of the story, `created_at` the earliest, `title` and `status` the representative; with `lang` the version itself. Ties end with `story_id` descending.
|
||||
|
||||
**Counts.** `{ all, online, draft, suggestions, missing }`. They follow `lang`, `q`, `author`, `seo` and `site` but never `status` or `tab`, and count the unit of the mode (stories, or versions of the selected language). `online` and `draft` count the units with a readable online or draft version. `missing` is the `total` the not-in view would give under the same filters (0 without `lang` for a caller who cannot create). `suggestions` is the number of stories with a readable version whose `seo_status` is `suggestions`, ignoring every parameter.
|
||||
|
||||
`seo_status`, `seo_suggestions`, the `seo` filter and `counts.suggestions` read the stored SEO status of each version (see [SEO status and suggestions](articles.md#seo-status-and-suggestions)); no request reads the bodies of the visible versions to compute them.
|
||||
|
||||
A repository failure at any step, counts included, answers 500 `Failed to retrieve articles`.
|
||||
|
||||
## Mock-ready examples
|
||||
@@ -213,6 +258,9 @@ Statuses of the new and changed calls. Bodies are plain text unless shown as JSO
|
||||
| `GET /v1/article/10` (see above) | 200 | | article object with `revision`, `capabilities`, `translations`, `source` |
|
||||
| `POST /v1/article/10/translations` `{"lang":"fr","title":"Couts","copy":true}` | 201 | | article object, `lang` `"fr"`, `"is_online": false`, `"revision": 1`, `"source": {"article_id": 10, "revision": 4}` |
|
||||
| `DELETE /v1/article/12` | 204 | | empty |
|
||||
| `DELETE /v1/story/5` by the owner | 204 | | empty |
|
||||
| `DELETE /v1/story/5` by a user holding `content.delete` at `own` | 403 | `X-Missing-Permission: content.delete` | `missing_permission` |
|
||||
| `DELETE /v1/story/5` when the database fails | 500 | | `Failed to delete article` |
|
||||
| `POST /v1/story/5/move` `{"site":"atelier"}` | 200 | | `{"story_id":5,"site":"atelier","site_id":2,"article_ids":[10,12,11,13]}` |
|
||||
| `POST /v1/article/10/translations` `{"lang":"fr","title":"Couts"}` | 400 | | `The request cannot be processed due to a mismatch in the format of the body.` |
|
||||
| `PATCH /v1/article/10` `{"title":"ab"}` | 400 | | `Article title must be between 3 and 200 characters.` |
|
||||
@@ -243,6 +291,8 @@ No new action code. The existing events carry `params.lang` (the language of the
|
||||
|
||||
Contract for `features/localization_public_api_and_surfaces.feature`. Both routes are open to everyone: a `jwt` cookie or an API key is ignored, so a draft is never returned, whoever asks. They sit under `/v1/public` and are read only.
|
||||
|
||||
They are rate limited per client address together with `GET /v1/site/{key}/sitemap`: 120 requests per minute in total, the same budget an API key gets on the authenticated routes. The client address is the connection address, or `X-Real-IP` when the connection comes from one of `TRUSTED_PROXIES`. Over the budget the answer is `429` with `Retry-After` (seconds) and the body `Too many requests. Please try again later.`
|
||||
|
||||
A version is **published** when `is_online` is true, and **eligible** when it is published and its language is enabled on the site. A disabled language stays readable on both routes; only the alternates leave it out.
|
||||
|
||||
### Exact lookup: `GET /v1/public/site/{key}/lang/{lang}/article/{slug}`
|
||||
@@ -308,6 +358,7 @@ Plain text, the body is the code followed by a line feed. `language_disabled` is
|
||||
| 404 | `article_not_found` | exact lookup: no published version for the site, language and slug |
|
||||
| 404 | `language_not_available` | story resolution: no version can be served under the rules above |
|
||||
| 404 | `site_not_found` | unknown site key |
|
||||
| 429 | `Too many requests. Please try again later.` | more than 120 requests in a minute from one client address on the public routes and the sitemap together (`Retry-After` in seconds) |
|
||||
| 500 | `Internal server error` | a repository failure |
|
||||
|
||||
## Resolved SEO and sitemap, language aware
|
||||
@@ -327,7 +378,9 @@ Plain text, the body is the code followed by a line feed. `language_disabled` is
|
||||
|
||||
**Fallbacks in the language of the version.** The title and description chains are unchanged except for the site text: `default_description` is treated as written in the current default language of the site, so it is used only by an unassigned article or a version in that language. For any other language the description stays empty, `fallbacks.description` is `none` and `fallbacks.site_text_skipped` is `true`. Changing the default language changes which versions may use the site text. `title_template`, `site_name` and the image chain stay shared; `cover_image_alt` stays per version.
|
||||
|
||||
**Sitemap.** `GET /v1/site/{key}/sitemap` lists every eligible version once (published, language enabled; unassigned published articles keep their legacy entry). Each item is `{url, modified_at, noindex, lang}`; `url` is the localized address when the site has a pattern, otherwise `base_url/slug`. `X-Total-Count` counts the listed items.
|
||||
**Sitemap.** `GET /v1/site/{key}/sitemap` lists every eligible version once (published, language enabled; unassigned published articles keep their legacy entry). Each item is `{url, modified_at, noindex, lang}`; `url` is the localized address when the site has a pattern, otherwise `base_url/slug`. `X-Total-Count` counts the listed items. The route shares the rate limit of the public language routes.
|
||||
|
||||
**SEO check and status.** `POST /v1/article/{id}/seo/check`, the stored `seo_status` of the lists and the `seo` filter apply the same language rule as the resolved description: the site `default_description` can fill the description (and so avoid `desc_short` or raise `desc_long`) only for a version in the current default language of the site or without language.
|
||||
|
||||
## Media usage
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ Anonymous and unverified callers only see online articles, and an offline articl
|
||||
- `items` is `[]` when empty, never `null`. Each item is the article object below.
|
||||
- Items follow `sort` (default `updated_at:desc`) across the whole list before paging. `title` ignores case, `status` orders on online state (`status:asc` puts drafts first), and every ordering ends with `id` descending so pages never overlap.
|
||||
- `total` is the number of visible articles matching `q` and `status`, before paging. `page` and `size` echo the values applied.
|
||||
- `counts` ignores `q`, `status`, `site`, `seo`, `sort`, `page` and `size`, so tab counts do not move while searching. `suggestions` is the number of visible articles whose `seo_status` is `suggestions` (hidden and ready articles and dismissed codes are not counted); it equals the `total` of `?seo=suggestions` sent with no other filter by the same caller. For anonymous and unverified callers `all` equals `online` and `draft` is `0`, and `status=draft` answers `items: []` and `total: 0`.
|
||||
- `counts` ignores `q`, `status`, `site`, `seo`, `sort`, `page` and `size` (and `suggestions` also ignores `lang` and `author`), so tab counts do not move while searching. `suggestions` is the number of visible articles whose `seo_status` is `suggestions` (hidden and ready articles and dismissed codes are not counted); it equals the `total` of `?seo=suggestions` sent with no other filter by the same caller. For anonymous and unverified callers `all` equals `online` and `draft` is `0`, and `status=draft` answers `items: []` and `total: 0`.
|
||||
- Each item also carries `seo_status` and `seo_suggestions` (see [SEO status and suggestions](#seo-status-and-suggestions)). `seo` filters on the status before paging, so `total` follows it while `counts` do not.
|
||||
- A page beyond the last answers `200` with `items: []` and the real `total`.
|
||||
- An invalid `page`, `size`, `status`, `sort` or an over-long `q` answers `400` with the plain-text message `The request cannot be processed due to a mismatch in the format of the body.` An invalid `format` is reported first.
|
||||
@@ -167,6 +167,10 @@ The fallback chain, first non-empty wins:
|
||||
|
||||
Suggestions come in that order. Codes in `seo_dismissed` are never reported or counted. The status is `hidden` when `noindex` is true, else `suggestions` when at least one suggestion remains, else `ready`. The list items carry `seo_status` and `seo_suggestions` (the count, also for a hidden article).
|
||||
|
||||
The resolved description uses the site `default_description` only for a version in the current default language of its site or without language, so `desc_short` and `desc_long` follow the language of the version, in the check and in the lists alike.
|
||||
|
||||
**Stored status.** Each version stores its `seo_status` and `seo_suggestions`. Every write that can change them clears the stored value in the same statement: an edit of the version (title, body, slug, SEO fields, authors, site), a cover or social image deleted, restored or merged, a credited author deleted, a move to another site (story move or site deletion with reassignment), a language assigned by the setup, a change of the site `default_description`, and a change of the site default language. The lists recompute the cleared versions (and only those) before answering, and the server recomputes them at startup, so `seo_status`, `?seo=` and `counts.suggestions` always reflect the current data without reading every visible body per request. A value computed from data that changed in the meantime is never stored.
|
||||
|
||||
## Markdown output
|
||||
|
||||
Headings, bold, italic, links, bullet and ordered lists, blockquotes, fenced code blocks, rules, images and inline code are converted. Anything else is reduced to its text. `script` and `style` content is removed, and the source HTML is never passed through; the only HTML in the output is the alignment wrapper described below. List indentation attributes are dropped. Links with a scheme other than http, https, mailto or tel are reduced to their text. Characters that markdown would read as formatting (`*`, `_`, `` ` ``, `[`, `]`, `<`, `\`) and line starts that would read as a heading, list, quote or rule (`#`, `-`, `+`, `>`, `1.`) are backslash-escaped, so plain text stays plain text. A line break becomes a backslash followed by a newline.
|
||||
|
||||
@@ -73,7 +73,7 @@ Checks run in this order:
|
||||
| 7 | 409 | `language_in_use` | a removed language still has an article, draft, offline, online or soft-deleted (the display count `version_count` stays non-deleted only) |
|
||||
| any | 409 | `languages_changed` | the database could not lock the site in time because another change is running (lock contention); nothing is applied, reload and retry |
|
||||
|
||||
Entries are checked in list order for step 4. A rejected request changes nothing. Disabling a language never touches its articles, and no article changes `lang`, status or publication. A `social_locale` is stored as `language_REGION` (`fr_fr` becomes `fr_FR`). The language part rule only compares with the tag; matching `Site.Locale` is deferred to slice 6.
|
||||
Entries are checked in list order for step 4. A rejected request changes nothing. Disabling a language never touches its articles, and no article changes `lang`, status or publication. A site always has exactly one default language: the rows are planned with one default and checked again before they are written. Changing the default language re-evaluates the stored SEO status of every article of the site, because the site `default_description` only counts for the default language (see [SEO status and suggestions](articles.md#seo-status-and-suggestions)). A `social_locale` is stored as `language_REGION` (`fr_fr` becomes `fr_FR`). The language part rule only compares with the tag; matching `Site.Locale` is deferred to slice 6.
|
||||
|
||||
## First-time language setup
|
||||
|
||||
|
||||
@@ -94,7 +94,9 @@ The use cases `ListAuthors`, `CreateAuthor`, `CreateMyAuthor`, `UpdateAuthor`, `
|
||||
|
||||
`internal/domain/slug` holds the one slug rule (`Generate`, `WithSuffix`, `IsValid`: lowercase, accents folded, runs of other characters become one dash, 100 characters, a caller-chosen fallback). `internal/domain/article` uses it with the fallback `article` and holds the twelve SEO fields of `Article`, the `Update` fields for them (`Nullable` for the two image ids), `ValidateCanonicalURL`, the eleven dismiss codes and `NormalizeDismissed`, and the sentinels `ErrInvalidSlug`, `ErrSlugTaken` and `ErrInvalidCanonicalURL`. `*SlugTakenError` wraps `ErrSlugTaken` and carries the suggested free slug; the handler turns it into the `X-Slug-Suggestion` header of the 409. `author.GenerateSlug` delegates to the same package.
|
||||
|
||||
Slugs are unique per site and language, enforced by the unique index `idx_articles_site_slug` on `(site_id, lang, slug)` (the name is kept from the earlier `(site_id, slug)` index, which the migration replaces once, tracked by the `article_slug_index_per_language` marker). Each article belongs to a story (`stories` table) and a second unique index `idx_articles_story_lang` on `(story_id, lang)` allows one version per language per story; both indexes are created after the `article_stories` backfill marker, which gives every pre-existing article its own story. `site_languages` holds one default language per site, seeded once from `Site.Locale` as an unconfirmed suggestion (`site_language_suggestions` marker). `CreateArticle` owns the rule that a new article takes the confirmed default: it runs through `transaction.Transactor`, asks `language.Repository.LockSiteLanguages` (which locks the site row and returns the site's language rows), lets `language.StartingTag` pick the confirmed default tag (or `""` while it is unconfirmed), picks the free slug for that language and calls `Create` with the resulting `Lang`, all in one transaction. When the site row is gone by then (deleted after the use case looked it up), the use case answers `article.ErrTargetSiteDeleted` (`409 target_site_deleted`) instead of the lookup's `404`. `ArticleRepository.Create` only stores what it receives, checks that a given story row exists and opens the new story in the same transaction (the story rolls back with the article, `content_revision` starts at 1). `ConfirmDefaultLanguageUseCase` (lock, read the rows, `language.DefaultOf`, refuse with `ErrNoDefaultLanguage`, flag it confirmed through `MarkConfirmed`, assign unassigned articles) holds the same site lock, so no article is left unassigned once the confirmation commits; it is not wired to a route (the HTTP setup goes through `ApplyLanguageSetupUseCase`, which takes the site locks with `NOWAIT`). `article.Repository.SlugTaken` takes the language, matching the index. The use cases decide first: `CreateArticle` generates the slug from the title and takes the first free `-N` on the default site (retrying when the repository reports a race), `UpdateArticle` re-checks the slug on the target site for an edited slug, an empty slug (regenerated from the title) and a move, and computes the suggestion as the first free `-N` from 2, fitting 100 characters. `ArticleRepository` locks the site row and leaves uniqueness to the two indexes: each driver says whether an error is a unique violation (`Driver.IsUniqueViolation`: SQLite extended codes `2067` and `1555`, MySQL `1062` and `1586`, PostgreSQL `23505`, behind `database.IsUniqueViolation`), and the repository then re-reads which rule was hit and returns `ErrSlugTaken` or `ErrLanguageTaken`. `PATCH /v1/article/{id}` is one atomic edit: when it moves a story (the caller reads one version of a larger story), `UpdateArticleUseCase` depends on the `StoryMover` port and runs the move of the other versions and its own field edit in the single transaction of `MoveStoryUseCase.MoveStoryThen`, so a refusal after the move step moves nobody and changes no revision. Validation that needs no lock (authors, SEO fields, slug rules and the slug of the target site) runs before the transaction. Row locks are taken site first (ascending ids), then article rows (`ArticleRepository.Update` finds the site with a read outside its transaction, because a read inside would open the snapshot that MariaDB 11 then holds against the locked row, and re-checks the site after locking the article), in `ArticleRepository.Update`, `Create`, `Delete`, `CreateTranslation` and the story move, so a slug edit cannot deadlock a move; the move re-reads and re-locks until the set of sites of the story is stable. Publish and `UpdateBody` are targeted `UPDATE`s of their own columns (plus `updated_at`), so they never overwrite a concurrent edit and never touch the revisions. Driver lock errors (PostgreSQL `55P03`, `40P01` and `40001`, MySQL `3572`, `1205` and `1213`, MariaDB `1020` raised by its snapshot isolation) all become `transaction.ErrLockContention` (`409 story_busy`). SQLite ignores `FOR UPDATE`: there the single writer plus the compare-and-set on `edit_revision` in `writeArticleColumns` are what keep concurrent edits correct, not the row locks. Deleting an article replaces its slug with the tombstone `#deleted-<id>` (it can never match a valid slug) before the soft delete, so the slug is free again while the unique index keeps covering soft-deleted rows.
|
||||
Slugs are unique per site and language, enforced by the unique index `idx_articles_site_slug` on `(site_id, lang, slug)` (the name is kept from the earlier `(site_id, slug)` index, which the migration replaces once, tracked by the `article_slug_index_per_language` marker). Each article belongs to a story (`stories` table) and a second unique index `idx_articles_story_lang` on `(story_id, lang)` allows one version per language per story; both indexes are created after the `article_stories` backfill marker, which gives every pre-existing article its own story. `site_languages` holds one default language per site, seeded once from `Site.Locale` as an unconfirmed suggestion (`site_language_suggestions` marker). `CreateArticle` owns the rule that a new article takes the confirmed default: it runs through `transaction.Transactor`, locks the site row (`site.Repository.LockSite`), reads the site's language rows through the `localization.Repository` port, lets `localization.ConfirmedDefaultTag` pick the confirmed default tag (or `""` while it is unconfirmed), picks the free slug for that language and calls `Create` with the resulting `Lang`, all in one transaction. `localization.Repository` is the only port for site languages; the former `language.Repository` and its `SiteLanguageRepository` adapter are gone. When the site row is gone by then (deleted after the use case looked it up), the use case answers `article.ErrTargetSiteDeleted` (`409 target_site_deleted`) instead of the lookup's `404`. `ArticleRepository.Create` only stores what it receives, checks that a given story row exists and opens the new story in the same transaction (the story rolls back with the article, `content_revision` starts at 1). `ApplyLanguageSetupUseCase` confirms a default and assigns the unassigned articles under the same site locks (taken with `NOWAIT`), so no article is left unassigned once the confirmation commits. The one-default-per-site invariant lives in the domain (`localization.Plan` builds rows with one default, `localization.RequireSingleDefault` checks it before `UpdateSiteLanguagesUseCase` writes); `LocalizationRepository.Replace` only writes the rows it receives. `article.Repository.SlugTaken` takes the language, matching the index. The use cases decide first: `CreateArticle` generates the slug from the title and takes the first free `-N` on the default site (retrying when the repository reports a race), `UpdateArticle` re-checks the slug on the target site for an edited slug, an empty slug (regenerated from the title) and a move, and computes the suggestion as the first free `-N` from 2, fitting 100 characters. `ArticleRepository` locks the site row and leaves uniqueness to the two indexes: each driver says whether an error is a unique violation (`Driver.IsUniqueViolation`: SQLite extended codes `2067` and `1555`, MySQL `1062` and `1586`, PostgreSQL `23505`, behind `database.IsUniqueViolation`), and the repository then re-reads which rule was hit and returns `ErrSlugTaken` or `ErrLanguageTaken`. `PATCH /v1/article/{id}` is one atomic edit: when it moves a story (the caller reads one version of a larger story), `UpdateArticleUseCase` depends on the `StoryMover` port and runs the move of the other versions and its own field edit in the single transaction of `MoveStoryUseCase.MoveStoryThen`, so a refusal after the move step moves nobody and changes no revision. Validation that needs no lock (authors, SEO fields, slug rules and the slug of the target site) runs before the transaction. Row locks are taken site first (ascending ids), then article rows (`ArticleRepository.Update` finds the site with a read outside its transaction, because a read inside would open the snapshot that MariaDB 11 then holds against the locked row, and re-checks the site after locking the article), in `ArticleRepository.Update`, `Create`, `Delete`, `CreateTranslation` and the story move, so a slug edit cannot deadlock a move; the move re-reads and re-locks until the set of sites of the story is stable. Publish and `UpdateBody` are targeted `UPDATE`s of their own columns (plus `updated_at`), so they never overwrite a concurrent edit and never touch the revisions. Driver lock errors (PostgreSQL `55P03`, `40P01` and `40001`, MySQL `3572`, `1205` and `1213`, MariaDB `1020` raised by its snapshot isolation) all become `transaction.ErrLockContention` (`409 story_busy`). SQLite ignores `FOR UPDATE`: there the single writer plus the compare-and-set on `edit_revision` in `writeArticleColumns` are what keep concurrent edits correct, not the row locks. Deleting an article replaces its slug with the tombstone `#deleted-<id>` (it can never match a valid slug) before the soft delete, so the slug is free again while the unique index keeps covering soft-deleted rows. `DeleteStoryUseCase` (`DELETE /v1/story/{story_id}`) refuses a caller whose `content.delete` is not at scope `all` before reading anything, locks every site of the story (re-reading until the set is stabLine truncated
|
||||
|
||||
The article persistence is split into ports by use: `article.Repository` (single versions and writes), `article.StoryReader` (the editorial list by story: pages, counts, versions of a page) and `article.SEOIndex` (the stored SEO status: `StaleSEO`, `StoreSEO`, `MarkSiteSEOStale`). `article.ListingRepository` joins the three for `ListArticlesUseCase`, the only use case that needs all of them; `ArticleRepository` implements every port, and `transaction.Repositories` exposes `ArticleSEO` so a transaction that changes the default language can clear the statuses of its site. The rules that compute the status stay in the use cases (they parse HTML through `golang.org/x/net/html`), so the repositories only clear and store it; see [performance.md](performance.md#the-seo-article-filter).
|
||||
|
||||
`database.Migrate` first sets `source_revision` to `0` where an older nullable column holds `NULL` (`BackfillNullSourceRevisions`, a no-op once no `NULL` is left), so `AutoMigrate` can make it `NOT NULL`; it then adds the columns with `AutoMigrate`, then runs `BackfillSiteTitleTemplates` and `BackfillArticleSlugs` (only rows with an empty slug or a duplicate within the same site and language, in id order, per site and language, applying the same rule and the same suffixes, with the tombstone for soft-deleted rows) and only then creates the unique index, so it is idempotent and completes a partly backfilled table. The default `title_template` is written by the backfill and the seed, not by a column default, because the SQLite migrator rebuilds the table on every start when the default contains `%` and `|`.
|
||||
|
||||
|
||||
@@ -32,6 +32,10 @@ They belong to the MCP binary only and are not server variables.
|
||||
|
||||
Routes that need a browser session, such as creating a site, are not exposed. API errors come back as tool errors with the status and the plain-text message.
|
||||
|
||||
## Why it calls the HTTP API
|
||||
|
||||
`renewcms-mcp` runs on the machine of the person using the model (Claude Desktop, Cursor), usually not on the server, and talks to a remote instance. Calling the use cases directly would need the database, the storage and the secrets of the instance in that process, and would skip what the HTTP layer enforces for every client: API-key authentication and its rate limit, the role permissions checked by the router middleware, the activity context, validation of the request bodies and the plain-text error contract. Going through the published routes keeps the model inside the same limits as any API key and keeps one contract to test. It uses the authenticated routes (`/v1/article`, `/v1/media`, `/v1/site`), not the public `/v1/public` routes, so drafts stay reachable for the key's user. Running it in-process with the server is not supported.
|
||||
|
||||
## Resources and prompts
|
||||
|
||||
The resource `renewcms://docs/llms-full.txt` is the instance's own [`/llms-full.txt`](llms.md), so the model reads the documentation of the exact version it talks to. The prompt `write_and_publish_article` drafts, checks the SEO and asks before publishing.
|
||||
+7
-6
@@ -2,17 +2,18 @@
|
||||
|
||||
## The `?seo=` article filter
|
||||
|
||||
`GET /v1/article/` assesses the SEO status of articles in memory, because the status depends on the body, the site template and the authors. Every request assesses every visible article once, to fill `counts.suggestions`, and `?seo=` reuses that assessment to filter before paging. The cost therefore grows with the number of articles, not with the page size.
|
||||
The SEO status of an article depends on its body, its fields, its authors, its site (`default_description`) and the site default language. It is computed by `RefreshArticleSEOUseCase` and stored on the article (`seo_status`, `seo_suggestions`). Every repository write to one of those inputs clears the stored status and raises `seo_version` in the same `UPDATE` (`withStaleSEO`); site-level changes (default description, default language) clear the whole site through `article.SEOIndex.MarkSiteSEOStale`. Before answering, `GET /v1/article/` recomputes the cleared articles only, in batches of 500, reading their bodies once; `StoreSEO` writes a status only while `seo_version` is still the one it read, so a concurrent edit is never overwritten by an older assessment; the site and its languages are read after each batch of stale rows, never cached across batches, so a site-level change that commits during a refresh is either seen or makes the store miss. The server recomputes everything that is cleared at startup (the backfill of existing rows after the upgrade). `?seo=` and `counts.suggestions` are then plain SQL conditions on `seo_status`, so the cost of a list no longer grows with the number of visible bodies.
|
||||
|
||||
`TestSeoFilterPerformance` in `test/seo_filter_performance_test.go` (skipped with `-short`) measures 5,000 articles of about 5 KB each in one site on a temporary SQLite file, every article with suggestions, slowest of 3 runs:
|
||||
`TestSeoFilterPerformance` in `test/seo_filter_performance_test.go` (skipped with `-short`) measures 5,000 articles of about 5 KB each in one site on a temporary SQLite file, every article with suggestions, slowest of 3 runs after the startup backfill:
|
||||
|
||||
| Request | Time |
|
||||
| --- | --- |
|
||||
| `?seo=suggestions&size=25` | 315 to 338 ms |
|
||||
| default page, no `?seo=` | 307 to 329 ms |
|
||||
| startup backfill of the 5,000 articles (once) | 270 ms |
|
||||
| `?seo=suggestions&size=25` | 24 ms |
|
||||
| default page, no `?seo=` | 23 ms |
|
||||
|
||||
The target is 300 ms for 5,000 articles. Both requests sit slightly above it on the development machine, so the test logs the target but only fails above 500 ms; no cache or index was added. Re-run it with `go test -tags nofrontend ./test/... -run TestSeoFilterPerformance -v -count=1` after a change to the assessment rules or the article repository.
|
||||
The target is 300 ms for 5,000 articles; the test fails above 500 ms. Re-run it with `go test -tags nofrontend ./test/... -run TestSeoFilterPerformance -v -count=1` after a change to the assessment rules or the article repository.
|
||||
|
||||
## The `?seo=` filter of the grouped list
|
||||
|
||||
`GET /v1/article/?group=story` uses the same assessment. The versions whose status matches (and, with `q`, whose title matches: the title search runs once per request through `article.Repository.FindTitleMatches`) are handed to the repository as `StoryFilters.VersionIDs`. That set can be larger than any bind-parameter limit (SQLite accepts 32,766 parameters), so it never becomes an `IN (?, …)` list: the repository reads the qualifying `(story_id, id)` pairs and the ordered story rows with id-only projections and keeps the members of the set in memory before paging. `ListByStories` reads the versions of the page without their bodies and only those the caller may read; the representative's body comes from the assessment pass, which still reads every visible body because the status depends on it. `TestArticleStoryListSEOFilterAtScaleSQLite` covers 40,001 versions.
|
||||
`GET /v1/article/?group=story` filters on the stored status in SQL (`q.seo_status = ?` among the conditions of the qualifying versions) and counts `suggestions` with `COUNT(DISTINCT story_id)` over the readable versions, so no id list is built for the SEO filter. With `q`, the title search still runs once per request through `article.StoryReader.FindTitleMatches` (accent folding happens in Go) and its matches are handed to the repository as `StoryFilters.VersionIDs`. That set can be larger than any bind-parameter limit (SQLite accepts 32,766 parameters), so it never becomes an `IN (?, …)` list: the repository reads the qualifying `(story_id, id)` pairs and the ordered story rows with id-only projections and keeps the members of the set in memory before paging. `ListByStories` reads the versions of the page without their bodies and only those the caller may read; `ListVersions` reads the bodies of the page's representatives only. `TestArticleStoryListSEOFilterAtScaleSQLite` covers 40,001 versions: the backfill assesses each of them once, and the list requests that follow read no stale version and no page of bodies.
|
||||
Reference in new issue
Block a user