feat: SEO backend data (1/4)

2026-10-15 19:30:29 +02:00
1 parent cb1410f0b0
commit 0eb46b0d74
@@ -74,8 +74,8 @@ never changes the response or the change itself.

 Target types: `article`, `site`, `author`, `image`, `image_variant` (id and name are the variant name), `user`, `invite`,
 `api_key` and `release` (id and name are the tag).
-Only the fields `title`, `online`, `body`, `site`, `name`, `base_url`, `default_robots`, `locale`, `default_social_image`, `slug`, `bio`, `website`, `social_links`, `avatar`, `user`, `authors`, `filename`, `display_name`, `width`, `height`, `fit`, `format` and `quality` appear in `changes` (a site key is never a change; moving an article records `site` with the site names; changing the credited authors records `authors` with the names in order; the `user` change of an author holds usernames, never an email), with the
-kinds `text` and `body`; a body edit carries a signed character count delta and never the text. A
+Only the fields `title`, `online`, `body`, `site`, `name`, `base_url`, `default_robots`, `locale`, `default_social_image`, `slug`, `canonical_url`, `noindex`, `cover_image`, `social_image`, `excerpt`, `seo_title`, `seo_description`, `social_title`, `social_description`, `cover_image_alt`, `site_name`, `title_template`, `default_description`, `bio`, `website`, `social_links`, `avatar`, `user`, `authors`, `filename`, `display_name`, `width`, `height`, `fit`, `format` and `quality` appear in `changes` (a site key is never a change; moving an article records `site` with the site names; changing the credited authors records `authors` with the names in order; the `user` change of an author holds usernames, never an email), with the
+kinds `text` and `body`; a body edit carries a signed character count delta and never the text; the free-text SEO fields (`excerpt`, `seo_title`, `seo_description`, `social_title`, `social_description`, `cover_image_alt`) do the same under their own field name, while `slug`, `canonical_url`, `noindex`, `cover_image`, `social_image` (image file names) and the site fields `site_name`, `title_template`, `default_description` carry `before` and `after`. `seo_dismissed` is not an edit and records nothing. A
 change that leaves a value as it was (same body, same publish state, same file name, same display
 name) records nothing. Params: `media.merge` and `media.purge` and `system.media_purge` carry
 `merged` and `articles_updated`, plus `failed_groups` when a group failed (still outcome `success`);
@@ -132,33 +132,82 @@ session." with a key). The role is checked before the body.
 | Route | Purpose |
 | --- | --- |
 | `GET /v1/site/` | The sites, default first and then by name, each with `article_count`. |
-| `POST /v1/site/` | Create. 201 with the site. `is_default: true` makes it the default. |
-| `PATCH /v1/site/{id}` | Edit `name`, `base_url`, `default_robots`, `locale`, `default_social_image_id`. Absent fields are untouched. `key` may only be repeated unchanged; `is_default` is refused. |
+| `POST /v1/site/` | Create. 201 with the site. `is_default: true` makes it the default. `site_name`, `title_template` and `default_description` are optional and validated like on `PATCH`. |
+| `PATCH /v1/site/{id}` | Edit `name`, `base_url`, `default_robots`, `locale`, `default_social_image_id`, `site_name`, `title_template`, `default_description`. Absent fields are untouched. `key` may only be repeated unchanged; `is_default` is refused. |
 | `PATCH /v1/site/{id}/default` | Make the site the default, atomically. Setting the current default is a no-op. |
 | `DELETE /v1/site/{id}?reassign_to={id}` | 204. A site with articles needs `reassign_to`; the articles move and the site is deleted in one transaction. `reassign_to` is ignored when the site has no article. |

 Errors are plain text whose body is exactly the token: 409 `site_key_taken`, 409 `site_has_articles`,
 403 `default_site` (checked before the articles), 400 `key_immutable`, 400 `invalid_site_url`, 400
-`invalid_locale`, 400 `image_not_found`, 400 `use_set_default`, 400 `invalid_reassign` (missing
+`invalid_locale`, 400 `invalid_title_template`, 400 `image_not_found`, 400 `use_set_default`, 400 `invalid_reassign` (missing
 number, the site itself or not a whole number) and 404 `site_not_found`. Refused writes except `site_not_found` and `image_not_found` are recorded as failed events. A body that cannot be
 decoded is 400 with the body-mismatch message, a malformed id is 400 with the uint32 message, and a
-field that fails validation (`name`, `key` format, `default_robots`) is 400 with the validator text.
+field that fails validation (`name`, `key` format, `default_robots`, `site_name` over 100 characters, `title_template` over 200, `default_description` over 320) is 400 with the validator text.
+
+A site also carries `site_name` (at most 100 characters, `""` by default), `title_template` (at most 200 characters, `%title% | %site%` by default; it must contain `%title%`, case sensitive, otherwise 400 `invalid_title_template`, an empty template included) and `default_description` (at most 320 characters, `""` by default). They appear on every site response, `POST` accepts them and `PATCH` edits them like the other site fields. Existing sites get the defaults when the server starts after the upgrade.

 Articles carry `site_id`. A new article starts on the default site. `GET /v1/article/?site=<key>`
 keeps only the articles of that site and composes with `status`, `q`, `sort`, `size` and `page`;
 `counts` stay global, an unknown key answers 200 with no item and an empty `site=` is 400 with the
-body-mismatch message. `PATCH /v1/article/{id}` takes `body` and `site_id` (an
-API key may send it too) and nothing else, such as `title`: absent fields are untouched, an empty object, an object without
-those fields, a `null` value or a malformed value is 400 with the body-mismatch message, `site_id: 0` is
+body-mismatch message. `PATCH /v1/article/{id}` takes `body`, `site_id`, `author_ids` and the SEO
+fields below, and nothing else, such as `title`: absent fields are untouched, an empty object, an object without
+those fields, a `null` value (except on `cover_image_id` and `social_image_id`) or a malformed value is 400 with the body-mismatch message, `site_id: 0` is
 400 with the validator text and an unknown site is 404 `site_not_found`. Moving an article never
 changes `is_online`. `{"body": ""}` still sets an empty body, but `{}` and `{"body": null}` no longer do: they are 400.

+### Article SEO fields
+
+An article carries twelve search and sharing fields, all edited through `PATCH /v1/article/{id}` and returned by
+every single-article read (`GET /v1/article/{id}`, `POST`, `PATCH`, `publish`, `unpublish`):
+`slug`, `excerpt`, `seo_title`, `seo_description`, `cover_image_id`, `cover_image_alt`, `social_title`,
+`social_description`, `social_image_id`, `canonical_url`, `noindex` and `seo_dismissed`. Text fields are `""`
+when empty, the two image ids are `null` when unset, `noindex` is `false` and `seo_dismissed` is `[]` by
+default. Each item of `GET /v1/article/` carries `excerpt` and `slug` and none of the other ten.
+The limits are counted in characters: `excerpt` 300, `seo_title` 120, `seo_description` 320,
+`cover_image_alt` 200, `social_title` 120, `social_description` 320, `canonical_url` 2048; going over is
+400 with the validator text. `cover_image_id` and `social_image_id` accept `null` (and only those two do);
+`0` is 400 with the validator text and an unknown image is 400 `image_not_found`. `canonical_url` is `""` or an absolute
+`https` URL with a host, otherwise 422 `invalid_url`. `seo_dismissed` is a list of the codes `title_long`,
+`title_empty`, `desc_short`, `desc_long`, `cover_alt`, `cover_none`, `slug_long`, `no_au`, `no_h`, `body_alt`
+and `short`: an unknown code is 400 with the validator text and duplicates are collapsed with the order
+kept. `noindex` is only a flag, it does not take the article offline.
+
+The `slug` matches `^[a-z0-9]+(-[a-z0-9]+)*$` with 1 to 100 characters (400 `invalid_slug`) and is unique per
+site, never globally. It is generated from the title when the article is created: lowercase, accents folded,
+every run of other characters becomes one dash, dashes trimmed, cut to 100 characters, and `article` when no
+letter or digit is left; a slug already used on the site gets `-2`, `-3`... silently. The same rule
+backfills the existing articles when the server starts after the upgrade (in id order within each site, so
+the oldest article keeps the plain slug). Sending `slug: ""` regenerates the slug from the title. A slug
+used by another article on the target site (the new site when `site_id` is sent too, and a move re-checks
+the current slug) is 409 with the plain-text body `slug_taken`; the response then carries the header
+`X-Slug-Suggestion` holding the first free `<slug>-N` from 2 on that site, fitting 100 characters. The header
+is set on this 409 only, never on another response. A regeneration that collides is refused the same way, never suffixed
+silently. Saving an article with its own slug is not a collision. Publishing never changes a slug, a deleted article
+frees its slug.
+
+New error tokens are plain text from the handler: 409 `slug_taken`, 400 `invalid_slug`, 422 `invalid_url`, 400
+`image_not_found` and 400 `invalid_title_template` (sites). Edits that change a SEO field are recorded as
+`article.edit` events (`slug`, `canonical_url`, `noindex`, `cover_image`, `social_image` with before and after;
+the free-text fields as a size delta only); a refused slug is recorded as failed.
+
 A site's `default_social_image_id` counts as a use of that image: it raises `used_in` and keeps the
-image out of the `unused` filter, while `GET /v1/media/{id}/usage` keeps listing articles only.
-Deleting the image sets the field to `null` and succeeds. `POST /v1/media/merge` and `POST
+image out of the `unused` filter. Deleting the image sets the field to `null` and succeeds. `POST /v1/media/merge` and `POST
 /v1/media/purge` repoint the sites of the merged copies to the kept image and report it as
 `sites_updated` next to `articles_updated`.

+An article's `cover_image_id` and `social_image_id` count the same way. `used_in` counts every article that uses the
+image in its body, as cover or as social image once, plus the authors using it as an avatar and the sites using it as default
+social image. Deleting an image never fails because of its uses: the covers and social images pointing at it are set to `null`
+(the `cover_image_alt` and the bodies are untouched), and a failed delete leaves every reference in place. A merge or a purge repoints
+the covers and social images of the copies to the kept image, and an article rewritten in its body and its cover counts once in
+`articles_updated`.
+
+`GET /v1/media/{id}/usage` stays an array. Every item keeps the keys it had and gains `type` (`article`,
+`author` or `site`) and `fields`. An article item is `{ "type": "article", "id", "title", "published", "fields" }` with
+`fields` from `body`, `cover` and `social` (one item per article, in that order); an author item is
+`{ "type": "author", "id", "title": <name>, "fields": ["avatar"] }` and a site item is `{ "type": "site", "id", "title": <name>,
+"fields": ["default_social_image"] }`; author and site items have no `published` key. The array is `[]` for an unused image.
+
 ## Authors

 An author is a public byline, with or without an account. It is `{ id, name, slug, bio, avatar_id,
@@ -203,8 +252,8 @@ creator's linked author when there is one. `GET /v1/auth/me` returns the existin
 `social_links`, `article_count`) or `null`.

 An author's `avatar_id` counts as a use of that image like a site's social image: it raises `used_in`
-and keeps the image out of the `unused` filter, while `GET /v1/media/{id}/usage` keeps listing articles
-only. Deleting the image sets `avatar_id` to `null`. `POST /v1/media/merge` and `POST /v1/media/purge`
+and keeps the image out of the `unused` filter, and `GET /v1/media/{id}/usage` lists the author with the field `avatar`.
+Deleting the image sets `avatar_id` to `null`. `POST /v1/media/merge` and `POST /v1/media/purge`
 repoint the avatars of the merged copies and report it as `authors_updated`.

 ## Image variants
@@ -2,7 +2,7 @@

 Every route that returns an article accepts the optional `format` query parameter and returns the article body in that format. Articles are stored as HTML (Tiptap output). Storage and the request bodies of `POST /v1/article/` and `PATCH /v1/article/{id}` stay HTML.

-`PATCH /v1/article/{id}` takes `body`, `site_id` and `author_ids` (the whole ordered list of author ids, see `docs/api.md`), and nothing else. The request must contain at least one of them, and `null` for either answers `400`. Behaviour change: `{}` and `{"body": null}` used to set an empty body and now answer `400`; send `{"body": ""}` to empty a body.
+`PATCH /v1/article/{id}` takes `body`, `site_id`, `author_ids` (the whole ordered list of author ids, see `docs/api.md`) and the SEO fields listed below, and nothing else. The request must contain at least one of them, and `null` answers `400` except for `cover_image_id` and `social_image_id`. Behaviour change: `{}` and `{"body": null}` used to set an empty body and now answer `400`; send `{"body": ""}` to empty a body.

 ## Routes that honour `format`

@@ -71,11 +71,52 @@ Breaking change: the default used to be HTML. Clients that need HTML must send `
  "is_online": true,
  "site_id": 1,
  "authors": [],
+  "slug": "backyard-renovation-update",
+  "excerpt": "",
+  "seo_title": "",
+  "seo_description": "",
+  "cover_image_id": null,
+  "cover_image_alt": "",
+  "social_title": "",
+  "social_description": "",
+  "social_image_id": null,
+  "canonical_url": "",
+  "noindex": false,
+  "seo_dismissed": [],
  "created_at": "2026-03-12T09:14:00Z",
  "updated_at": "2026-09-10T07:00:00Z"
 }
 ```

+Every single-article read (`GET /v1/article/{id}`, `POST`, `PATCH`, `publish`, `unpublish`) carries the twelve SEO fields. Each item of `GET /v1/article/` carries `slug` and `excerpt` only, and none of the other ten.
+
+## SEO fields
+
+| Field | Type | Limit and rule |
+| --- | --- | --- |
+| `slug` | string | `^[a-z0-9]+(-[a-z0-9]+)*$`, 1 to 100 characters, unique per site. `""` regenerates it from the title. |
+| `excerpt` | string | 300 characters |
+| `seo_title` | string | 120 characters |
+| `seo_description` | string | 320 characters |
+| `cover_image_id` | number or `null` | an existing image id (`0` is refused) |
+| `cover_image_alt` | string | 200 characters, independent of the cover |
+| `social_title` | string | 120 characters |
+| `social_description` | string | 320 characters |
+| `social_image_id` | number or `null` | an existing image id (`0` is refused) |
+| `canonical_url` | string | `""` or an absolute `https` URL with a host, at most 2048 characters |
+| `noindex` | boolean | only a flag: the article stays online |
+| `seo_dismissed` | array of strings | any of `title_long`, `title_empty`, `desc_short`, `desc_long`, `cover_alt`, `cover_none`, `slug_long`, `no_au`, `no_h`, `body_alt`, `short`; duplicates are collapsed, order kept |
+
+Limits are counted in characters (runes), not bytes. Absent fields are untouched; text fields and `noindex` are cleared with `""`, `false` and `[]`, the two image ids with `null`.
+
+### The slug
+
+A slug is unique per site (`site_id`, `slug`), never globally. At creation it is generated from the title: lowercase, accents folded, every run of other characters becomes one dash, dashes trimmed, cut to 100 characters without a trailing dash, and `article` when no letter or digit is left. A slug already used on the site gets `-2`, `-3`... silently, the base being truncated so the result fits 100 characters. The same rule backfills existing articles on the first start after the upgrade, in id order within each site.
+
+Editing a slug that another article of the target site already uses (the new site when `site_id` is sent in the same request; moving an article also re-checks its current slug there) answers `409` with the plain-text body `slug_taken`, and the response header `X-Slug-Suggestion` holds the first free `<slug>-N` from 2 on that site, fitting 100 characters. `X-Slug-Suggestion` is set on this `409` only and on no other response. A request with `slug: ""` that regenerates onto another article's slug is refused the same way and never suffixed silently. Re-sending an article's own slug is not a conflict. A refused request changes nothing.
+
+Publishing, unpublishing and `noindex` never change a slug. A deleted article frees its slug.
+
 ## Markdown output

 Headings, bold, italic, links, bullet and ordered lists, blockquotes, fenced code blocks, rules, images and inline code are converted. Anything else is reduced to its text. `script` and `style` content is removed, and the source HTML is never passed through; the only HTML in the output is the alignment wrapper described below. List indentation attributes are dropped. Links with a scheme other than http, https, mailto or tel are reduced to their text. Characters that markdown would read as formatting (`*`, `_`, `` ` ``, `[`, `]`, `<`, `\`) and line starts that would read as a heading, list, quote or rule (`#`, `-`, `+`, `>`, `1.`) are backslash-escaped, so plain text stays plain text. A line break becomes a backslash followed by a newline.
@@ -129,5 +170,8 @@ Example for `<h2>Plan</h2><p>Replace the <strong>old</strong> deck.</p>`:
 | --- | --- |
 | `400` | `The server expects the ID to be in the format of an unsigned 32-bit integer (uint32).` for an invalid id |
 | `400` | `The format must be one of: markdown, html, json.` for an unsupported format |
+| `400` | `invalid_slug`, `image_not_found` or the validator text for an over-long SEO field, `cover_image_id: 0` or an unknown `seo_dismissed` code |
 | `404` | `Article not found` |
+| `409` | `slug_taken`, with the header `X-Slug-Suggestion: <first free slug>` (this response only) |
+| `422` | `invalid_url` for a `canonical_url` that is not an absolute `https` URL with a host |
 | `500` | `Internal server error` when the body cannot be converted |
@@ -81,6 +81,16 @@ The use cases `ListSites`, `CreateSite`, `UpdateSite`, `SetDefaultSite` and `Del

 The use cases `ListAuthors`, `CreateAuthor`, `CreateMyAuthor`, `UpdateAuthor`, `DeleteAuthor` and `GetMyAuthor` live in `internal/infrastructure/useCases`; `CreateArticle` credits the creator's author, `UpdateArticle` validates and records `author_ids`, and the media, purge and activity use cases take the author repository as a required constructor argument (avatars count as media usage, are cleared on delete and repointed on merge and purge). `api/handlers/author` serves `/v1/author`: the list accepts a session or an API key, every write is session only, `POST /` and `DELETE` sit behind `RequireOwnerOrAdmin`, and the edit rights (owner or admin, or the caller's own linked author) are decided by `UpdateAuthor` from the caller the handler passes in. `GET /v1/auth/me` adds the linked author through `GetMyAuthor`.

+### Article SEO data
+
+`internal/domain/slug` holds the one slug rule (`Generate`, `WithSuffix`, `IsValid`: lowercase, accents folded, runs of other characters become one dash, 100 characters, a caller-chosen fallback). `internal/domain/article` uses it with the fallback `article` and holds the twelve SEO fields of `Article`, the `Update` fields for them (`Nullable` for the two image ids), `ValidateCanonicalURL`, the eleven dismiss codes and `NormalizeDismissed`, and the sentinels `ErrInvalidSlug`, `ErrSlugTaken` and `ErrInvalidCanonicalURL`. `*SlugTakenError` wraps `ErrSlugTaken` and carries the suggested free slug; the handler turns it into the `X-Slug-Suggestion` header of the 409. `author.GenerateSlug` delegates to the same package.
+
+Slugs are unique per site, enforced by the unique index `idx_articles_site_slug` on `(site_id, slug)`. The use cases decide first: `CreateArticle` generates the slug from the title and takes the first free `-N` on the default site (retrying when the repository reports a race), `UpdateArticle` re-checks the slug on the target site for an edited slug, an empty slug (regenerated from the title) and a move, and computes the suggestion as the first free `-N` from 2, fitting 100 characters. `ArticleRepository` locks the site row, checks the slug inside the transaction and re-explains a failed write as `ErrSlugTaken`. Deleting an article replaces its slug with the tombstone `#deleted-<id>` (it can never match a valid slug) before the soft delete, so the slug is free again while the unique index keeps covering soft-deleted rows.
+
+`database.Migrate` adds the columns with `AutoMigrate`, then runs `BackfillSiteTitleTemplates` and `BackfillArticleSlugs` (only rows with an empty slug, in id order, per site, applying the same rule and the same suffixes, with the tombstone for soft-deleted rows) and only then creates the unique index, so it is idempotent and completes a partly backfilled table. The default `title_template` is written by the backfill and the seed, not by a column default, because the SQLite migrator rebuilds the table on every start when the default contains `%` and `|`.
+
+Media usage has more sources than article bodies: `Article.CoverImageID` and `SocialImageID`, `Author.AvatarID` and `Site.DefaultSocialImageID`. `ListMedia`, the purge plan and `GetMediaUsage` count an article once however many places use the image. `DeleteMedia` clears the cover and social references (`article.Repository.ClearImage`, restored with `RestoreImage` when the delete fails), `MergeMedia` repoints them (`RepointImage`) and counts an article rewritten in its body and its cover once, and `GetMediaUsage` (now built with the site and author repositories) returns articles, authors and sites with `type` and `fields`.
+
 ### Image variants

 `internal/domain/image` holds the `Variant` entity and its rules (`VariantSettings.Normalized`, the settings `Fingerprint`, `VariantETag`, `ETagMatches`, `IsAnimatedGIF`), the sentinels (`ErrVariantNotFound`, `ErrVariantExists`, `ErrVariantInvalid`, `ErrVariantFormatUnavailable`, `ErrUnknownVariant`, `ErrUnsupportedParameter`, `ErrSourceTooLarge`) and three ports: `VariantRepository` (definitions), `VariantFileRepository` (the index of generated files: variant name, source stored name, cached stored name, content type, size and the settings fingerprint, needed because `Storage.Put` chooses the stored name itself) and `VariantRenderer` (decode a source, encode a variant, report the formats it can encode). `internal/infrastructure/imaging` is the renderer adapter, free of GORM and HTTP; the persistence adapters are `VariantRepository` and `VariantFileRepository` over `models.ImageVariant` and `models.ImageVariantFile`.
@@ -15,6 +15,31 @@ Body, in this order. Omit a section only when it would be empty, except Upgradin

 Rules: bullets are full sentences ending with a period; user-facing wording, no commit prefixes; sentence-case headings; no horizontal rules, no metadata block, no "Pre-1.0" boilerplate (the prerelease flag says it); every release ships all 6 binaries from `task release`.

+## Next release (draft)
+
+Articles and sites now store the data a website needs for search and sharing, and every article gets an address that is unique on its site.
+
+### New
+
+**Article SEO fields**
+
+- An article has a slug, an excerpt, a search title and description, a cover image with its alt text, a social title, description and image, a canonical address, a noindex flag and a list of dismissed suggestions.
+- The slug is generated from the title, is unique per site and can be changed at any time; an empty slug regenerates it from the title.
+- A site has a site name, a title template and a default description.
+
+### API
+
+- `PATCH /v1/article/{id}` also takes the SEO fields. `cover_image_id` and `social_image_id` accept `null`. A slug already used on the target site answers `409` `slug_taken` with the header `X-Slug-Suggestion` holding the first free slug.
+- `GET /v1/article/{id}`, `POST /v1/article/`, `PATCH /v1/article/{id}`, `PATCH /v1/article/{id}/publish` and `PATCH /v1/article/{id}/unpublish` return the twelve SEO fields; the items of `GET /v1/article/` return `slug` and `excerpt`.
+- `POST /v1/site/` and `PATCH /v1/site/{id}` take `site_name`, `title_template` and `default_description`; a template without `%title%` is `400` `invalid_title_template`.
+- Changed: `GET /v1/media/{id}/usage` stays an array, and every item gains `type` and `fields` without losing `id`, `title` and `published`. Authors and sites that use the image are now listed, without a `published` key.
+- Changed: a cover image, a social image, an author avatar and a site default social image all count in `used_in` and in the `unused` filter. Deleting an image clears the covers and social images that use it, and merging or purging repoints them.
+
+### Upgrading
+
+- The first start after the upgrade adds the new columns, gives every existing article a slug generated from its title (in id order within each site, with `-2`, `-3` on conflicts) and creates the unique `(site_id, slug)` index. Nothing else changes, and a second start changes nothing.
+- A deleted article now frees its slug.
+
 ## Published releases

 Each body lives in `docs/releases/<tag>.md`. Publish with `tea releases create --tag <tag> --title "<tag> - <theme>" --note-file docs/releases/<tag>.md --prerelease --asset <file>` after `task release`, with one `--asset` per file: the six binaries, `bin/checksums.txt` and `bin/checksums.txt.sig`. `task release` produces all eight and fails before building when the signing key is missing or wrong (see [deployment.md](deployment.md#release-signing-key-maintainers-once)). Never publish a release without the two checksum files.