feat: storage, language rules and migration
This commit is contained in:
Florian Sylvain committed
1 parent
8c4a1825ab
commit
0806733d5c
70 files changed
+1771
-132
No files matched your search
@@ -0,0 +1,150 @@
|
||||
Feature: Article language, story and per-site language storage
|
||||
Articles gain a full BCP 47 language tag, a story they belong to and a
|
||||
content revision. Each site gets a language table with one default. Existing
|
||||
articles keep their ids, bodies, publication and public API. They are
|
||||
migrated without a guessed language: each gets its own story and an
|
||||
unassigned language until the owner confirms the site default.
|
||||
|
||||
Background:
|
||||
Given the supported languages are "en", "en-GB", "en-US", "fr", "fr-CA", "pt", "pt-BR", "es", "de", "ar"
|
||||
And a language tag keeps its region, "pt-BR" is never reduced to "pt"
|
||||
And the domain package internal/domain/language imports only the stdlib
|
||||
And persistence is portable across SQLite, MySQL and PostgreSQL
|
||||
|
||||
Scenario Outline: Language tag normalization
|
||||
When the tag "<input>" is normalized
|
||||
Then the result is "<result>"
|
||||
|
||||
Examples:
|
||||
| input | result |
|
||||
| en | en |
|
||||
| EN | en |
|
||||
| pt-br | pt-BR |
|
||||
| PT_BR | ErrInvalidLanguage |
|
||||
| fr-ca | fr-CA |
|
||||
| fr | fr |
|
||||
| e | ErrInvalidLanguage |
|
||||
| english | ErrInvalidLanguage |
|
||||
| en-ZZZZ | ErrInvalidLanguage |
|
||||
| | ErrInvalidLanguage |
|
||||
| xx | ErrUnsupportedLanguage |
|
||||
|
||||
Scenario Outline: Suggestion for a site's default language from Site.Locale
|
||||
Given a site whose Locale is "<locale>"
|
||||
When the default language suggestion is computed
|
||||
Then the suggestion is "<suggested>" and the substitution flag is <substituted>
|
||||
|
||||
Examples:
|
||||
| locale | suggested | substituted |
|
||||
| pt-BR | pt-BR | false |
|
||||
| en-GB | en-GB | false |
|
||||
| fr-BE | fr | true |
|
||||
| fr | fr | false |
|
||||
| xx-YY | none | false |
|
||||
| null | none | false |
|
||||
|
||||
Scenario: Migration of an existing site and its articles
|
||||
Given a database created before this change with site "default" (Locale "fr-CA") and articles 1, 2 and 3 in it, article 3 online
|
||||
When database.Migrate runs
|
||||
Then every article keeps its id, title, body, slug, is_online and timestamps
|
||||
And each article has lang "" and its own distinct story_id
|
||||
And each article has content_revision 1
|
||||
And site "default" has one site_languages row "fr-CA" with is_default true, enabled true and confirmed false
|
||||
And the migration marker for the article story backfill is recorded
|
||||
|
||||
Scenario: Site without a usable locale gets no suggestion row
|
||||
Given a site with Locale null
|
||||
When database.Migrate runs
|
||||
Then the site has no site_languages row
|
||||
|
||||
Scenario: Migration runs twice
|
||||
Given a migrated database
|
||||
When database.Migrate runs again
|
||||
Then no story is created or changed, no article changes and no site_languages row changes
|
||||
And the schema is identical
|
||||
|
||||
Scenario: Migration is atomic
|
||||
Given a database with 3 existing articles
|
||||
And the backfill write fails while handling the second article
|
||||
When database.Migrate runs
|
||||
Then no article has a story_id and the marker is not recorded
|
||||
And the next Migrate completes the whole backfill
|
||||
|
||||
Scenario: Soft-deleted articles are migrated
|
||||
Given an article that is soft-deleted before the change
|
||||
When database.Migrate runs
|
||||
Then it has its own story_id and its tombstone slug is unchanged
|
||||
|
||||
Scenario: Slug is unique per site and language
|
||||
Given site 1 holds an article with slug "bonjour" in language "fr"
|
||||
When another article is created in site 1, language "fr", slug "bonjour"
|
||||
Then the write fails with a unique violation mapped to ErrSlugTaken
|
||||
When an article with slug "bonjour" is created in site 1, language "en"
|
||||
Then it is stored
|
||||
When an article with slug "bonjour" is created in site 2, language "fr"
|
||||
Then it is stored
|
||||
|
||||
Scenario: Existing unassigned articles keep the current per-site slug rule
|
||||
Given two unassigned articles in site 1
|
||||
When the second is written with the first one's slug
|
||||
Then ErrSlugTaken is returned
|
||||
|
||||
Scenario: One version per language in a story
|
||||
Given story 7 holds an article in language "fr"
|
||||
When a second article is stored in story 7, language "fr"
|
||||
Then the write fails with a unique violation
|
||||
When an article is stored in story 7, language "en"
|
||||
Then it is stored
|
||||
|
||||
Scenario: Unassigned articles never collide on language
|
||||
Given 3 articles each with lang "" and their own story
|
||||
Then all 3 are stored
|
||||
|
||||
Scenario: Creating an article before the default is confirmed
|
||||
Given site "default" has an unconfirmed suggestion "fr-CA"
|
||||
When a user creates an article with title "Bonjour le monde"
|
||||
Then the article has lang ""
|
||||
And it has a new story_id and content_revision 1
|
||||
|
||||
Scenario: Creating an article after the default is confirmed
|
||||
Given site "default" has a confirmed default "fr-CA"
|
||||
When a user creates an article with title "Bonjour le monde"
|
||||
Then the article has lang "fr-CA"
|
||||
And it has a new story_id distinct from every other article
|
||||
|
||||
Scenario: Create racing a confirmation never strands an article
|
||||
Given site "default" has unassigned articles
|
||||
When the default is confirmed and assigned to unassigned articles in one transaction while articles are created concurrently
|
||||
Then every article of the site ends with the confirmed default
|
||||
And no article has a different language than the confirmed default
|
||||
|
||||
Scenario: Assigning a language is idempotent
|
||||
Given a site whose unassigned articles were assigned "fr-CA"
|
||||
When the assignment runs again
|
||||
Then no article changes
|
||||
|
||||
Scenario Outline: Article reads expose lang and story_id
|
||||
Given article 12 with lang "<lang>" in story 5
|
||||
When it is read through "<read>"
|
||||
Then the JSON contains "lang": "<lang>" and "story_id": 5
|
||||
And every key present before this change is present with an unchanged value and type
|
||||
|
||||
Examples:
|
||||
| lang | read |
|
||||
| fr-CA | GET /v1/article/12 with a valid jwt |
|
||||
| | GET /v1/article/12 with a valid jwt |
|
||||
| fr-CA | GET /v1/article/ list item |
|
||||
| fr-CA | public article read of an online article |
|
||||
|
||||
Scenario: Article write payloads cannot set lang or story_id
|
||||
When POST or PATCH /v1/article carries "lang" or "story_id"
|
||||
Then the fields are ignored and the stored values are unchanged
|
||||
|
||||
Scenario: Existing behaviour is unchanged
|
||||
Then the existing article, pagination, publishing and SEO tests pass without edits
|
||||
And offline articles still read as not found publicly
|
||||
|
||||
Scenario: Persistence portability
|
||||
Given the repository contract cases above
|
||||
Then they pass on SQLite, MySQL 8.4, MariaDB 11 and PostgreSQL
|
||||
And no raw SQL uses a dialect-only construct
|
||||
@@ -0,0 +1,28 @@
|
||||
# QA: Article localization storage (backend)
|
||||
|
||||
Backend only. Go tests in test/, black-box. No design states. No new endpoints.
|
||||
|
||||
## Automated, no server (task test)
|
||||
1. Language domain: table-driven over "Language tag normalization" and "Suggestion for a site's default language". Assert the sentinel errors with errors.Is.
|
||||
2. Use case: with fake repositories, run "Creating an article before the default is confirmed" and "after the default is confirmed". Assert that CreateArticle runs inside the Transactor, that the starting language is obtained through LockStartingLanguage of the site language repository (under the site-row lock) and passed to Create as Lang, and that the story is created by the article repository inside the same transaction. Existing create tests stay green.
|
||||
3. HTTP: NewHandler over use cases with in-memory fakes, a fresh chi.NewRouter mirroring api/router.go, validator.InitValidator(), httptest. Cover "Article reads expose lang and story_id" for GET /v1/article/{id} with a jwt cookie signed by JWT_SECRET (the test secret), the list GET /v1/article/, and the public read of an online article. Assert that old keys are unchanged, that lang and story_id are the added keys, and that unknown "lang" or "story_id" in the request body is ignored. Errors stay plain text.
|
||||
4. Regression: run the full existing article, pagination, publishing and SEO tests untouched.
|
||||
|
||||
## Automated, SQLite file (task test)
|
||||
5. Fixture: a pre-change database built with the old models, holding a site with Locale "fr-CA", 3 articles (1 online, 1 soft-deleted) and article authors. Run Migrate and check the "Migration of an existing site" scenario. Check ids, bodies and slugs are unchanged, distinct story_ids, content_revision 1, the confirmed=false suggestion row, and the marker row.
|
||||
6. Migrate twice: nothing changes. Compare sqlite_master and a hash of all rows.
|
||||
7. Atomicity: inject a failing write (a wrapped *gorm.DB callback that errors on the second update). Assert no story_id is set and the marker is absent, then re-run successfully.
|
||||
8. Unique indexes: (story_id, lang) and (site_id, lang, slug) reject duplicates, and lang "" rows with different stories coexist. The old (site_id, slug) index definition is replaced by (site_id, lang, slug) under the same name idx_articles_site_slug, kept so existing tests keep passing.
|
||||
9. Concurrency: N goroutines create articles while the confirm-and-assign transaction runs. Assert the final state of "Create racing a confirmation". On SQLite the repository sets busy_timeout only (no WAL), so the test retries SQLITE_BUSY, counts the retries and asserts a bound.
|
||||
|
||||
## Automated, real databases (task test:mysql, task test:postgres)
|
||||
10. Repeat 5-9 on MySQL 8.4, MariaDB 11 and PostgreSQL. Compare the unique indexes through information_schema or pg_indexes. Record NOT RUN if the daemon is down, or use the podman shim (see memory).
|
||||
11. On MySQL and MariaDB, check the index key lengths (utf8mb4): lang size is at most 35 and the slug index is within limits.
|
||||
|
||||
## Build and lint
|
||||
12. task test, task lint:go, and CGO_ENABLED=0 go build -tags nofrontend ./... pass.
|
||||
13. docs/ gains the new article fields in the HTTP contract (lang, story_id) for the SPA slices.
|
||||
14. task mutate scoped to the new use case code with MUTATE_FILES. No comments in the added Go files.
|
||||
|
||||
## Manual
|
||||
15. Back up a real database copy with article rows, start the new binary on it, check the articles list and one public read in a browser (no shape change), and restart to check the migration does not run twice.
|
||||
Reference in new issue
Block a user