feat: media duplicate purge, part 1 of 3: persisted purge settings and the purge API
This commit is contained in:
fsylvain committed
1 parent
161ddc0f7d
commit
e82d0ce254
25 files changed
+2459
-41
No files matched your search
@@ -0,0 +1,432 @@
|
||||
Feature: Owners and admins can merge duplicate images in use, with persisted purge settings
|
||||
|
||||
As a RenewCMS owner or admin
|
||||
I want to see what a duplicate purge would do, run it, and keep the automatic-purge settings
|
||||
So that articles end up pointing at the most used copy of an image and the other copies are removed
|
||||
|
||||
Background:
|
||||
Given internal/domain/purge defines the entity Settings with AutoPurgeEnabled (default false),
|
||||
AutoPurgeIntervalDays (default 30, valid 1 to 365), LastPurgeAt, LastPurgeMerged,
|
||||
LastPurgeArticlesUpdated and EnabledAt (unset or 0 by default), plus a Repository
|
||||
interface with Get and Save, and the sentinels ErrInvalidPurgeInterval, ErrPurgeConflict
|
||||
(the library changed since the fingerprint) and ErrPurgeBusy (a run is already in progress),
|
||||
all asserted with errors.Is
|
||||
And Repository.Get returns the defaults when nothing was ever saved (a single-row table)
|
||||
And the use cases GetPurgeSettings, UpdatePurgeSettings, PreviewPurge and RunPurge depend only
|
||||
on domain interfaces, take no HTTP types and are reusable by a scheduler
|
||||
And the use cases receive an injectable clock and an injected *log.Logger
|
||||
And an image's "used in" is the count of articles, online or draft, whose body contains the
|
||||
image's stored name, read from article bodies each time, as in ListMedia
|
||||
And a duplicate group is the set of images sharing one non-empty sha256, as from
|
||||
image.Repository.ListDuplicates
|
||||
And the purge plan is: a group is planned only if at least one copy is used in an article, the
|
||||
kept copy is the most used one (ties: oldest created_at, then lowest id), and every other
|
||||
copy of the group, used or unused, is merged into it. A group with no used copy is skipped
|
||||
And the fingerprint is a hex SHA-256 of the planned groups ordered by kept id, with their kept
|
||||
id and sorted merged ids; an empty plan has a fixed fingerprint
|
||||
And useCases.MergeMediaResult keeps ArticlesUpdated and Merged, and gains UpdatedArticleIDs,
|
||||
the ids of the articles actually rewritten (empty when the merge was rolled back or failed)
|
||||
And RunPurge merges each group with the MergeMedia semantics and unions UpdatedArticleIDs of the
|
||||
fully merged groups; it never recomputes the articles from the plan
|
||||
And next_purge_at is max(last_purge_at, enabled_at) + auto_purge_interval_days when automatic
|
||||
purge is on and null when it is off; with no last_purge_at it is enabled_at + interval
|
||||
And /v1/media/purge-settings, /v1/media/purge-preview and /v1/media/purge are mounted in
|
||||
api/router.go inside the NewMediaRouter chain (Verifier, Authenticator, RequireCurrentToken,
|
||||
RequireVerifiedUser), then a group guarded by the new middleware.RequireOwnerOrAdmin, and
|
||||
registered before the /{id} routes
|
||||
And RequireOwnerOrAdmin is an allowlist: only the roles owner and admin pass; any other value,
|
||||
including an empty role, answers 403 with apperr.OwnerOrAdminRequiredErrorMessage
|
||||
("Only owners and admins can do this.") as plain text from http.Error
|
||||
And a stale fingerprint answers 409 with apperr.PurgeConflictErrorMessage ("The library changed,
|
||||
so nothing was purged. Check the new summary and try again.") and a busy run answers 409 with
|
||||
apperr.PurgeBusyErrorMessage ("A purge is already running.")
|
||||
And the purge settings range error is apperr.PurgeIntervalErrorMessage ("Enter a whole number of
|
||||
days between 1 and 365.")
|
||||
And request and response bodies are JSON with snake_case fields; errors are plain text
|
||||
And the request body of PATCH is validated by validate tags and validator.Validate.Struct
|
||||
|
||||
Scenario Outline: Every purge route refuses callers that are not allowed
|
||||
When the client sends <method> <path> <credentials>
|
||||
Then the response status is <status>
|
||||
And the body is <body>
|
||||
And no purge setting, image or article is read or changed
|
||||
|
||||
Examples:
|
||||
| method | path | credentials | status | body |
|
||||
| GET | /v1/media/purge-settings | no cookie | 401 | the existing jwtauth response (status only) |
|
||||
| PATCH | /v1/media/purge-settings | no cookie | 401 | the existing jwtauth response (status only) |
|
||||
| GET | /v1/media/purge-preview | no cookie | 401 | the existing jwtauth response (status only) |
|
||||
| POST | /v1/media/purge | no cookie | 401 | the existing jwtauth response (status only) |
|
||||
| GET | /v1/media/purge-settings | a stale token_version cookie | 401 | apperr.UnauthorizedErrorMessage |
|
||||
| GET | /v1/media/purge-settings | cookie of an unverified owner | 403 | apperr.VerificationRequiredErrorMessage |
|
||||
| GET | /v1/media/purge-settings | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| PATCH | /v1/media/purge-settings | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| GET | /v1/media/purge-preview | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| POST | /v1/media/purge | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| GET | /v1/media/purge-settings | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| PATCH | /v1/media/purge-settings | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| GET | /v1/media/purge-preview | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
| POST | /v1/media/purge | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
|
||||
|
||||
Scenario Outline: Owners and admins both pass the role guard
|
||||
When the client sends <method> <path> with a valid cookie of a verified <role>
|
||||
Then the response status is not 401 and not 403
|
||||
|
||||
Examples:
|
||||
| role | method | path |
|
||||
| owner | GET | /v1/media/purge-settings |
|
||||
| admin | GET | /v1/media/purge-settings |
|
||||
| owner | GET | /v1/media/purge-preview |
|
||||
| admin | GET | /v1/media/purge-preview |
|
||||
|
||||
Rule: Purge settings
|
||||
|
||||
Scenario: Nothing was ever saved, so the defaults are served
|
||||
When the client sends GET /v1/media/purge-settings with a valid cookie of the owner
|
||||
Then the response status is 200
|
||||
And the body is {"auto_purge_enabled": false, "auto_purge_interval_days": 30,
|
||||
"last_purge_at": null, "last_purge_merged": 0, "last_purge_articles_updated": 0,
|
||||
"enabled_at": null, "next_purge_at": null}
|
||||
|
||||
Scenario: Turning automatic purge on stamps enabled_at and computes next_purge_at
|
||||
Given the clock reads 2026-10-05T09:00:00Z and the settings are the defaults
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
|
||||
Then the response status is 200
|
||||
And "auto_purge_enabled" is true, "auto_purge_interval_days" is 30
|
||||
And "enabled_at" is "2026-10-05T09:00:00Z" and "last_purge_at" is null
|
||||
And "next_purge_at" is "2026-11-04T09:00:00Z"
|
||||
And a following GET /v1/media/purge-settings returns the same body
|
||||
|
||||
Scenario: next_purge_at counts from the last purge when it is later than enabled_at
|
||||
Given automatic purge is on with interval 10, enabled_at 2026-10-01T00:00:00Z and last_purge_at 2026-10-04T12:00:00Z
|
||||
When the client sends GET /v1/media/purge-settings
|
||||
Then "next_purge_at" is "2026-10-14T12:00:00Z"
|
||||
|
||||
Scenario: next_purge_at counts from enabled_at when it is later than the last purge
|
||||
Given automatic purge is on with interval 10, enabled_at 2026-10-05T09:00:00Z and last_purge_at 2026-09-01T00:00:00Z
|
||||
When the client sends GET /v1/media/purge-settings
|
||||
Then "next_purge_at" is "2026-10-15T09:00:00Z"
|
||||
|
||||
Scenario: Turning automatic purge off and on again never yields a next_purge_at in the past
|
||||
Given automatic purge was on, last_purge_at is 2026-09-01T00:00:00Z, and it was turned off
|
||||
And the clock reads 2026-10-05T09:00:00Z and the interval is 30
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
|
||||
Then "enabled_at" is "2026-10-05T09:00:00Z"
|
||||
And "last_purge_at" is still "2026-09-01T00:00:00Z"
|
||||
And "next_purge_at" is "2026-11-04T09:00:00Z"
|
||||
|
||||
Scenario: Changing only the interval keeps enabled_at
|
||||
Given automatic purge is on since 2026-10-01T00:00:00Z with interval 30, no purge yet, and the clock reads 2026-10-05T09:00:00Z
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": 7}
|
||||
Then the response status is 200
|
||||
And "auto_purge_interval_days" is 7 and "enabled_at" is still "2026-10-01T00:00:00Z"
|
||||
And "next_purge_at" is "2026-10-08T00:00:00Z"
|
||||
|
||||
Scenario: Patching "on" while already on does not restamp enabled_at
|
||||
Given automatic purge is on since 2026-10-01T00:00:00Z and the clock reads 2026-10-05T09:00:00Z
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
|
||||
Then "enabled_at" is still "2026-10-01T00:00:00Z"
|
||||
|
||||
Scenario: Turning automatic purge off nulls next_purge_at and keeps the history
|
||||
Given automatic purge is on with last_purge_at 2026-10-04T12:00:00Z, last_purge_merged 3 and last_purge_articles_updated 5
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": false}
|
||||
Then "auto_purge_enabled" is false and "next_purge_at" is null
|
||||
And "last_purge_at", "last_purge_merged" (3) and "last_purge_articles_updated" (5) are unchanged
|
||||
|
||||
Scenario Outline: The interval accepts whole numbers from 1 to 365
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": <days>}
|
||||
Then the response status is 200 and "auto_purge_interval_days" is <days>
|
||||
|
||||
Examples:
|
||||
| days |
|
||||
| 1 |
|
||||
| 30 |
|
||||
| 365 |
|
||||
|
||||
Scenario Outline: An invalid patch is 400 and nothing changes
|
||||
When the client sends PATCH /v1/media/purge-settings with <body>
|
||||
Then the response status is 400
|
||||
And the body is <message>
|
||||
And the stored settings are unchanged
|
||||
|
||||
Examples:
|
||||
| body | message |
|
||||
| {"auto_purge_interval_days": 0} | apperr.PurgeIntervalErrorMessage |
|
||||
| {"auto_purge_interval_days": 366} | apperr.PurgeIntervalErrorMessage |
|
||||
| {"auto_purge_interval_days": -1} | apperr.PurgeIntervalErrorMessage |
|
||||
| {"auto_purge_interval_days": 1.5} | apperr.BodyErrorMessage |
|
||||
| {"auto_purge_interval_days": "7"} | apperr.BodyErrorMessage |
|
||||
| {"auto_purge_enabled": "yes"} | apperr.BodyErrorMessage |
|
||||
| not json | apperr.BodyErrorMessage |
|
||||
|
||||
Scenario: An empty object changes nothing
|
||||
When the client sends PATCH /v1/media/purge-settings with {}
|
||||
Then the response status is 200 and the body equals the current settings
|
||||
|
||||
Scenario: The use case rejects an out-of-range interval with the domain sentinel
|
||||
When UpdatePurgeSettings receives interval 0, or 366
|
||||
Then errors.Is(err, purge.ErrInvalidPurgeInterval) is true
|
||||
And Repository.Save is not called
|
||||
|
||||
Scenario: A repository failure is a generic 500
|
||||
Given the purge settings repository's next Get fails
|
||||
When the client sends GET /v1/media/purge-settings
|
||||
Then the response status is 500 and the body is "Internal server error"
|
||||
|
||||
Scenario: A failing Save on PATCH is a generic 500
|
||||
Given the purge settings repository's next Save fails
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
|
||||
Then the response status is 500 and the body is "Internal server error"
|
||||
|
||||
Scenario: The purge routes are not taken by the /{id} routes
|
||||
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": 7}
|
||||
Then the settings are updated and no image is renamed
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then the preview use case is invoked
|
||||
|
||||
Rule: Purge plan and preview
|
||||
|
||||
Background:
|
||||
Given the clock reads 2026-10-05T09:00:00Z
|
||||
And images are created in id order, oldest first
|
||||
And articles reference images by their stored name
|
||||
|
||||
Scenario: A group with a used copy is planned, merging every other copy into the most used
|
||||
Given images 1, 2 and 3 share sha256 A with filenames "hero.jpg", "hero-copy.jpg", "hero-2.jpg"
|
||||
And article 10 uses image 1, articles 11 and 12 use image 2, article 13 uses image 3
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then the response status is 200
|
||||
And "groups" is 1, "copies" is 2 and "articles" is 2
|
||||
And "list" is [{"keep_id": 2, "filename": "hero-copy.jpg", "url": "<stored url of image 2>",
|
||||
"used_in": 2, "merged": 2}]
|
||||
And "more" is 0 and "fingerprint" is a 64-character hex string
|
||||
|
||||
Scenario: Ties on usage keep the oldest copy, then the lowest id
|
||||
Given images 4 and 5 share sha256 B, created at the same instant, each used in one article
|
||||
And images 6 and 7 share sha256 C, image 7 is older than image 6, each used in one article
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then the "keep_id" of the group B is 4 and the "keep_id" of the group C is 7
|
||||
|
||||
Scenario: A single used copy with unused siblings keeps the used one
|
||||
Given images 8, 9 and 10 share sha256 D and only image 10 is used, in one article
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then the group has keep_id 10, used_in 1, merged 2
|
||||
And "articles" is 0, since no article references a merged-away copy
|
||||
|
||||
Scenario: A group whose copies are all unused is skipped
|
||||
Given images 11 and 12 share sha256 E and no article references either
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then "groups" is 0, "copies" is 0, "articles" is 0, "list" is [] and "more" is 0
|
||||
|
||||
Scenario: A library with no duplicates
|
||||
Given the library holds three images with distinct sha256 values
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then the response status is 200
|
||||
And "groups" is 0 and "list" is an empty array in the raw JSON, not null
|
||||
And "fingerprint" equals the fingerprint of any other empty plan
|
||||
|
||||
Scenario: Images without a hash are never grouped
|
||||
Given images 13 and 14 both have an empty sha256 and both are used
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then "groups" is 0
|
||||
|
||||
Scenario: An article that uses copies of two groups counts once
|
||||
Given groups A (keep 1, copy 2) and B (keep 3, copy 4) and one article that uses images 2 and 4
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then "groups" is 2, "copies" is 2 and "articles" is 1
|
||||
|
||||
Scenario: An article that uses both the kept copy and a copy is rewritten
|
||||
Given an article uses images 1 and 2 of one group where image 1 is kept
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then "articles" is 1
|
||||
|
||||
Scenario: The list shows at most 5 groups and reports the rest
|
||||
Given 7 planned groups
|
||||
When the client sends GET /v1/media/purge-preview
|
||||
Then "groups" is 7, "list" has 5 entries in ascending keep_id order and "more" is 2
|
||||
|
||||
Scenario: The preview changes nothing
|
||||
Given a planned group
|
||||
When the client sends GET /v1/media/purge-preview twice
|
||||
Then both bodies are equal
|
||||
And no image is deleted, no article body changes and no purge setting is saved
|
||||
|
||||
Scenario Outline: The fingerprint reacts to the plan and to nothing else
|
||||
Given a planned group with kept image 2 and merged image 3
|
||||
When <change>
|
||||
Then the preview fingerprint is <result>
|
||||
|
||||
Examples:
|
||||
| change | result |
|
||||
| nothing changes | identical |
|
||||
| a new unused copy is uploaded to the group | different |
|
||||
| a copy is deleted from the group | different |
|
||||
| article usage changes so that another copy becomes kept | different |
|
||||
| an unrelated image with a unique sha256 is uploaded | identical |
|
||||
| an article body changes without changing which is kept | identical |
|
||||
| a group with no used copy gains an unused copy | identical |
|
||||
|
||||
Rule: Running a purge
|
||||
|
||||
Background:
|
||||
Given the clock reads 2026-10-05T09:00:00Z
|
||||
|
||||
Scenario: A run merges every planned group and persists the result
|
||||
Given group A: images 1 (used in article 10), 2 (used in articles 11 and 12) and 3 (unused)
|
||||
And group B: images 4 (used in article 12) and 5 (unused)
|
||||
And group C: images 6 and 7, both unused
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 200
|
||||
And the body is {"groups_total": 2, "groups_purged": 2, "merged": 3, "articles_updated": 1, "failed_groups": 0}
|
||||
And images 1 and 3 are gone from the image repository, image 2 remains
|
||||
And image 5 is gone, image 4 remains, and images 6 and 7 both remain
|
||||
And article 10's body now uses image 2's stored name
|
||||
And the stored settings have last_purge_at "2026-10-05T09:00:00Z", last_purge_merged 3 and last_purge_articles_updated 1
|
||||
And only article 10 is rewritten, since article 12 uses only kept copies
|
||||
And "auto_purge_enabled", "auto_purge_interval_days" and "enabled_at" are untouched
|
||||
|
||||
Scenario: An article rewritten by two groups is counted once in articles_updated
|
||||
Given an article uses a copy of group A and a copy of group B
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then "articles_updated" counts that article once, from the union of the groups' UpdatedArticleIDs
|
||||
|
||||
Scenario: A single used copy with unused siblings is merged without rewriting any article
|
||||
Given images 8, 9 and 10 share a hash and only image 10 is used
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the body is {"groups_total": 1, "groups_purged": 1, "merged": 2, "articles_updated": 0, "failed_groups": 0}
|
||||
And images 8 and 9 are gone, image 10 remains and no article body changed
|
||||
|
||||
Scenario: A matching fingerprint is accepted
|
||||
Given the preview returned fingerprint F
|
||||
When the client sends POST /v1/media/purge with {"fingerprint": "F"}
|
||||
Then the response status is 200 and the purge ran
|
||||
|
||||
Scenario: A stale fingerprint answers 409 and changes nothing
|
||||
Given the preview returned fingerprint F
|
||||
And an unused copy is then uploaded to a planned group
|
||||
When the client sends POST /v1/media/purge with {"fingerprint": "F"}
|
||||
Then the response status is 409
|
||||
And the body is apperr.PurgeConflictErrorMessage
|
||||
And no image is deleted, no article body changes and the stored last_purge_* values are unchanged
|
||||
|
||||
Scenario: A fingerprint that matches no plan at all is a conflict, not a validation error
|
||||
When the client sends POST /v1/media/purge with {"fingerprint": "not-a-real-fingerprint"}
|
||||
Then the response status is 409 and the body is apperr.PurgeConflictErrorMessage
|
||||
And nothing changed
|
||||
|
||||
Scenario: A run without a fingerprint always runs the current plan
|
||||
Given the library changed since any preview
|
||||
When the client sends POST /v1/media/purge with an empty body
|
||||
Then the response status is 200 and the current plan ran
|
||||
|
||||
Scenario: A library with nothing to purge answers zeros and is still recorded
|
||||
Given the library holds no planned group
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 200
|
||||
And the body is {"groups_total": 0, "groups_purged": 0, "merged": 0, "articles_updated": 0, "failed_groups": 0}
|
||||
And the stored settings have last_purge_at "2026-10-05T09:00:00Z", last_purge_merged 0 and last_purge_articles_updated 0
|
||||
And a scheduler reading next_purge_at sees it move forward from that run
|
||||
|
||||
Scenario: The fingerprint of the empty plan on an empty plan runs as a no-op
|
||||
Given the library holds no planned group
|
||||
When the client sends POST /v1/media/purge with the empty plan's fingerprint
|
||||
Then the response status is 200 and the body has groups_total 0
|
||||
|
||||
Scenario: A group that fails while rewriting bodies is rolled back and contributes no ids
|
||||
Given groups A, B and C are planned, in that order
|
||||
And the article repository's UpdateBody fails for the second article that group B must rewrite
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 200
|
||||
And the body is {"groups_total": 3, "groups_purged": 2, "merged": <copies of A and C>, "articles_updated": <distinct articles of A and C>, "failed_groups": 1}
|
||||
And group B's article bodies are back to their original text and all of group B's images remain
|
||||
And group B's UpdatedArticleIDs is empty, so no article of group B is in "articles_updated"
|
||||
And the stored last_purge_merged and last_purge_articles_updated equal the response totals
|
||||
|
||||
Scenario: A group that fails after rewriting bodies is reported as failed and contributes nothing
|
||||
Given the image repository's Delete fails for a merged image of group A, and group B is planned
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 200, "failed_groups" is 1 and "groups_purged" is 1
|
||||
And group A contributes 0 to "merged" and its article ids are not added to "articles_updated", even though its article bodies stay switched to the kept copy
|
||||
And "articles_updated" and "merged" count only the fully merged group B
|
||||
And a following GET /v1/media/purge-preview plans group A again, and the next run converges because the plan is recomputed from current data
|
||||
|
||||
Scenario: Every group failing is still a 200 with the counters
|
||||
Given every UpdateBody fails
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 200
|
||||
And "groups_purged" is 0 and "failed_groups" equals "groups_total"
|
||||
And the stored last_purge_at is set
|
||||
|
||||
Scenario: A second run while one is in progress is rejected as busy, not queued
|
||||
Given a purge run is in progress and blocked inside the image repository
|
||||
When a second POST /v1/media/purge arrives, with or without a fingerprint
|
||||
Then it answers 409 with apperr.PurgeBusyErrorMessage at once
|
||||
And it changed nothing and saved no last_purge_* values
|
||||
When the first run is released
|
||||
Then it finishes normally with its own counters
|
||||
And a later POST /v1/media/purge is accepted
|
||||
|
||||
Scenario: The use case distinguishes a busy run from a stale fingerprint
|
||||
When RunPurge receives a non-matching fingerprint
|
||||
Then errors.Is(err, purge.ErrPurgeConflict) is true and errors.Is(err, purge.ErrPurgeBusy) is false
|
||||
When RunPurge is called while another run holds the guard
|
||||
Then errors.Is(err, purge.ErrPurgeBusy) is true and errors.Is(err, purge.ErrPurgeConflict) is false
|
||||
And in both cases no repository write happens
|
||||
|
||||
Scenario: A run via the use case with no fingerprint needs no HTTP types
|
||||
When a scheduler later calls RunPurge with no fingerprint
|
||||
Then it gets the result struct and the same last_purge_* persistence applies
|
||||
|
||||
Scenario: A malformed body is 400
|
||||
When the client sends POST /v1/media/purge with "not json"
|
||||
Then the response status is 400 and the body is apperr.BodyErrorMessage
|
||||
And nothing changed
|
||||
|
||||
Scenario: A repository failure while planning is a generic 500 and nothing changes
|
||||
Given the image repository's ListDuplicates fails
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 500 and the body is "Internal server error"
|
||||
And no last_purge_* values are saved
|
||||
|
||||
Scenario: A failing save of the run result is a logged 500 and the merges stay
|
||||
Given the purge settings repository's Save fails after the merges succeeded
|
||||
When the client sends POST /v1/media/purge with {}
|
||||
Then the response status is 500 and the body is "Internal server error"
|
||||
And the merges that already happened stay done
|
||||
And the error is written to the injected *log.Logger
|
||||
And the next POST /v1/media/purge converges, because the plan is recomputed from current data
|
||||
|
||||
Scenario: The preview and the run use the same plan
|
||||
When the client sends GET /v1/media/purge-preview and then POST /v1/media/purge with the returned fingerprint
|
||||
Then the run's "groups_total" equals the preview's "groups"
|
||||
And the run's "merged" equals the preview's "copies" and its "articles_updated" equals the preview's "articles"
|
||||
|
||||
Rule: MergeMedia keeps its contract while exposing the rewritten ids
|
||||
|
||||
Scenario: Existing merges report the same counts and now also the ids
|
||||
When MergeMedia.Merge succeeds for a group that rewrites articles 1 and 2
|
||||
Then Merged and ArticlesUpdated are unchanged for existing callers
|
||||
And UpdatedArticleIDs is [1, 2]
|
||||
|
||||
Scenario: A rolled-back merge returns no ids
|
||||
When MergeMedia.Merge fails while rewriting a body
|
||||
Then it returns the error and an empty result
|
||||
|
||||
@manual
|
||||
Rule: Persistence and wiring (verified by hand, see the qa.md)
|
||||
|
||||
Scenario: The purge settings table is a single row created by AutoMigrate
|
||||
Given a database with no purge settings table
|
||||
When the server boots
|
||||
Then the table exists, GET /v1/media/purge-settings returns the defaults and no row is required beforehand
|
||||
And repeated PATCH calls keep exactly one row
|
||||
|
||||
Scenario: Settings survive a restart
|
||||
Given automatic purge was turned on with interval 7
|
||||
When the server restarts
|
||||
Then GET /v1/media/purge-settings returns the same values
|
||||
|
||||
Scenario: A real purge removes the merged files from UPLOAD_DIR
|
||||
Given duplicates uploaded through the real server
|
||||
When an owner runs POST /v1/media/purge
|
||||
Then the merged copies' files are gone from UPLOAD_DIR and the kept file remains
|
||||
@@ -0,0 +1,78 @@
|
||||
# QA procedure: media-purge
|
||||
|
||||
Package: `test` (black-box). New files `test/media_purge_http_test.go` and `test/media_purge_test.go` (use case level). Fake type names must be unique in the package. Go only: no SPA, no scheduler, no env var.
|
||||
|
||||
## Setup shared by every case
|
||||
1. Call `validator.InitValidator()` once.
|
||||
2. Fakes:
|
||||
- `mockPurgeSettingsRepo` (domain `purge.Repository`): holds one `Settings`, `Get` returns the defaults when nothing was saved, `Save` records the call count, with armed failure switches `failGet` and `failSave`.
|
||||
- Reuse or extend `mockImageRepo` (`test/image_test.go`): `ListDuplicates()` (images with a non-empty sha256 shared by at least two rows, ordered `sha256, id`), `Delete` with `failDeleteFor`, `Get`, `FindByHash`, `Counts`. Add a hook that lets a test block inside `ListDuplicates` or `Delete` on a channel, to hold a run in progress.
|
||||
- Reuse or extend `mockArticleRepo` (`test/article_test.go`): `FindByFilters`, `FindByBodyContaining`, `UpdateBody` with a per-id failure switch.
|
||||
- The user fake: a verified owner (id 1), a verified admin (id 2), a verified user with `Role("editor")` (id 3), a verified user with an empty role (id 5), and an unverified owner (id 4).
|
||||
3. Wire the real use cases over the fakes: the four purge use cases (constructors chosen by the coder), `NewGetUserUseCase(userRepo)`, plus the existing media use cases. Inject a fixed clock (2026-10-05T09:00:00Z) and a `*log.Logger` writing to a `bytes.Buffer`. Add the four purge handler methods to the media handler, built with `media.NewHandler(...)`.
|
||||
4. Build `jwtauth.New("HS256", []byte("test-secret"), nil)` and mount on a fresh `chi.NewRouter()` mirroring `NewMediaRouter` in `api/router.go`:
|
||||
- `jwtauth.Verifier` at the top, then `Authenticator`, `RequireCurrentToken(getUser)`, `RequireVerifiedUser(getUser)`.
|
||||
- The purge routes inside `r.Group` with `middleware.RequireOwnerOrAdmin(getUser)`, registered BEFORE `Patch("/{id}")` and `Delete("/{id}")`.
|
||||
- Also register `Get("/")`, `Post("/merge")`, `Patch("/{id}")` and `Delete("/{id}")` so route precedence is exercised.
|
||||
- Mount under `/v1/media`.
|
||||
5. Mint cookies via `token.Encode(map[string]any{"user_id": uint32(n), "token_version": ...})` as in other HTTP tests. Build a stale-token cookie with a lower `token_version`.
|
||||
6. Seeding helper `seedGroup(images ...)` assigning ids in order, strictly increasing `CreatedAt` unless the case says otherwise, and `articleUsing(id, images ...)` building bodies that contain each image's stored name (`image.StoredName(path)`).
|
||||
7. Compare JSON bodies by decoding, except where the case says "raw body" (null versus `[]`, null versus absent). Error bodies are plain text: compare to `message + "\n"`.
|
||||
|
||||
## Auth and role checks (table-driven)
|
||||
8. For each row of the "refuse callers" outline (4 routes x no cookie, stale, unverified, role "editor", empty role), assert the status and body. Assert the settings, image and article fakes saw no call (only the user lookup of the middleware).
|
||||
9. Owner and admin cookies: GET `/v1/media/purge-settings` and GET `/v1/media/purge-preview` return a status other than 401 and 403.
|
||||
10. Allowlist guard: add a case proving a role other than `owner`/`admin` (for example "OWNER" in upper case, or "superuser") is refused with 403, so the middleware is not a blocklist.
|
||||
|
||||
## Settings
|
||||
11. Defaults: empty repo, GET returns the exact default body. Assert the raw JSON has `"last_purge_at":null`, `"enabled_at":null` and `"next_purge_at":null` (not omitted).
|
||||
12. PATCH on: assert `enabled_at` equals the fixed clock, `next_purge_at` = enabled_at + 30 days, the GET after returns the same body, and `Save` was called once.
|
||||
13. next_purge_at basis (max rule): last_purge_at later than enabled_at counts from last_purge_at; enabled_at later than last_purge_at counts from enabled_at; no last_purge_at counts from enabled_at.
|
||||
14. Re-enable step: seed on with `last_purge_at` 2026-09-01, PATCH `false`, then PATCH `true` at the fixed clock with interval 30. Assert `enabled_at` is the clock, `last_purge_at` is still 2026-09-01, and `next_purge_at` is 2026-11-04T09:00:00Z, strictly after the clock.
|
||||
15. Interval-only PATCH keeps `enabled_at`. PATCH `true` while already on keeps `enabled_at`. PATCH `false` nulls `next_purge_at` and keeps last_purge_*.
|
||||
16. Interval table (1, 30, 365 accepted). Invalid table (0, 366, -1 give `apperr.PurgeIntervalErrorMessage`; 1.5, "7", "yes", not json give `apperr.BodyErrorMessage`). Assert the repo state and the `Save` count are unchanged after each 400.
|
||||
17. `{}` PATCH returns 200 equal to the current settings.
|
||||
18. Use-case-level: `UpdatePurgeSettings` with interval 0 and 366 returns an error with `errors.Is(err, purge.ErrInvalidPurgeInterval)` and no `Save`. At the handler, a value of 0 must be rejected by the validate tag and never reach the use case.
|
||||
19. Failure rows: `failGet` gives 500 "Internal server error\n"; `failSave` on PATCH gives 500.
|
||||
20. Route precedence: PATCH `/v1/media/purge-settings` reaches the settings handler and does not touch `Rename`.
|
||||
|
||||
## Preview
|
||||
21. Build each Background scenario with the seed helpers and assert the decoded `groups`, `copies`, `articles`, `list` and `more`:
|
||||
- used group (keep 2 with used_in 2; copies 2; articles 2);
|
||||
- tie-breaks: same CreatedAt gives the lowest id, different CreatedAt gives the oldest (assert both orders by swapping ids);
|
||||
- a single used copy with unused siblings (articles 0);
|
||||
- an all-unused group;
|
||||
- no duplicates (raw `"list":[]`);
|
||||
- empty-hash images ignored;
|
||||
- an article touching two groups counts once;
|
||||
- an article using kept plus a copy counts once;
|
||||
- 7 groups give 5 list entries in ascending keep_id order and `more` 2. Each entry has `keep_id`, `filename`, `url`, `used_in`, `merged`.
|
||||
22. Preview purity: call twice, assert equal bodies, assert zero `Delete`, `UpdateBody` and `Save` calls.
|
||||
23. Fingerprint table: compute a baseline, apply each change from the outline, and assert equal or different. Assert it is a 64-char lowercase hex string and that two empty plans share it.
|
||||
|
||||
## Run
|
||||
24. Full run scenario: assert the response body, the remaining images, the rewritten article bodies (check bytes), and `Save` called once with `last_purge_at` = clock, `last_purge_merged` = 3 and `last_purge_articles_updated` = 1, and that `auto_purge_*` and `enabled_at` are untouched. Article 12 uses only kept copies, so only article 10 is rewritten.
|
||||
25. `articles_updated` source: it must come from the union of `MergeMediaResult.UpdatedArticleIDs` returned by the group merges, never from the plan. Prove it with the rolled-back case in step 29: the plan contains B's articles, the response does not.
|
||||
26. Used-sibling run with zero article rewrites (`articles_updated` 0, `UpdateBody` never called).
|
||||
27. Fingerprint: match gives 200. A stale fingerprint gives 409 with `apperr.PurgeConflictErrorMessage + "\n"`, no `Delete`, no `UpdateBody`, no `Save`. A garbage string gives 409 with the same body. No fingerprint, `{}` and an empty body all run.
|
||||
28. Empty plan: 200 zeros, `Save` with `last_purge_at` = clock and zero counters, and a run with the empty plan's fingerprint is also 200.
|
||||
29. Partial failure, rolled back: arm `UpdateBody` to fail for the second article of group B (plan order A, B, C). Assert the body counters, B's bodies restored byte for byte, all B images present, A and C merged, B's articles absent from `articles_updated`, and `Save` counters equal the response.
|
||||
30. Partial failure, delete: arm `failDeleteFor` for one of group A's merged images and plan B too. Assert `failed_groups` 1, `groups_purged` 1, A contributes 0 to `merged` and none of its ids to `articles_updated` (only B counts), A's article bodies stay switched to the kept copy, and a following preview plans A again. Run the purge again with the failure disarmed and assert it converges (A's leftover copy is merged away). Then arm every `UpdateBody` and assert `groups_purged` 0, `failed_groups` = `groups_total` and `last_purge_at` still saved.
|
||||
31. Concurrency: start run 1 in a goroutine, blocked in the image fake (wait on a "started" channel). Send run 2 (once with a fingerprint, once without). Assert both answer 409 with `apperr.PurgeBusyErrorMessage + "\n"` immediately (use a timeout) and no `Save`/`Delete` calls from them. Release run 1, assert its 200 body and that a later run is accepted. Run this test with `-race`.
|
||||
32. Sentinels at the use-case level, asserted separately: a stale fingerprint gives `errors.Is(err, purge.ErrPurgeConflict)` true and `errors.Is(err, purge.ErrPurgeBusy)` false; a busy guard gives `ErrPurgeBusy` true and `ErrPurgeConflict` false. Call RunPurge without any HTTP package to prove it is reusable (that test file section imports no `net/http`).
|
||||
33. Malformed body gives 400 `apperr.BodyErrorMessage`. `ListDuplicates` failure gives 500 and no `Save`.
|
||||
34. Persist failure: arm `failSave` on the run-result save after merges succeed. Assert 500 "Internal server error\n", the merged images are still gone, the logger buffer contains the error text, then disarm and run again and assert 200 with no remaining groups (convergence from current data).
|
||||
35. Preview versus run parity: the run's `groups_total`, `merged` and `articles_updated` equal the preview's `groups`, `copies` and `articles` on the same seed.
|
||||
36. MergeMedia contract: extend the existing merge tests only by adding assertions. `Merged` and `ArticlesUpdated` are unchanged, `UpdatedArticleIDs` lists the rewritten ids in ascending order, and a rolled-back merge returns the error and an empty result. Existing callers and tests must compile and pass unchanged.
|
||||
|
||||
## Layering guards
|
||||
37. `api/handlers/media` imports no `persistence`, `mailer`, `models` or `gorm` package. The new use cases import no `net/http`, chi, gorm, `persistence`, `mailer` or `api/...` package. `internal/domain/purge` imports only the stdlib and other `internal/domain/*` packages. Use a `go list -deps` or grep-style test as in media_library.qa.md.
|
||||
|
||||
## Manual verification (not exercised by `task test`)
|
||||
- AutoMigrate: boot against a scratch SQLite file (`task dev`) and check the purge settings table exists, GET returns the defaults, repeated PATCH keeps one row, and values survive a restart. The mapper round-trips `last_purge_at`/`enabled_at` including null.
|
||||
- Real purge against real uploads: upload duplicates through `POST /v1/media/`, reference them from articles, run the preview then the purge. The merged files disappear from `UPLOAD_DIR`, the kept file loads through `/static/uploadedImages/`, and article bodies are switched.
|
||||
- Concurrent real requests with two shells (`curl` with the `jwt` cookie): the second gets 409 "A purge is already running." while the first runs.
|
||||
- Role check with a real admin account (invite then accept). Use a hand-edited user row with an empty or unknown role to check the 403.
|
||||
- The persist-failure log line appears in the server output (check by making the database read-only during a purge).
|
||||
- Docs: `docs/architecture.md` lists the `purge` domain package, the four use cases, the settings repository, the new routes and `RequireOwnerOrAdmin`. Read it and compare. No SMTP and no env var are involved.
|
||||
- Out of scope: the scheduler (part 2) and the SPA (part 3).
|
||||
Reference in new issue
Block a user