Files
RenewCMS/features/media_purge.feature
T

433 lines
26 KiB
Gherkin

Feature: Owners and admins can merge duplicate images in use, with persisted purge settings
As a RenewCMS owner or admin
I want to see what a duplicate purge would do, run it, and keep the automatic-purge settings
So that articles end up pointing at the most used copy of an image and the other copies are removed
Background:
Given internal/domain/purge defines the entity Settings with AutoPurgeEnabled (default false),
AutoPurgeIntervalDays (default 30, valid 1 to 365), LastPurgeAt, LastPurgeMerged,
LastPurgeArticlesUpdated and EnabledAt (unset or 0 by default), plus a Repository
interface with Get, Save and SaveLastRun, and the sentinels ErrInvalidPurgeInterval, ErrPurgeConflict
(the library changed since the fingerprint) and ErrPurgeBusy (a run is already in progress),
all asserted with errors.Is
And Repository.Get returns the defaults when nothing was ever saved (a single-row table)
And the use cases GetPurgeSettings, UpdatePurgeSettings, PreviewPurge and RunPurge depend only
on domain interfaces, take no HTTP types and are reusable by a scheduler
And the use cases receive an injectable clock and an injected *log.Logger
And an image's "used in" is the count of articles, online or draft, whose body contains the
image's stored name, read from article bodies each time, as in ListMedia
And a duplicate group is the set of images sharing one non-empty sha256, as from
image.Repository.ListDuplicates
And the purge plan is: a group is planned only if at least one copy is used in an article, the
kept copy is the most used one (ties: oldest created_at, then lowest id), and every other
copy of the group, used or unused, is merged into it. A group with no used copy is skipped
And the fingerprint is a hex SHA-256 of the planned groups ordered by kept id, with their kept
id and sorted merged ids; an empty plan has a fixed fingerprint
And useCases.MergeMediaResult keeps ArticlesUpdated and Merged, and gains UpdatedArticleIDs,
the ids of the articles actually rewritten (empty when the merge was rolled back or failed)
And RunPurge merges each group with the MergeMedia semantics and unions UpdatedArticleIDs of the
fully merged groups; it never recomputes the articles from the plan
And next_purge_at is max(last_purge_at, enabled_at) + auto_purge_interval_days when automatic
purge is on and null when it is off; with no last_purge_at it is enabled_at + interval
And /v1/media/purge-settings, /v1/media/purge-preview and /v1/media/purge are mounted in
api/router.go inside the NewMediaRouter chain (Verifier, Authenticator, RequireCurrentToken,
RequireVerifiedUser), then a group guarded by the new middleware.RequireOwnerOrAdmin, and
registered before the /{id} routes
And RequireOwnerOrAdmin is an allowlist: only the roles owner and admin pass; any other value,
including an empty role, answers 403 with apperr.OwnerOrAdminRequiredErrorMessage
("Only owners and admins can do this.") as plain text from http.Error
And a stale fingerprint answers 409 with apperr.PurgeConflictErrorMessage ("The library changed,
so nothing was purged. Check the new summary and try again.") and a busy run answers 409 with
apperr.PurgeBusyErrorMessage ("A purge is already running.")
And the purge settings range error is apperr.PurgeIntervalErrorMessage ("Enter a whole number of
days between 1 and 365.")
And request and response bodies are JSON with snake_case fields; errors are plain text
And the request body of PATCH is validated by validate tags and validator.Validate.Struct
Scenario Outline: Every purge route refuses callers that are not allowed
When the client sends <method> <path> <credentials>
Then the response status is <status>
And the body is <body>
And no purge setting, image or article is read or changed
Examples:
| method | path | credentials | status | body |
| GET | /v1/media/purge-settings | no cookie | 401 | the existing jwtauth response (status only) |
| PATCH | /v1/media/purge-settings | no cookie | 401 | the existing jwtauth response (status only) |
| GET | /v1/media/purge-preview | no cookie | 401 | the existing jwtauth response (status only) |
| POST | /v1/media/purge | no cookie | 401 | the existing jwtauth response (status only) |
| GET | /v1/media/purge-settings | a stale token_version cookie | 401 | apperr.UnauthorizedErrorMessage |
| GET | /v1/media/purge-settings | cookie of an unverified owner | 403 | apperr.VerificationRequiredErrorMessage |
| GET | /v1/media/purge-settings | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| PATCH | /v1/media/purge-settings | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| GET | /v1/media/purge-preview | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| POST | /v1/media/purge | cookie of a verified user, role "editor" | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| GET | /v1/media/purge-settings | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| PATCH | /v1/media/purge-settings | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| GET | /v1/media/purge-preview | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
| POST | /v1/media/purge | cookie of a verified user, empty role | 403 | apperr.OwnerOrAdminRequiredErrorMessage |
Scenario Outline: Owners and admins both pass the role guard
When the client sends <method> <path> with a valid cookie of a verified <role>
Then the response status is not 401 and not 403
Examples:
| role | method | path |
| owner | GET | /v1/media/purge-settings |
| admin | GET | /v1/media/purge-settings |
| owner | GET | /v1/media/purge-preview |
| admin | GET | /v1/media/purge-preview |
Rule: Purge settings
Scenario: Nothing was ever saved, so the defaults are served
When the client sends GET /v1/media/purge-settings with a valid cookie of the owner
Then the response status is 200
And the body is {"auto_purge_enabled": false, "auto_purge_interval_days": 30,
"last_purge_at": null, "last_purge_merged": 0, "last_purge_articles_updated": 0,
"enabled_at": null, "next_purge_at": null}
Scenario: Turning automatic purge on stamps enabled_at and computes next_purge_at
Given the clock reads 2026-10-05T09:00:00Z and the settings are the defaults
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
Then the response status is 200
And "auto_purge_enabled" is true, "auto_purge_interval_days" is 30
And "enabled_at" is "2026-10-05T09:00:00Z" and "last_purge_at" is null
And "next_purge_at" is "2026-11-04T09:00:00Z"
And a following GET /v1/media/purge-settings returns the same body
Scenario: next_purge_at counts from the last purge when it is later than enabled_at
Given automatic purge is on with interval 10, enabled_at 2026-10-01T00:00:00Z and last_purge_at 2026-10-04T12:00:00Z
When the client sends GET /v1/media/purge-settings
Then "next_purge_at" is "2026-10-14T12:00:00Z"
Scenario: next_purge_at counts from enabled_at when it is later than the last purge
Given automatic purge is on with interval 10, enabled_at 2026-10-05T09:00:00Z and last_purge_at 2026-09-01T00:00:00Z
When the client sends GET /v1/media/purge-settings
Then "next_purge_at" is "2026-10-15T09:00:00Z"
Scenario: Turning automatic purge off and on again never yields a next_purge_at in the past
Given automatic purge was on, last_purge_at is 2026-09-01T00:00:00Z, and it was turned off
And the clock reads 2026-10-05T09:00:00Z and the interval is 30
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
Then "enabled_at" is "2026-10-05T09:00:00Z"
And "last_purge_at" is still "2026-09-01T00:00:00Z"
And "next_purge_at" is "2026-11-04T09:00:00Z"
Scenario: Changing only the interval keeps enabled_at
Given automatic purge is on since 2026-10-01T00:00:00Z with interval 30, no purge yet, and the clock reads 2026-10-05T09:00:00Z
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": 7}
Then the response status is 200
And "auto_purge_interval_days" is 7 and "enabled_at" is still "2026-10-01T00:00:00Z"
And "next_purge_at" is "2026-10-08T00:00:00Z"
Scenario: Patching "on" while already on does not restamp enabled_at
Given automatic purge is on since 2026-10-01T00:00:00Z and the clock reads 2026-10-05T09:00:00Z
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
Then "enabled_at" is still "2026-10-01T00:00:00Z"
Scenario: Turning automatic purge off nulls next_purge_at and keeps the history
Given automatic purge is on with last_purge_at 2026-10-04T12:00:00Z, last_purge_merged 3 and last_purge_articles_updated 5
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": false}
Then "auto_purge_enabled" is false and "next_purge_at" is null
And "last_purge_at", "last_purge_merged" (3) and "last_purge_articles_updated" (5) are unchanged
Scenario Outline: The interval accepts whole numbers from 1 to 365
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": <days>}
Then the response status is 200 and "auto_purge_interval_days" is <days>
Examples:
| days |
| 1 |
| 30 |
| 365 |
Scenario Outline: An invalid patch is 400 and nothing changes
When the client sends PATCH /v1/media/purge-settings with <body>
Then the response status is 400
And the body is <message>
And the stored settings are unchanged
Examples:
| body | message |
| {"auto_purge_interval_days": 0} | apperr.PurgeIntervalErrorMessage |
| {"auto_purge_interval_days": 366} | apperr.PurgeIntervalErrorMessage |
| {"auto_purge_interval_days": -1} | apperr.PurgeIntervalErrorMessage |
| {"auto_purge_interval_days": 1.5} | apperr.BodyErrorMessage |
| {"auto_purge_interval_days": "7"} | apperr.BodyErrorMessage |
| {"auto_purge_enabled": "yes"} | apperr.BodyErrorMessage |
| not json | apperr.BodyErrorMessage |
Scenario: An empty object changes nothing
When the client sends PATCH /v1/media/purge-settings with {}
Then the response status is 200 and the body equals the current settings
Scenario: The use case rejects an out-of-range interval with the domain sentinel
When UpdatePurgeSettings receives interval 0, or 366
Then errors.Is(err, purge.ErrInvalidPurgeInterval) is true
And Repository.Save is not called
Scenario: A repository failure is a generic 500
Given the purge settings repository's next Get fails
When the client sends GET /v1/media/purge-settings
Then the response status is 500 and the body is "Internal server error"
Scenario: A failing Save on PATCH is a generic 500
Given the purge settings repository's next Save fails
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_enabled": true}
Then the response status is 500 and the body is "Internal server error"
Scenario: The purge routes are not taken by the /{id} routes
When the client sends PATCH /v1/media/purge-settings with {"auto_purge_interval_days": 7}
Then the settings are updated and no image is renamed
When the client sends GET /v1/media/purge-preview
Then the preview use case is invoked
Rule: Purge plan and preview
Background:
Given the clock reads 2026-10-05T09:00:00Z
And images are created in id order, oldest first
And articles reference images by their stored name
Scenario: A group with a used copy is planned, merging every other copy into the most used
Given images 1, 2 and 3 share sha256 A with filenames "hero.jpg", "hero-copy.jpg", "hero-2.jpg"
And article 10 uses image 1, articles 11 and 12 use image 2, article 13 uses image 3
When the client sends GET /v1/media/purge-preview
Then the response status is 200
And "groups" is 1, "copies" is 2 and "articles" is 2
And "list" is [{"keep_id": 2, "filename": "hero-copy.jpg", "url": "<stored url of image 2>",
"used_in": 2, "merged": 2}]
And "more" is 0 and "fingerprint" is a 64-character hex string
Scenario: Ties on usage keep the oldest copy, then the lowest id
Given images 4 and 5 share sha256 B, created at the same instant, each used in one article
And images 6 and 7 share sha256 C, image 7 is older than image 6, each used in one article
When the client sends GET /v1/media/purge-preview
Then the "keep_id" of the group B is 4 and the "keep_id" of the group C is 7
Scenario: A single used copy with unused siblings keeps the used one
Given images 8, 9 and 10 share sha256 D and only image 10 is used, in one article
When the client sends GET /v1/media/purge-preview
Then the group has keep_id 10, used_in 1, merged 2
And "articles" is 0, since no article references a merged-away copy
Scenario: A group whose copies are all unused is skipped
Given images 11 and 12 share sha256 E and no article references either
When the client sends GET /v1/media/purge-preview
Then "groups" is 0, "copies" is 0, "articles" is 0, "list" is [] and "more" is 0
Scenario: A library with no duplicates
Given the library holds three images with distinct sha256 values
When the client sends GET /v1/media/purge-preview
Then the response status is 200
And "groups" is 0 and "list" is an empty array in the raw JSON, not null
And "fingerprint" equals the fingerprint of any other empty plan
Scenario: Images without a hash are never grouped
Given images 13 and 14 both have an empty sha256 and both are used
When the client sends GET /v1/media/purge-preview
Then "groups" is 0
Scenario: An article that uses copies of two groups counts once
Given groups A (keep 1, copy 2) and B (keep 3, copy 4) and one article that uses images 2 and 4
When the client sends GET /v1/media/purge-preview
Then "groups" is 2, "copies" is 2 and "articles" is 1
Scenario: An article that uses both the kept copy and a copy is rewritten
Given an article uses images 1 and 2 of one group where image 1 is kept
When the client sends GET /v1/media/purge-preview
Then "articles" is 1
Scenario: The list shows at most 5 groups and reports the rest
Given 7 planned groups
When the client sends GET /v1/media/purge-preview
Then "groups" is 7, "list" has 5 entries in ascending keep_id order and "more" is 2
Scenario: The preview changes nothing
Given a planned group
When the client sends GET /v1/media/purge-preview twice
Then both bodies are equal
And no image is deleted, no article body changes and no purge setting is saved
Scenario Outline: The fingerprint reacts to the plan and to nothing else
Given a planned group with kept image 2 and merged image 3
When <change>
Then the preview fingerprint is <result>
Examples:
| change | result |
| nothing changes | identical |
| a new unused copy is uploaded to the group | different |
| a copy is deleted from the group | different |
| article usage changes so that another copy becomes kept | different |
| an unrelated image with a unique sha256 is uploaded | identical |
| an article body changes without changing which is kept | identical |
| a group with no used copy gains an unused copy | identical |
Rule: Running a purge
Background:
Given the clock reads 2026-10-05T09:00:00Z
Scenario: A run merges every planned group and persists the result
Given group A: images 1 (used in article 10), 2 (used in articles 11 and 12) and 3 (unused)
And group B: images 4 (used in article 12) and 5 (unused)
And group C: images 6 and 7, both unused
When the client sends POST /v1/media/purge with {}
Then the response status is 200
And the body is {"groups_total": 2, "groups_purged": 2, "merged": 3, "articles_updated": 1, "failed_groups": 0}
And images 1 and 3 are gone from the image repository, image 2 remains
And image 5 is gone, image 4 remains, and images 6 and 7 both remain
And article 10's body now uses image 2's stored name
And the stored settings have last_purge_at "2026-10-05T09:00:00Z", last_purge_merged 3 and last_purge_articles_updated 1
And only article 10 is rewritten, since article 12 uses only kept copies
And "auto_purge_enabled", "auto_purge_interval_days" and "enabled_at" are untouched
Scenario: An article rewritten by two groups is counted once in articles_updated
Given an article uses a copy of group A and a copy of group B
When the client sends POST /v1/media/purge with {}
Then "articles_updated" counts that article once, from the union of the groups' UpdatedArticleIDs
Scenario: A single used copy with unused siblings is merged without rewriting any article
Given images 8, 9 and 10 share a hash and only image 10 is used
When the client sends POST /v1/media/purge with {}
Then the body is {"groups_total": 1, "groups_purged": 1, "merged": 2, "articles_updated": 0, "failed_groups": 0}
And images 8 and 9 are gone, image 10 remains and no article body changed
Scenario: A matching fingerprint is accepted
Given the preview returned fingerprint F
When the client sends POST /v1/media/purge with {"fingerprint": "F"}
Then the response status is 200 and the purge ran
Scenario: A stale fingerprint answers 409 and changes nothing
Given the preview returned fingerprint F
And an unused copy is then uploaded to a planned group
When the client sends POST /v1/media/purge with {"fingerprint": "F"}
Then the response status is 409
And the body is apperr.PurgeConflictErrorMessage
And no image is deleted, no article body changes and the stored last_purge_* values are unchanged
Scenario: A fingerprint that matches no plan at all is a conflict, not a validation error
When the client sends POST /v1/media/purge with {"fingerprint": "not-a-real-fingerprint"}
Then the response status is 409 and the body is apperr.PurgeConflictErrorMessage
And nothing changed
Scenario: A run without a fingerprint always runs the current plan
Given the library changed since any preview
When the client sends POST /v1/media/purge with an empty body
Then the response status is 200 and the current plan ran
Scenario: A library with nothing to purge answers zeros and is still recorded
Given the library holds no planned group
When the client sends POST /v1/media/purge with {}
Then the response status is 200
And the body is {"groups_total": 0, "groups_purged": 0, "merged": 0, "articles_updated": 0, "failed_groups": 0}
And the stored settings have last_purge_at "2026-10-05T09:00:00Z", last_purge_merged 0 and last_purge_articles_updated 0
And a scheduler reading next_purge_at sees it move forward from that run
Scenario: The fingerprint of the empty plan on an empty plan runs as a no-op
Given the library holds no planned group
When the client sends POST /v1/media/purge with the empty plan's fingerprint
Then the response status is 200 and the body has groups_total 0
Scenario: A group that fails while rewriting bodies is rolled back and contributes no ids
Given groups A, B and C are planned, in that order
And the article repository's UpdateBody fails for the second article that group B must rewrite
When the client sends POST /v1/media/purge with {}
Then the response status is 200
And the body is {"groups_total": 3, "groups_purged": 2, "merged": <copies of A and C>, "articles_updated": <distinct articles of A and C>, "failed_groups": 1}
And group B's article bodies are back to their original text and all of group B's images remain
And group B's UpdatedArticleIDs is empty, so no article of group B is in "articles_updated"
And the stored last_purge_merged and last_purge_articles_updated equal the response totals
Scenario: A group that fails after rewriting bodies is reported as failed and contributes nothing
Given the image repository's Delete fails for a merged image of group A, and group B is planned
When the client sends POST /v1/media/purge with {}
Then the response status is 200, "failed_groups" is 1 and "groups_purged" is 1
And group A contributes 0 to "merged" and its article ids are not added to "articles_updated", even though its article bodies stay switched to the kept copy
And "articles_updated" and "merged" count only the fully merged group B
And a following GET /v1/media/purge-preview plans group A again, and the next run converges because the plan is recomputed from current data
Scenario: Every group failing is still a 200 with the counters
Given every UpdateBody fails
When the client sends POST /v1/media/purge with {}
Then the response status is 200
And "groups_purged" is 0 and "failed_groups" equals "groups_total"
And the stored last_purge_at is set
Scenario: A second run while one is in progress is rejected as busy, not queued
Given a purge run is in progress and blocked inside the image repository
When a second POST /v1/media/purge arrives, with or without a fingerprint
Then it answers 409 with apperr.PurgeBusyErrorMessage at once
And it changed nothing and saved no last_purge_* values
When the first run is released
Then it finishes normally with its own counters
And a later POST /v1/media/purge is accepted
Scenario: The use case distinguishes a busy run from a stale fingerprint
When RunPurge receives a non-matching fingerprint
Then errors.Is(err, purge.ErrPurgeConflict) is true and errors.Is(err, purge.ErrPurgeBusy) is false
When RunPurge is called while another run holds the guard
Then errors.Is(err, purge.ErrPurgeBusy) is true and errors.Is(err, purge.ErrPurgeConflict) is false
And in both cases no repository write happens
Scenario: A run via the use case with no fingerprint needs no HTTP types
When a scheduler later calls RunPurge with no fingerprint
Then it gets the result struct and the same last_purge_* persistence applies
Scenario: A malformed body is 400
When the client sends POST /v1/media/purge with "not json"
Then the response status is 400 and the body is apperr.BodyErrorMessage
And nothing changed
Scenario: A repository failure while planning is a generic 500 and nothing changes
Given the image repository's ListDuplicates fails
When the client sends POST /v1/media/purge with {}
Then the response status is 500 and the body is "Internal server error"
And no last_purge_* values are saved
Scenario: A failing save of the run result is a logged 500 and the merges stay
Given the purge settings repository's SaveLastRun fails after the merges succeeded
When the client sends POST /v1/media/purge with {}
Then the response status is 500 and the body is "Internal server error"
And the merges that already happened stay done
And the error is written to the injected *log.Logger
And the next POST /v1/media/purge converges, because the plan is recomputed from current data
Scenario: The preview and the run use the same plan
When the client sends GET /v1/media/purge-preview and then POST /v1/media/purge with the returned fingerprint
Then the run's "groups_total" equals the preview's "groups"
And the run's "merged" equals the preview's "copies" and its "articles_updated" equals the preview's "articles"
Rule: MergeMedia keeps its contract while exposing the rewritten ids
Scenario: Existing merges report the same counts and now also the ids
When MergeMedia.Merge succeeds for a group that rewrites articles 1 and 2
Then Merged and ArticlesUpdated are unchanged for existing callers
And UpdatedArticleIDs is [1, 2]
Scenario: A rolled-back merge returns no ids
When MergeMedia.Merge fails while rewriting a body
Then it returns the error and an empty result
@manual
Rule: Persistence and wiring (verified by hand, see the qa.md)
Scenario: The purge settings table is a single row created by AutoMigrate
Given a database with no purge settings table
When the server boots
Then the table exists, GET /v1/media/purge-settings returns the defaults and no row is required beforehand
And repeated PATCH calls keep exactly one row
Scenario: Settings survive a restart
Given automatic purge was turned on with interval 7
When the server restarts
Then GET /v1/media/purge-settings returns the same values
Scenario: A real purge removes the merged files from UPLOAD_DIR
Given duplicates uploaded through the real server
When an owner runs POST /v1/media/purge
Then the merged copies' files are gone from UPLOAD_DIR and the kept file remains