feat: media duplicate purge, part 1 of 3: persisted purge settings and the purge API

This commit is contained in:
fsylvain committed 2026-10-05 18:24:03 +02:00
1 parent 161ddc0f7d
commit e82d0ce254
25 files changed
+2459 -41

No files matched your search

+15
View File
@@ -30,6 +30,21 @@ Nothing under `internal/` imports `api/`.
| `api/pkg/upload` | Multipart `file` reading and the mapping of upload errors (415, 413, 400) shared by both upload routes. |
| `api/pkg/publicuser` | `PublicUser`, the JSON shape of a user with no password, codes or token version. `publicuser.New` converts a domain `user.User`, so handlers never serialise the domain struct directly. |
### Media purge
`internal/domain/purge` holds `Settings` (automatic purge on or off, interval of 1 to 365 days, `last_purge_*` history, `enabled_at`), the `Repository` interface (`Get` returns the defaults when nothing was saved, `Save`) and the sentinels `ErrInvalidPurgeInterval`, `ErrPurgeConflict` (stale fingerprint) and `ErrPurgeBusy` (a run is already in progress). `Settings.NextPurgeAt` is `max(last_purge_at, enabled_at)` plus the interval when automatic purge is on, and null otherwise. The settings live in a single-row `purge_settings` table (`models.PurgeSettings`, `mappers.PurgeSettings*`, `persistence.PurgeSettingsRepository`), created by `AutoMigrate`.
The use cases `GetPurgeSettings`, `UpdatePurgeSettings`, `PreviewPurge` and `RunPurge` depend only on domain interfaces, an injected clock and an injected `*log.Logger`, so a scheduler can call them. `PreviewPurge` and `RunPurge` share one plan (`purgePlan.go`): a duplicate group (same non-empty sha256) is planned only when a copy is used in an article, the most used copy is kept (ties: oldest `created_at`, then lowest id) and every other copy is merged into it. The fingerprint is a SHA-256 over the planned groups. `RunPurge` takes a non-blocking in-process lock (`ErrPurgeBusy` for a concurrent call), refuses a non-matching fingerprint with `ErrPurgeConflict`, merges group by group with `MergeMedia` (a failing group is counted and skipped), unions the `UpdatedArticleIDs` of the fully merged groups, and records `last_purge_*` after every run.
`middleware.RequireOwnerOrAdmin` lets only the roles owner and admin through (403 otherwise, including an empty role). The media router mounts these routes after the verified chain and before the `/{id}` routes:
| Route | Purpose |
|---|---|
| `GET /v1/media/purge-settings` | Settings with `next_purge_at`. |
| `PATCH /v1/media/purge-settings` | Partial update of `auto_purge_enabled` and `auto_purge_interval_days`. |
| `GET /v1/media/purge-preview` | Groups, copies, distinct articles, the first 5 groups, `more` and the fingerprint. |
| `POST /v1/media/purge` | Runs the purge. Optional `fingerprint`; 409 when stale or when a run is in progress. |
### Image metadata backfill
At boot, `BackfillImageMetadata` fills `filename`, `size`, `content_type`, `width`, `height` and `sha256` for image rows that lack them, reading the files in `UPLOAD_DIR`. It is idempotent, and a row whose file is missing or unreadable is left as it is without stopping the boot. The old `article_id` column and the `article_images` table are kept and never written.