feat: media duplicate purge, part 2 of 3: the automatic purge scheduler (Go only)
This commit is contained in:
1 parent
e82d0ce254
commit
1a55a39425
18 files changed
+1389
-39
No files matched your search
@@ -32,10 +32,18 @@ Nothing under `internal/` imports `api/`.
|
||||
|
||||
### Media purge
|
||||
|
||||
`internal/domain/purge` holds `Settings` (automatic purge on or off, interval of 1 to 365 days, `last_purge_*` history, `enabled_at`), the `Repository` interface (`Get` returns the defaults when nothing was saved, `Save`) and the sentinels `ErrInvalidPurgeInterval`, `ErrPurgeConflict` (stale fingerprint) and `ErrPurgeBusy` (a run is already in progress). `Settings.NextPurgeAt` is `max(last_purge_at, enabled_at)` plus the interval when automatic purge is on, and null otherwise. The settings live in a single-row `purge_settings` table (`models.PurgeSettings`, `mappers.PurgeSettings*`, `persistence.PurgeSettingsRepository`), created by `AutoMigrate`.
|
||||
`internal/domain/purge` holds `Settings` (automatic purge on or off, interval of 1 to 365 days, `last_purge_*` history, `enabled_at`), the `Repository` interface (`Get` returns the defaults when nothing was saved, `Save`, and `SaveLastRun`, which writes only the `last_purge_*` columns so a settings change made during a run is never overwritten) and the sentinels `ErrInvalidPurgeInterval`, `ErrPurgeConflict` (stale fingerprint) and `ErrPurgeBusy` (a run is already in progress). `Settings.NextPurgeAt` is `max(last_purge_at, enabled_at)` plus the interval when automatic purge is on, and null otherwise. The settings live in a single-row `purge_settings` table (`models.PurgeSettings`, `mappers.PurgeSettings*`, `persistence.PurgeSettingsRepository`), created by `AutoMigrate`.
|
||||
|
||||
The use cases `GetPurgeSettings`, `UpdatePurgeSettings`, `PreviewPurge` and `RunPurge` depend only on domain interfaces, an injected clock and an injected `*log.Logger`, so a scheduler can call them. `PreviewPurge` and `RunPurge` share one plan (`purgePlan.go`): a duplicate group (same non-empty sha256) is planned only when a copy is used in an article, the most used copy is kept (ties: oldest `created_at`, then lowest id) and every other copy is merged into it. The fingerprint is a SHA-256 over the planned groups. `RunPurge` takes a non-blocking in-process lock (`ErrPurgeBusy` for a concurrent call), refuses a non-matching fingerprint with `ErrPurgeConflict`, merges group by group with `MergeMedia` (a failing group is counted and skipped), unions the `UpdatedArticleIDs` of the fully merged groups, and records `last_purge_*` after every run.
|
||||
|
||||
#### Automatic purge scheduler
|
||||
|
||||
`PurgeSchedulerUseCase.Tick(now)` returns the `purge.RunResult` of the run, whether it ran and an error. It reads the purge settings on every call and runs `RunPurge.Run(nil)` (no fingerprint) when `Settings.NextPurgeAt()` is set and `now` is not before it. It never writes settings: only `RunPurge` records `last_purge_*`, through `SaveLastRun`, which moves the anchor forward even when nothing was purged. The runner in `internal/infrastructure/scheduler` calls `Tick` and logs one line with `groups_purged`, `merged`, `articles_updated` and `failed_groups` when a run happened. It calls `Tick` once after a startup delay of 1 minute, which catches up a run missed during downtime in a single purge, then once every hour. `api/dependecyInjection.go` builds it and `cmd/server/main.go` starts it with the signal context of the server. Changes to the settings apply on the next tick, with no restart.
|
||||
|
||||
- Overlap guard: a manual run holds the `RunPurge` lock, so `Tick` returns `ErrPurgeBusy` at once. The runner treats it as a skip, logs nothing and re-evaluates at the next tick.
|
||||
- Failures: any other error is logged and the loop keeps going. There is no retry inside a tick; the next hourly tick evaluates the schedule again.
|
||||
- Shutdown: on SIGINT or SIGTERM the runner stops taking ticks and waits, with no time limit, for a `Tick` in progress. `main` waits for the runner to return before the database is closed.
|
||||
|
||||
`middleware.RequireOwnerOrAdmin` lets only the roles owner and admin through (403 otherwise, including an empty role). The media router mounts these routes after the verified chain and before the `/{id}` routes:
|
||||
|
||||
| Route | Purpose |
|
||||
|
||||
Reference in new issue
Block a user