feat: activity log backend foundation

This commit is contained in:
fsylvain committed 2026-10-06 20:35:03 +02:00
1 parent 8907bb49fe
commit 50b1649965
74 files changed
+5949 -239

No files matched your search

+22
View File
@@ -27,6 +27,28 @@ for example `https://cms.example/v1/article/`.
| Users and invites | GET /v1/user/, PATCH /v1/user/me, POST /v1/user/invite (owner only), DELETE /v1/user/invite/{id} (owner only), POST /v1/user/invite/{id}/renew (owner only), GET /v1/auth/me |
| Settings | GET /v1/settings |
## Activity log
The activity routes are session only: a key answers 403 "This action needs a real session." and a
key never reads the audit trail. Events are append-only; there is no `POST`, `PUT`, `PATCH` or
`DELETE` under `/v1/activity`.
| Route | Who | Purpose |
| --- | --- | --- |
| `GET /v1/activity` | Any verified user | Filtered, paged events. Owner and admin see every event; any other role is scoped to its own events, and `actor` set to someone else answers 403 "Only owners and admins can do this." |
| `GET /v1/activity/summary` | Owner, admin | Counts for the range, the previous period, active people and seven-bucket series. |
| `GET /v1/activity/export` | Owner, admin | The same filters as the list without `page` or `size`, as a `text/csv; charset=utf-8` attachment (`when,actor,action,target,outcome,source,ip`). Cells starting with `=`, `+`, `-` or `@` get a leading quote. |
Query: `q`, `actor` (`user:ID`, `key:ID`, `system`), `category` (`articles`, `media`, `users`,
`api_keys`, `auth`, `system`), `outcome` (`all`, `success`, `failed`), `security` (`true`), `range`
(`today`, `7d`, `30d`, `custom`; default `7d`), `from` and `to` (dates, with `custom`), `page` (from
1) and `size` (25, 50 or 100). Any invalid value answers 400 with the body-mismatch message. Dates
are UTC. The list answers `{ items, total, page, size }`, newest first (`created_at`, then `id`).
`GET /v1/settings` also returns `activity_retention_days` (30, 90, 365 or 0 for forever, default
90), `activity_last_cleanup_at` and `activity_last_cleanup_removed`. `PATCH /v1/settings` with
`{"activity_retention_days": 30}` is session only and owner only (403 "Only the owner can do this.");
any other value answers 400.
Article routes accept `?format=markdown|html|json` (default `markdown`). `GET /v1/media/` accepts
`q`, `hash`, `cursor`, `filter`, `sort` and `limit`.
+12
View File
@@ -55,6 +55,18 @@ The use cases `GetPurgeSettings`, `UpdatePurgeSettings`, `PreviewPurge` and `Run
| `GET /v1/media/purge-preview` | Groups, copies, distinct articles, the first 5 groups, `more` and the fingerprint. |
| `POST /v1/media/purge` | Runs the purge. Optional `fingerprint`; 409 when stale or when a run is in progress. |
### Activity log
`internal/domain/audit` holds the append-only `Event` (actor, target and failure-reason snapshots, outcome, security flag, source, ip, user agent, `params` and `changes`), the closed vocabularies (actor type, category, outcome, source, change kind), the `Filter`, `Summary` and `Settings` types, the `Repository` interface (`Append`, `List`, `Summary`, `DeleteOlderThan`: no way to edit or delete one event), the `SettingsRepository` (`Get`, `SaveRetention`, `SaveLastCleanup`; no row means 90 days) and the sentinels `ErrInvalidFilter`, `ErrInvalidRetention`, `ErrInvalidEvent` and `ErrActorNotAllowed`. The `Recorder` port is `Record(ctx, event)`; it returns nothing, so a caller can never fail on it.
`internal/infrastructure/auditing.Recorder` is the adapter. It fills a missing actor, source, ip and user agent from the `audit.Origin` carried by the request context, stamps `created_at` from the injected clock in UTC, bounds long names, drops an event outside the vocabulary, and recovers from any error or panic of the repository by writing one line to the process log (the action and the error only). `middleware.AuditContext` builds the origin after the credential middleware: a session gives a `user` actor and source `web`, an API key an `api_key` actor and source `api_key`, no valid credential an `anonymous` actor; the ip is the peer address, or `X-Real-IP` when the peer is a trusted proxy; the user agent is cut to 255 characters. Use cases that produce events (authenticate and logout, password reset request and completion, API key create and revoke, retention change, cleanup) take the context and an `audit.Recorder`; secrets (passwords, raw keys, reset codes, tokens and their hashes) are never put in an event or in its log line.
Persistence is `models.AuditEvent` (`audit_events`, indexes on `created_at`, `(actor_type, actor_id, created_at)` and `(category, created_at)`; `params` and `changes` are JSON text) and `models.AuditSettings` (one row), both in `AutoMigrate`. The queries use only portable SQL (`LOWER(...) LIKE ... ESCAPE '!'`, `COUNT(DISTINCT ...)`, `CASE` sums), so they run on SQLite, MySQL/MariaDB and PostgreSQL. The summary splits the range into seven equal buckets, oldest first.
The use cases `ListActivity`, `GetActivitySummary` and `ExportActivity` turn the raw query into a domain filter (`range` defaults to 7 days; `custom` needs two dates, `to` included), scope a caller who is not owner or admin to its own events, and resolve `target.exists` at read time. The export reads the log through `List` in pages of 500 and writes the CSV with `encoding/csv`. `/v1/activity` and the settings `PATCH` sit behind `RequireSession`, the verifier, the current-token and verified-user checks; summary and export also behind `RequireOwnerOrAdmin`, the settings `PATCH` behind `RequireOwner`.
`CleanupActivityUseCase.Tick(now)` follows the `PurgeScheduler` model. With a retention of 0 it never runs. Otherwise it is due once a day, at or after 03:00 in the location of `now` and when the last cleanup is before today's 03:00. It deletes events older than `now` minus the retention days (never newer than 24 hours), records the run (`activity_last_cleanup_at`, `activity_last_cleanup_removed`) even when nothing was removed, and only then records a `system.activity_cleanup` event when something was removed. A failing delete returns the error and leaves the last-cleanup fields as they were, so the next tick retries. `scheduler.Runner` now wraps a task with a label: `NewRunner` keeps the purge behaviour and log line, and `NewCleanupRunner` ticks the cleanup. `UseCases.RunSchedulers` runs both runners and `cmd/server/main.go` starts it with the server's signal context.
### Image metadata backfill
At boot, `BackfillImageMetadataUseCase` (through `image.Repository.ListMissingMetadata` and `UpdateMetadata`) fills `filename`, `size`, `content_type`, `width`, `height` and `sha256` for image rows that lack them, reading the bytes through `image.Storage` (the files in `UPLOAD_DIR` with the local adapter, the bucket objects with the S3 adapter). It is idempotent, and a row whose file is missing or unreadable is left as it is without stopping the boot. The old `article_id` column and the `article_images` table are kept and never written.
+4
View File
@@ -103,6 +103,10 @@ Relative paths are resolved against the process's working directory. In producti
For example, `WorkingDirectory=/var/lib/renewcms` with `DB_FILE=./data/renewcms.db` puts the database in `/var/lib/renewcms/data`. Browser-update `--data-dir` must name that directory, because the app reads helper status and update records beside the database. It is not necessarily `WorkingDirectory` or `ReadWritePaths`. Do not change an existing `DB_FILE` merely to match the helper: that could open a different, empty database. See [browser-update setup](deployment.md#browser-updates).
## Activity log retention
The activity log needs no environment variable. Its retention period (30, 90, 365 days or forever, 90 by default) is a setting the owner changes with `PATCH /v1/settings`, and the daily cleanup runs at 03:00 in the server's time zone (`TZ`, or the host's zone).
## Adding a variable
1. Read it in the code where it is needed (infrastructure or `api/`, never `internal/domain`).