RenewCMS/docs/architecture.md

# Architecture

RenewCMS is a Go HTTP API and a Vue 3 SPA, shipped as a single binary.

## Backend layering

Dependencies point inward:

```
api/  ──▶  internal/infrastructure/useCases  ──▶  internal/domain  ◀──  internal/infrastructure/{persistence,mailer}
```

| Layer | Location | Rules |
|---|---|---|
| Domain | `internal/domain/<name>/` | Entities, `Err…` sentinel errors, `Repository` interfaces. Imports only the stdlib and other domain packages, with no GORM tags. |
| Use cases | `internal/infrastructure/useCases/` | Application logic. Depends only on domain interfaces, never on persistence, mailer, GORM or `net/http`. |
| Adapters | `internal/infrastructure/persistence/{models,mappers,repositories}`, `internal/infrastructure/mailer` | Implement a domain `Repository` (assert with `var _ domain.Repository = &X{}`). GORM structs stay in `models` and are converted by `mappers`. |
| Database drivers | `internal/infrastructure/persistence/database` | `database.Connect(Config{Driver, Source})` looks the driver up in a registry and returns a `Connection` (GORM handle, `NewBackup`, `Close`). A `Driver` implements `Open` and `NewBackup`, so a new database is one new file plus one registry entry. `database.Migrate` owns the `AutoMigrate` model list and the user-role backfill. Only `sqlite` exists today. |
| Blob storage | `domain/image.Storage`, `internal/infrastructure/storage` | `Storage` (`Put`, `Open`, `Delete`) is the port for uploaded image bytes. `storage.Open(Config{Driver, Location})` picks the adapter; `local` is the only one. Repositories never touch files: use cases coordinate `Storage` and `image.Repository`. |
| Delivery | `api/handlers/<name>/`, `api/router.go` | HTTP handlers receive use cases through `NewHandler(...)`. They never import persistence, mailer or GORM. |
| Composition root | `api/dependecyInjection.go` | The only place that picks the database and storage drivers, builds repositories and use cases, and runs `database.Migrate` and the image metadata backfill at boot. |
| Entry point | `cmd/server/main.go` | Env/config, HTTP server, graceful shutdown. |

Nothing under `internal/` imports `api/`.

### Delivery packages

| Package | Role |
|---|---|
| `api/handlers/user` | Authenticated user endpoints: profile update (`UpdateMe`) and the users list with invitations (`CreateInvite`, `ListUsers`). |
| `api/handlers/media` | The media library under `/v1/media` (list, upload, rename, delete, merge, usage). Images belong to the library, not to an article: deleting an article never deletes an image, and an image's "used in" is read from article bodies each time. |
| `api/pkg/upload` | Multipart `file` reading and the mapping of upload errors (415, 413, 400) shared by both upload routes. |
| `api/pkg/publicuser` | `PublicUser`, the JSON shape of a user with no password, codes or token version. `publicuser.New` converts a domain `user.User`, so handlers never serialise the domain struct directly. |

### Media purge

`internal/domain/purge` holds `Settings` (automatic purge on or off, interval of 1 to 365 days, `last_purge_*` history, `enabled_at`), the `Repository` interface (`Get` returns the defaults when nothing was saved, `Save`, and `SaveLastRun`, which writes only the `last_purge_*` columns so a settings change made during a run is never overwritten) and the sentinels `ErrInvalidPurgeInterval`, `ErrPurgeConflict` (stale fingerprint) and `ErrPurgeBusy` (a run is already in progress). `Settings.NextPurgeAt` is `max(last_purge_at, enabled_at)` plus the interval when automatic purge is on, and null otherwise. The settings live in a single-row `purge_settings` table (`models.PurgeSettings`, `mappers.PurgeSettings*`, `persistence.PurgeSettingsRepository`), created by `AutoMigrate`.

The use cases `GetPurgeSettings`, `UpdatePurgeSettings`, `PreviewPurge` and `RunPurge` depend only on domain interfaces, an injected clock and an injected `*log.Logger`, so a scheduler can call them. `PreviewPurge` and `RunPurge` share one plan (`purgePlan.go`): a duplicate group (same non-empty sha256) is planned only when a copy is used in an article, the most used copy is kept (ties: oldest `created_at`, then lowest id) and every other copy is merged into it. The fingerprint is a SHA-256 over the planned groups. `RunPurge` takes a non-blocking in-process lock (`ErrPurgeBusy` for a concurrent call), refuses a non-matching fingerprint with `ErrPurgeConflict`, merges group by group with `MergeMedia` (a failing group is counted and skipped), unions the `UpdatedArticleIDs` of the fully merged groups, and records `last_purge_*` after every run.

#### Automatic purge scheduler

`PurgeSchedulerUseCase.Tick(now)` returns the `purge.RunResult` of the run, whether it ran and an error. It reads the purge settings on every call and runs `RunPurge.Run(nil)` (no fingerprint) when `Settings.NextPurgeAt()` is set and `now` is not before it. It never writes settings: only `RunPurge` records `last_purge_*`, through `SaveLastRun`, which moves the anchor forward even when nothing was purged. The runner in `internal/infrastructure/scheduler` calls `Tick` and logs one line with `groups_purged`, `merged`, `articles_updated` and `failed_groups` when a run happened. It calls `Tick` once after a startup delay of 1 minute, which catches up a run missed during downtime in a single purge, then once every hour. `api/dependecyInjection.go` builds it and `cmd/server/main.go` starts it with the signal context of the server. Changes to the settings apply on the next tick, with no restart.

- Overlap guard: a manual run holds the `RunPurge` lock, so `Tick` returns `ErrPurgeBusy` at once. The runner treats it as a skip, logs nothing and re-evaluates at the next tick.
- Failures: any other error is logged and the loop keeps going. There is no retry inside a tick; the next hourly tick evaluates the schedule again.
- Shutdown: on SIGINT or SIGTERM the runner stops taking ticks and waits, with no time limit, for a `Tick` in progress. `main` waits for the runner to return before the database is closed.

`middleware.RequireOwnerOrAdmin` lets only the roles owner and admin through (403 otherwise, including an empty role). The media router mounts these routes after the verified chain and before the `/{id}` routes:

| Route | Purpose |
|---|---|
| `GET /v1/media/purge-settings` | Settings with `next_purge_at`. |
| `PATCH /v1/media/purge-settings` | Partial update of `auto_purge_enabled` and `auto_purge_interval_days`. |
| `GET /v1/media/purge-preview` | Groups, copies, distinct articles, the first 5 groups, `more` and the fingerprint. |
| `POST /v1/media/purge` | Runs the purge. Optional `fingerprint`; 409 when stale or when a run is in progress. |

### Image metadata backfill

At boot, `BackfillImageMetadataUseCase` (through `image.Repository.ListMissingMetadata` and `UpdateMetadata`) fills `filename`, `size`, `content_type`, `width`, `height` and `sha256` for image rows that lack them, reading the bytes through `image.Storage` (the files in `UPLOAD_DIR` with the local adapter, the bucket objects with the S3 adapter). It is idempotent, and a row whose file is missing or unreadable is left as it is without stopping the boot. The old `article_id` column and the `article_images` table are kept and never written.

## Frontend ↔ backend

```
web/ (Vue, Vite)
  │  task build:web  (vue-tsc + vite build)
  ▼
internal/infrastructure/assets/dist/        gitignored build output
  │  //go:embed all:dist   (assets_embed.go, default build)
  ▼
bin/renewcms  ──  /v1/*  API
               ──  /healthz
               ──  /*    SPA (api/handlers/frontend), index.html fallback
```

- **Production:** a single origin. The SPA calls the API with relative URLs, so the JWT cookie works without CORS. Files under `assets/` are content-hashed by Vite and cached for a year. `index.html` is never cached.
- **Development:** built with `-tags nofrontend` (`assets_stub.go`, `assets.Dist == nil`). Vite serves the SPA on :5173 and proxies `/v1` to the API on :8080. See [development.md](development.md).

## Storage and database seams

Uploaded images are addressed by a stored name (`<uuid>.<ext>`). `CreateImageUseCase` calls `Storage.Put` (which generates the name), then `Repository.Create` with `Path = /static/uploadedImages/<name>`, and deletes the blob again if the row cannot be created. `DeleteMediaUseCase` and `MergeMediaUseCase` remove the blob first and the row second (`mediaRemover`), so a failed blob delete keeps the row and the delete can be retried. `GET /static/uploadedImages/<name>` is served by `OpenMediaUseCase` through `api/handlers/staticimages`, which uses `http.ServeContent` when the blob is seekable and streams it otherwise; `ErrBlobNotFound` maps to 404 and any other error to 500.

SQL in repositories must run on SQLite, MySQL/MariaDB and PostgreSQL: substring searches use `LIKE ... ESCAPE '!'` through `containsPattern` instead of `instr`. `Settings.DataDir()` is where update state, staging and backups live; it is `DATA_DIR` when set and the directory of `DB_FILE` otherwise, so it does not depend on where the database is.

Three drivers are registered in `database.drivers`: `sqlite` (default), `mysql` (MySQL 8 and MariaDB 11, through `gorm.io/driver/mysql`) and `postgres` (PostgreSQL 16 and 17, alias `postgresql`, through `gorm.io/driver/postgres` on pgx v5); all are pure Go so `CGO_ENABLED=0` and the six release targets still build. `DB_DRIVER` picks one in `databaseConfig` (composition root); an unknown name fails in `database.Connect` with `unsupported database driver "<name>"`. `database.MySQLDSN` and `database.PostgresDSN` build the DSN from separate variables (the PostgreSQL one as an escaped URL with `sslmode`, `timezone=UTC` and a connect timeout), and the driver redacts the password from every error and names only host and port. `Connect` applies the optional `PoolSettings` to any driver. What keeps the repositories identical on both engines, all without touching the domain, use cases or handlers:

- Indexed and unique strings carry `size:` tags in the models (a MySQL index needs a bounded `varchar`). SQLite maps them to the same `text` column, so an existing SQLite file migrates with no schema or row change (`test/sqlite_schema_unchanged_test.go` builds a file from the old model definitions and compares `sqlite_master` and every row).
- `Migrate` creates MySQL tables with `utf8mb4_bin` (`gorm:table_options`), so `Bob` and `bob` are distinct like in SQLite. The DSN also sets `collation=utf8mb4_bin`.
- The DSN sets `clientFoundRows=true`: an update that matches a row but changes nothing still reports one affected row, as in SQLite. `InviteRepository.Renew`, `deleteOrNotFound` and `PurgeSettingsRepository.SaveLastRun` depend on that.
- The Go driver sends a zero `time.Time` as `0000-00-00`, which MySQL's default `NO_ZERO_DATE` rejects, so each session sets `sql_mode` to the server default minus `NO_ZERO_DATE`. Reading `0000-00-00` gives the zero time back.
- `UserRepository.Create` returns the filled model instead of re-reading it, because MySQL has no `RETURNING`.

On PostgreSQL the MySQL-only table options are not applied, pgx returns `timestamptz` values in the local zone so a query callback in the driver converts them to UTC, and only the `public` schema is used.

Backups are a `domain/update.DatabaseBackup` per driver. SQLite uses `VACUUM INTO`. MySQL uses `DumpBackup`, a pure Go logical dump (`SHOW CREATE TABLE` plus every row, strings escaped and binary values as hex, NULL and zero dates kept) written inside one repeatable-read transaction to `DATA_DIR/backups/renewcms-<time>.sql.partial` and renamed once complete. Any failure removes the partial file and returns an error, so the update download fails instead of claiming a backup. PostgreSQL reuses `DumpBackup` with its own `dumpDialect` (double-quoted identifiers, doubled single quotes, `\x` bytea literals): `CREATE TABLE` is rebuilt from `pg_catalog` (serial columns, defaults, primary key, unique and check constraints), followed by the indexes, the rows and a `setval` per `id` sequence, all inside one repeatable-read read-only transaction. All drivers keep the newest 3 files (`pruneBackups`).

`task test` runs everything that needs no server. The repository contract tests (`test/repository_contract_test.go`) run on SQLite always, and on MySQL when `RENEWCMS_TEST_MYSQL_DSN` points at a server (admin DSN; each test creates and drops its own database). `task test:mysql` starts `mysql:8.4` and then `mariadb:11` in Docker and runs the `MySQL` tests against each. The same contract tests run on PostgreSQL when `RENEWCMS_TEST_POSTGRES_DSN` points at a server (admin DSN, for example `postgres://postgres:pw@127.0.0.1:5432/postgres?sslmode=disable`), and `task test:postgres` starts `postgres:16` then `postgres:17` in Docker and runs the `Postgres` tests against each.

Two storage adapters implement `image.Storage`: `local` (default, files in `UPLOAD_DIR`) and `s3` (`storage.S3Storage`, `aws-sdk-go-v2` over plain HTTPS, pure Go so `CGO_ENABLED=0` still holds). `STORAGE_DRIVER` picks one in `imageStorageConfig` (composition root, from `config.StorageSettings`); `storage.Open` fails with `unsupported storage driver "<name>"` for anything else, and `openImageStorage` panics with the error. The S3 adapter works as follows, with no change to the domain, use cases, repositories or the `staticimages` handler:

- `Open` calls `GetObject` and returns a `Blob` whose `Content` is an `io.ReadSeeker` (`s3Reader`): `Size` and `ModTime` come from `ContentLength` and `LastModified`, the first bytes are read from that response and every `Seek` to another offset makes the next `Read` issue a ranged `GetObject`. `http.ServeContent` therefore answers `Range` (206, 416) and `If-Modified-Since` (304) with no `ETag`. A 404 maps to `image.ErrBlobNotFound`; any other failure is a plain error (500 in the handler).
- `Put` does one `PutObject` of `<S3_KEY_PREFIX><uuid>.<ext>` with the upload's `Content-Type`. `Delete` treats 404 as success. Names that are not plain file names are never requested, as in the local adapter.
- Each call has a 30 second timeout to get a response (the body then streams without a deadline) and the SDK default retries. Request checksums are only sent when required, so older S3-compatible servers work.
- `storage.Open` checks the bucket with `HeadBucket`. The failure text names the bucket and the reason (404, 403, other status or transport error) and never the credentials.
- At boot, `RunBackfillReportingReads` wraps the storage in a counter for the `BackfillImageMetadataUseCase` run and logs how many images it read; the use case itself is unchanged.
- Update backups (`DATA_DIR`) never include uploads, whatever the driver.

`task test:s3` starts SeaweedFS in Docker and runs `TestS3StorageIntegration` against it (skipped without Docker, and not part of `task test`). Everything else uses an in-process fake S3 server listening on `127.0.0.1`.

Adding a database driver: implement `database.Driver` (`Open`, and `NewBackup` returning a `domain/update.DatabaseBackup`), register it in `drivers`, then select it in `databaseConfig` in the composition root (and add the env vars to `main.go`, `.env.example` and `configuration.md`). Adding a storage backend: implement `image.Storage`, add a case to `storage.Open`, and select it in `openImageStorage`. No use case, handler or repository changes.