feat: optional S3-compatible storage for uploaded images, instead of the local upload directory

docs/architecture.md

@@ -57,7 +57,7 @@ The use cases `GetPurgeSettings`, `UpdatePurgeSettings`, `PreviewPurge` and `Run
 
 ### Image metadata backfill
 
-At boot, `BackfillImageMetadataUseCase` (through `image.Repository.ListMissingMetadata` and `UpdateMetadata`) fills `filename`, `size`, `content_type`, `width`, `height` and `sha256` for image rows that lack them, reading the bytes through `image.Storage` (the files in `UPLOAD_DIR` with the local adapter). It is idempotent, and a row whose file is missing or unreadable is left as it is without stopping the boot. The old `article_id` column and the `article_images` table are kept and never written.
+At boot, `BackfillImageMetadataUseCase` (through `image.Repository.ListMissingMetadata` and `UpdateMetadata`) fills `filename`, `size`, `content_type`, `width`, `height` and `sha256` for image rows that lack them, reading the bytes through `image.Storage` (the files in `UPLOAD_DIR` with the local adapter, the bucket objects with the S3 adapter). It is idempotent, and a row whose file is missing or unreadable is left as it is without stopping the boot. The old `article_id` column and the `article_images` table are kept and never written.
 
 ## Frontend ↔ backend
 
@@ -94,4 +94,15 @@ Backups are a `domain/update.DatabaseBackup` per driver. SQLite uses `VACUUM INT
 
 `task test` runs everything that needs no server. The repository contract tests (`test/repository_contract_test.go`) run on SQLite always, and on MySQL when `RENEWCMS_TEST_MYSQL_DSN` points at a server (admin DSN; each test creates and drops its own database). `task test:mysql` starts `mysql:8.4` and then `mariadb:11` in Docker and runs the `MySQL` tests against each.
 
+Two storage adapters implement `image.Storage`: `local` (default, files in `UPLOAD_DIR`) and `s3` (`storage.S3Storage`, `aws-sdk-go-v2` over plain HTTPS, pure Go so `CGO_ENABLED=0` still holds). `STORAGE_DRIVER` picks one in `imageStorageConfig` (composition root, from `config.StorageSettings`); `storage.Open` fails with `unsupported storage driver "<name>"` for anything else, and `openImageStorage` panics with the error. The S3 adapter works as follows, with no change to the domain, use cases, repositories or the `staticimages` handler:
+
+- `Open` calls `GetObject` and returns a `Blob` whose `Content` is an `io.ReadSeeker` (`s3Reader`): `Size` and `ModTime` come from `ContentLength` and `LastModified`, the first bytes are read from that response and every `Seek` to another offset makes the next `Read` issue a ranged `GetObject`. `http.ServeContent` therefore answers `Range` (206, 416) and `If-Modified-Since` (304) with no `ETag`. A 404 maps to `image.ErrBlobNotFound`; any other failure is a plain error (500 in the handler).
+- `Put` does one `PutObject` of `<S3_KEY_PREFIX><uuid>.<ext>` with the upload's `Content-Type`. `Delete` treats 404 as success. Names that are not plain file names are never requested, as in the local adapter.
+- Each call has a 30 second timeout to get a response (the body then streams without a deadline) and the SDK default retries. Request checksums are only sent when required, so older S3-compatible servers work.
+- `storage.Open` checks the bucket with `HeadBucket`. The failure text names the bucket and the reason (404, 403, other status or transport error) and never the credentials.
+- At boot, `RunBackfillReportingReads` wraps the storage in a counter for the `BackfillImageMetadataUseCase` run and logs how many images it read; the use case itself is unchanged.
+- Update backups (`DATA_DIR`) never include uploads, whatever the driver.
+
+`task test:s3` starts SeaweedFS in Docker and runs `TestS3StorageIntegration` against it (skipped without Docker, and not part of `task test`). Everything else uses an in-process fake S3 server listening on `127.0.0.1`.
+
 Adding a database driver: implement `database.Driver` (`Open`, and `NewBackup` returning a `domain/update.DatabaseBackup`), register it in `drivers`, then select it in `databaseConfig` in the composition root (and add the env vars to `main.go`, `.env.example` and `configuration.md`). Adding a storage backend: implement `image.Storage`, add a case to `storage.Open`, and select it in `openImageStorage`. No use case, handler or repository changes.

docs/configuration.md

@@ -27,7 +27,8 @@ Copy [`.env.example`](../.env.example) to `.env` to get started. When a required
 | `DB_DRIVER`            | `sqlite`             | `sqlite` or `mysql` (MySQL and MariaDB). Any other value stops the server with `unsupported database driver "<value>"`. See [Database](#database).                                                                             |
 | `DB_FILE`              | `./data/renewcms.db` | Path to the SQLite database file. Missing parent directories are created. The schema is migrated automatically at startup. Ignored when `DB_DRIVER=mysql`.                                                                     |
 | `DATA_DIR`             | directory of `DB_FILE` | Directory for update state, staged updates and database backups (`backups/`). Set it with `DB_DRIVER=mysql`, where there is no database file to sit beside (the default still follows `DB_FILE`). With SQLite, leave it unset to keep everything beside the database. |
-| `UPLOAD_DIR`           | `./data/uploads`     | Directory where uploaded images are stored.                                                                                                                                                                                     |
+| `STORAGE_DRIVER`       | `local`              | `local` (files in `UPLOAD_DIR`) or `s3` (an S3-compatible bucket). Any other value stops the server with `unsupported storage driver "<value>"`. See [Image storage](#image-storage). |
+| `UPLOAD_DIR`           | `./data/uploads`     | Directory where uploaded images are stored. Ignored, without error, when `STORAGE_DRIVER=s3`.                                                                                                                                                                                     |
 | `CORS_ALLOWED_ORIGINS` | *(empty)*            | Semicolon-separated list of origins allowed to call the API. When empty, no CORS headers are sent. That is the right setting whenever the SPA is served by RenewCMS itself (production) or through the Vite proxy (development). |
 | `AUTH_RATE_LIMIT_MAX_ATTEMPTS` | `5`           | Maximum number of requests a single client IP may send to `POST /v1/auth/login`, `POST /v1/auth/password-reset` or the invite-accept pair (`GET /v1/auth/invite` and `POST /v1/auth/invite/accept`, sharing one counter) within the window, tracked independently per route.                                                          |
 | `AUTH_RATE_LIMIT_WINDOW_SECONDS` | `60`        | Length, in seconds, of the fixed rate-limit window for the routes above.                                                                                                                                                    |
@@ -37,6 +38,38 @@ Copy [`.env.example`](../.env.example) to `.env` to get started. When a required
 
 Automatic media purge has no environment variable. Its schedule (on or off, interval, `enabled_at`, `last_purge_at`) lives in the purge settings row of the database, is changed from the Settings page or `PATCH /v1/media/purge-settings`, and is read again at every hourly tick.
 
+## Image storage
+
+By default uploaded images are files in `UPLOAD_DIR`. To keep them in an S3-compatible bucket (AWS S3, MinIO, Ceph, Cloudflare R2, Backblaze B2 and similar), set `STORAGE_DRIVER=s3` and the variables below. They are only read when `STORAGE_DRIVER=s3`. Image URLs do not change: the app still serves `/static/uploadedImages/<name>`, reading the object from the bucket (the bucket stays private, and `Range` and `If-Modified-Since` requests keep working; there is no `ETag`).
+
+| Name                   | Default     | Description                                                                                                                                                                      |
+|------------------------|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `S3_BUCKET`            | required    | Name of an existing bucket. A missing value stops the server with `Environment variable S3_BUCKET is not set`.                                                                  |
+| `S3_REGION`            | `us-east-1` | Bucket region. Most S3-compatible servers accept the default.                                                                                                                    |
+| `S3_ENDPOINT`          | *(empty)*   | Host (with optional port) or full URL of an S3-compatible server. Leave empty for AWS S3.                                                                                       |
+| `S3_ACCESS_KEY_ID`     | *(empty)*   | Access key. Set it together with `S3_SECRET_ACCESS_KEY`, or set neither. Setting only one stops the server with an error naming the missing one.                                |
+| `S3_SECRET_ACCESS_KEY` | *(empty)*   | Secret key. Never logged and never in an error message. When both keys are empty, the default AWS credential chain is used (`AWS_*` variables, shared credentials file, instance or task role). |
+| `S3_KEY_PREFIX`        | *(empty)*   | Prefix put before every object key, for example `uploads/`. Include the trailing slash.                                                                                          |
+| `S3_PATH_STYLE`        | `false`     | `true` addresses the bucket as `endpoint/bucket` instead of `bucket.endpoint`. SeaweedFS, MinIO and most self-hosted servers need `true`.                                                  |
+| `S3_TLS`               | `true`      | Only used when `S3_ENDPOINT` has no `http://` or `https://` prefix: `true` means `https`, `false` means `http`.                                                                 |
+
+At startup the server checks the bucket (`HeadBucket`). A missing bucket, refused credentials or an unreachable server stop startup with `Unable to open the image storage: S3 bucket "<name>" is not usable: <reason>`; the error never contains the secret. Each S3 call has a 30 second timeout and the SDK default retries. At boot, images whose metadata is missing are read back from the bucket to fill it; when there were some, one log line says how many.
+
+### Moving existing uploads to a bucket
+
+There is no automatic migration and the database does not change: image rows only hold the stored name, so the same names must exist in the bucket. With the site stopped, copy the files under the same names, under `S3_KEY_PREFIX` if you set one, then set `STORAGE_DRIVER=s3` and restart:
+
+```sh
+aws s3 sync ./data/uploads s3://my-bucket/uploads/
+rclone sync ./data/uploads remote:my-bucket/uploads
+```
+
+Keep `UPLOAD_DIR` until you have checked the site. Moving back is the same copy in the other direction.
+
+### Backups
+
+The pre-update backup covers the database only (and `DATA_DIR` state). Neither the update backup nor the data directory includes uploads, so back the bucket up with your provider (versioning, replication) or with `aws s3 sync` / `mc mirror`.
+
 ## Database
 
 SQLite is the default and needs nothing but `DB_FILE`. To run against MySQL 8 or MariaDB 11 instead, set `DB_DRIVER=mysql` and the connection variables below. They are only read (and the four required ones only checked) when `DB_DRIVER=mysql`. A required variable that is not set stops the server with `Environment variable DB_HOST is not set` (or `DB_USER`, `DB_PASSWORD`, `DB_NAME`).

docs/deployment.md

@@ -8,7 +8,7 @@ RenewCMS ships as **one self-contained executable**. It serves:
 | `/healthz` | Liveness check, returns `200 ok`        |
 | `/*`       | The Vue SPA (embedded), with fallback to `index.html` for client-side routes |
 
-Runtime state includes the SQLite database (`DB_FILE`, or a MySQL/MariaDB server, see [Using MySQL or MariaDB](#using-mysql-or-mariadb)), uploads (`UPLOAD_DIR`), and update records, staged binaries and backups beside the database. Running a release binary needs no Go, Bun or Node installation. Browser updates additionally require Linux with systemd and the helper setup below.
+Runtime state includes the SQLite database (`DB_FILE`, or a MySQL/MariaDB server, see [Using MySQL or MariaDB](#using-mysql-or-mariadb)), uploads (`UPLOAD_DIR`, or an S3-compatible bucket with `STORAGE_DRIVER=s3`, see [configuration.md](configuration.md#image-storage)), and update records, staged binaries and backups beside the database. Running a release binary needs no Go, Bun or Node installation. Browser updates additionally require Linux with systemd and the helper setup below.
 
 To run it in a container instead, see [docker.md](docker.md). The published image `git.floriansylvain.fr/fsylvain/renewcms` is upgraded with `docker compose pull && docker compose up -d`. Tags are `vX.Y.Z`, `X.Y.Z`, `X.Y` and `latest` for a stable release, and `vX.Y.Z-<pre>`, `X.Y.Z-<pre>` and the moving `<pre-name>` (`beta`, `rc`) for a pre-release, which never moves `latest`. Maintainers publish with `task docker:publish` after `docker login git.floriansylvain.fr`; there is no CI. See [Maintainer: publishing](docker.md#maintainer-publishing).
 

docs/docker.md

@@ -102,6 +102,19 @@ To use MariaDB, change the `mysql` service image to `mariadb:11`, its `MYSQL_*`
 
 Back the database up with `docker compose -f docker-compose.mysql.yml exec mysql mysqldump ...` (a `tar` copy of the volume is not consistent while it runs), and the `renewcms-data` volume as described above. Moving an existing SQLite volume is a manual procedure, see [deployment.md](deployment.md#moving-an-existing-sqlite-site-to-mysql).
 
+## S3-compatible storage (SeaweedFS)
+
+[`docker-compose.s3.yml`](../docker-compose.s3.yml) runs RenewCMS with `STORAGE_DRIVER=s3` next to a [SeaweedFS](https://github.com/seaweedfs/seaweedfs) server (Apache 2.0, image `chrislusf/seaweedfs`, override with `SEAWEEDFS_IMAGE`) with its S3 gateway enabled, and a one-shot `seaweedfs-init` step that creates the bucket with `weed shell`:
+
+```sh
+export S3_SECRET_ACCESS_KEY=<generated>
+docker compose -f docker-compose.s3.yml up -d
+```
+
+RenewCMS gets `S3_ENDPOINT=http://seaweedfs:8333`, `S3_PATH_STYLE=true`, the bucket name and the access key (`S3_ACCESS_KEY_ID`, default `renewcms`, and `S3_SECRET_ACCESS_KEY`) from the compose file, and the rest from `docker.env`. SeaweedFS reads the same key pair as its admin identity. The `renewcms-data` volume then holds the SQLite database and update state but no uploads; the images live in the `seaweedfs-data` volume. The bucket is not published and the app serves the images. SeaweedFS is used because MinIO stopped publishing community images. To use AWS S3 or another provider, drop the `seaweedfs` services and set the `S3_*` variables in `docker.env` (see [configuration.md](configuration.md#image-storage)). The volume backup above covers the database only, so back the bucket up separately. To move existing uploads see [configuration.md](configuration.md#moving-existing-uploads-to-a-bucket).
+
+`task test:s3` starts a throwaway SeaweedFS and runs the storage integration test against it; it skips with a message when Docker is not available.
+
 ## Reverse proxy and TLS
 
 With `ENVIRONMENT=production` the site must be served over HTTPS, so put a reverse proxy in front and set `PUBLIC_URL` to the public `https://` address. Bind the published port to the host only when the proxy runs on the same host, by changing `ports` to `"127.0.0.1:8080:8080"`.