feat: optional S3-compatible storage for uploaded images, instead of the local upload directory

2026-10-06 12:13:24 +02:00
1 parent d752a61e52
commit eb9bfabed5
@@ -27,7 +27,8 @@ Copy [`.env.example`](../.env.example) to `.env` to get started. When a required
 | `DB_DRIVER`            | `sqlite`             | `sqlite` or `mysql` (MySQL and MariaDB). Any other value stops the server with `unsupported database driver "<value>"`. See [Database](#database).                                                                             |
 | `DB_FILE`              | `./data/renewcms.db` | Path to the SQLite database file. Missing parent directories are created. The schema is migrated automatically at startup. Ignored when `DB_DRIVER=mysql`.                                                                     |
 | `DATA_DIR`             | directory of `DB_FILE` | Directory for update state, staged updates and database backups (`backups/`). Set it with `DB_DRIVER=mysql`, where there is no database file to sit beside (the default still follows `DB_FILE`). With SQLite, leave it unset to keep everything beside the database. |
-| `UPLOAD_DIR`           | `./data/uploads`     | Directory where uploaded images are stored.                                                                                                                                                                                     |
+| `STORAGE_DRIVER`       | `local`              | `local` (files in `UPLOAD_DIR`) or `s3` (an S3-compatible bucket). Any other value stops the server with `unsupported storage driver "<value>"`. See [Image storage](#image-storage). |
+| `UPLOAD_DIR`           | `./data/uploads`     | Directory where uploaded images are stored. Ignored, without error, when `STORAGE_DRIVER=s3`.                                                                                                                                                                                     |
 | `CORS_ALLOWED_ORIGINS` | *(empty)*            | Semicolon-separated list of origins allowed to call the API. When empty, no CORS headers are sent. That is the right setting whenever the SPA is served by RenewCMS itself (production) or through the Vite proxy (development). |
 | `AUTH_RATE_LIMIT_MAX_ATTEMPTS` | `5`           | Maximum number of requests a single client IP may send to `POST /v1/auth/login`, `POST /v1/auth/password-reset` or the invite-accept pair (`GET /v1/auth/invite` and `POST /v1/auth/invite/accept`, sharing one counter) within the window, tracked independently per route.                                                          |
 | `AUTH_RATE_LIMIT_WINDOW_SECONDS` | `60`        | Length, in seconds, of the fixed rate-limit window for the routes above.                                                                                                                                                    |
@@ -37,6 +38,38 @@ Copy [`.env.example`](../.env.example) to `.env` to get started. When a required

 Automatic media purge has no environment variable. Its schedule (on or off, interval, `enabled_at`, `last_purge_at`) lives in the purge settings row of the database, is changed from the Settings page or `PATCH /v1/media/purge-settings`, and is read again at every hourly tick.

+## Image storage
+
+By default uploaded images are files in `UPLOAD_DIR`. To keep them in an S3-compatible bucket (AWS S3, MinIO, Ceph, Cloudflare R2, Backblaze B2 and similar), set `STORAGE_DRIVER=s3` and the variables below. They are only read when `STORAGE_DRIVER=s3`. Image URLs do not change: the app still serves `/static/uploadedImages/<name>`, reading the object from the bucket (the bucket stays private, and `Range` and `If-Modified-Since` requests keep working; there is no `ETag`).
+
+| Name                   | Default     | Description                                                                                                                                                                      |
+|------------------------|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `S3_BUCKET`            | required    | Name of an existing bucket. A missing value stops the server with `Environment variable S3_BUCKET is not set`.                                                                  |
+| `S3_REGION`            | `us-east-1` | Bucket region. Most S3-compatible servers accept the default.                                                                                                                    |
+| `S3_ENDPOINT`          | *(empty)*   | Host (with optional port) or full URL of an S3-compatible server. Leave empty for AWS S3.                                                                                       |
+| `S3_ACCESS_KEY_ID`     | *(empty)*   | Access key. Set it together with `S3_SECRET_ACCESS_KEY`, or set neither. Setting only one stops the server with an error naming the missing one.                                |
+| `S3_SECRET_ACCESS_KEY` | *(empty)*   | Secret key. Never logged and never in an error message. When both keys are empty, the default AWS credential chain is used (`AWS_*` variables, shared credentials file, instance or task role). |
+| `S3_KEY_PREFIX`        | *(empty)*   | Prefix put before every object key, for example `uploads/`. Include the trailing slash.                                                                                          |
+| `S3_PATH_STYLE`        | `false`     | `true` addresses the bucket as `endpoint/bucket` instead of `bucket.endpoint`. SeaweedFS, MinIO and most self-hosted servers need `true`.                                                  |
+| `S3_TLS`               | `true`      | Only used when `S3_ENDPOINT` has no `http://` or `https://` prefix: `true` means `https`, `false` means `http`.                                                                 |
+
+At startup the server checks the bucket (`HeadBucket`). A missing bucket, refused credentials or an unreachable server stop startup with `Unable to open the image storage: S3 bucket "<name>" is not usable: <reason>`; the error never contains the secret. Each S3 call has a 30 second timeout and the SDK default retries. At boot, images whose metadata is missing are read back from the bucket to fill it; when there were some, one log line says how many.
+
+### Moving existing uploads to a bucket
+
+There is no automatic migration and the database does not change: image rows only hold the stored name, so the same names must exist in the bucket. With the site stopped, copy the files under the same names, under `S3_KEY_PREFIX` if you set one, then set `STORAGE_DRIVER=s3` and restart:
+
+```sh
+aws s3 sync ./data/uploads s3://my-bucket/uploads/
+rclone sync ./data/uploads remote:my-bucket/uploads
+```
+
+Keep `UPLOAD_DIR` until you have checked the site. Moving back is the same copy in the other direction.
+
+### Backups
+
+The pre-update backup covers the database only (and `DATA_DIR` state). Neither the update backup nor the data directory includes uploads, so back the bucket up with your provider (versioning, replication) or with `aws s3 sync` / `mc mirror`.
+
 ## Database

 SQLite is the default and needs nothing but `DB_FILE`. To run against MySQL 8 or MariaDB 11 instead, set `DB_DRIVER=mysql` and the connection variables below. They are only read (and the four required ones only checked) when `DB_DRIVER=mysql`. A required variable that is not set stops the server with `Environment variable DB_HOST is not set` (or `DB_USER`, `DB_PASSWORD`, `DB_NAME`).