Files
RenewCMS/docs/image-variants.md

128 lines
8.4 KiB
Markdown

# Image variants
An owner or admin defines named variants of the stored images. A request for
`/static/uploadedImages/<storedName>?v=<name>` generates the variant on first use, caches it and
serves the cached file afterwards. The original is never changed. Only a variant that exists and is
enabled is served. The visual spec is `docs/design/screens/image-variants.md`; the management routes
are in [api.md](api.md#image-variants); the layering is in [architecture.md](architecture.md#image-variants).
## Public contract
| Request | Result |
| --- | --- |
| `…/a1b2.png` | The stored file, exactly as before this feature. |
| `…/a1b2.png?v=thumb` | 200, the cached variant. Other query parameters (`t=123`) are ignored. |
| `?v=` with an unknown, disabled, deleted or badly written name | 400 `{"status":400,"error":"unknown_variant","message":"Unknown variant","detail":"<name>"}`. |
| `width`, `height`, `format` or `quality` | 400 `unsupported_parameter` with the parameter as `detail`. It wins over `unknown_variant`. |
| A missing source (also `..` and nested paths) | 404, after the parameter checks. |
| A blob that has no image record (a generated variant file, an orphan) | 404 for `?v=`, after the parameter checks and before any storage read. Without `?v=` the blob is served as before. |
| A source bigger than `IMAGE_VARIANT_MAX_SOURCE_BYTES` | 413 `The source image is too large to generate this variant.` (plain text). The original is never capped. |
| A source whose header declares more than `IMAGE_VARIANT_MAX_SOURCE_PIXELS` (width x height) | The same 413 and body. The header is read before any pixel buffer is allocated, so a small file declaring 30000 x 30000 is refused without being decoded. |
| A source that cannot be decoded, a header that cannot be read, or a failing render | 500, nothing is cached, the next request tries again. |
The 400 body is JSON with `Content-Type: application/json`. Every other error is plain text.
SVG sources and animated GIFs are served as they are, with the variant parameters validated but no
generation, even above the size cap. A one-frame GIF is a normal raster source.
The response carries an `ETag` derived from the variant settings (width, height, fit, format, quality)
and the source stored name. It is an opaque quoted value. `If-None-Match` answers 304 before the cache
is read. Changing `generate_on_upload` or `enabled` does not change it; changing any rendering
setting does, so a client holding an old value gets a fresh 200. `Cache-Control` stays
`public, no-cache`.
## Cache
Every generated file is a blob in `image.Storage` (local directory or S3) plus a row in
`image_variant_files` (variant, source stored name, cached stored name, content type, size, settings
fingerprint). The pair (variant, source) is unique. Concurrent first requests share one generation
(`singleflight` keyed by variant, settings and source); different variants or different images are
generated in parallel.
Only a stored name that has an image record can be a variant source, so generated files and orphan blobs
are never rendered. At most `IMAGE_VARIANT_MAX_CONCURRENT_RENDERS` (default 4) renders run at once:
the slot is taken inside the shared generation, so identical concurrent requests use one slot, and a
request waiting for a slot stops waiting when its client goes away, without rendering. A cached
file never takes a slot. The pixel cap defaults to 50000000 pixels.
- Editing a rendering setting (`width`, `height`, `fit`, `format`, `quality`) purges the variant's
files; the next request regenerates. Switching `generate_on_upload` or `enabled` keeps them, so
re-enabling a variant serves from its kept cache.
- A row whose fingerprint differs from the variant (an edit raced a generation) or whose blob is gone
is regenerated and replaced, and the old blob is removed.
- Deleting a variant, purging it, deleting a source image or merging copies removes the blobs first
and the rows second. A blob that cannot be deleted keeps its row, the call answers 500 and a retry
finishes the job. A blob that is already missing does not stop a purge.
- `generate_on_upload` is stored and returned only. Generating at upload time is not part of this
release.
### Request tracking and the Cache screen
`/settings/media/cache` shows what the cache holds and purges it by scope (all, unused, one variant,
one image). It reads `GET /v1/image-variants/cache` and writes `POST /v1/image-variants/cache/purge`
([api.md](api.md)).
Two things are tracked for it, without a database write per request. `usage.Tracker`
(`internal/infrastructure/usage`) keeps in memory, behind the `image.VariantUsageTracker` port, the
last time each (variant, source) file was requested and a counter of rejected requests per UTC day.
`ServeVariantUseCase` records a hit when a cached file is served, when a file is generated by the
request and on a 304; it counts a rejection when the request answers 400 (unknown or disabled
variant, unsupported parameter). A missing source (404), a request without `?v=` and the admin
generation routes record nothing. The tracker flushes one batch through `image.VariantUsageRepository`
every 30 seconds, as soon as 500 distinct files are pending, before `GET /v1/image-variants/cache`
reads, and on graceful shutdown; only `api/dependecyInjection.go` builds it and starts the loop. A
failed flush keeps the pending hits and counts, so nothing is lost or counted twice. The stored
`last_requested_at` of a file never moves back, a late update for a purged file changes nothing, and
day rows older than 7 days are deleted by the flush.
A file is unused when its reference time (`last_requested_at`, or `created_at` when never requested)
is older than 30 days and its variant is enabled. A purge reads the rows in batches of 500 and keeps
the rows whose blob cannot be deleted. Each scope records its own activity action
(`media.variant_purge_all`, `media.variant_purge_unused`, `media.variant_purge`,
`media.variant_purge_image`).
## Geometry
Images are never enlarged.
- With one dimension, the image keeps its aspect ratio and the output is at most that size.
- `inside` fits the whole image in the box, without padding.
- `cover` fills the box and crops around the centre. A box larger than the source is first scaled
down, keeping its aspect ratio, until it fits inside the source (300 x 200 into 400 x 300 gives
267 x 200).
- `contain` with both width and height scales the image to fit inside the box, never enlarging it, and
centres it on a canvas of exactly width x height: a 300 x 200 image in a 400 x 300 box stays
300 x 200 on a 400 x 300 canvas, and a 40 x 20 image in a 40 x 40 box is padded to 40 x 40. With a
single dimension there is no box, so no padding and no upscaling.
### Padding colour of `contain`
The design does not draw it, so it is decided here: the padding is transparent for `png` and `webp`
outputs and white (`#FFFFFF`) for `jpeg` outputs, which have no alpha. A `jpeg` output also flattens
any transparency of the source onto white (`#FFFFFF`).
## Formats
`original` keeps the source format (a GIF source becomes PNG). `jpeg` and `webp` use `quality`
(default 82); `png` ignores it and the API returns `quality: null`. A WebP source always decodes. The
renderer reports the formats it can encode; the API refuses a format it cannot encode with
`Format must be original, jpeg or png.`.
## Spike: a CGO-free lossy WebP encoder
The standard library and `golang.org/x/image` can decode WebP but not encode it, and the build must
stay `CGO_ENABLED=0` (six release targets, pure-Go SQLite).
| Candidate | Lossy | `CGO_ENABLED=0` | Verdict |
| --- | --- | --- | --- |
| `golang.org/x/image/webp` | decode only | yes | Cannot encode. |
| `github.com/gen2brain/webp` v0.6.4 (libwebp translated to Go with wasm2go; `purego` only for the optional dynamic library) | yes, with alpha | yes | Adopted. |
The spike built `gen2brain/webp` for linux, darwin and windows on amd64 and arm64 with
`CGO_ENABLED=0`, and `go list -deps` shows no package with cgo files. On a noisy 4032 x 3024 image
a lossy encode took about 0.75 s at quality 20 and 1.5 s at quality 95 (a smaller, real photograph is
faster). Output is `RIFF....WEBP`, and a lower quality gives a smaller file. Decision: `webp` is offered
as an output format. Should the dependency ever be dropped, `NewVariantRendererEncoding` already
builds a renderer without the WebP encoder: the API then refuses `webp` with 422, and a WebP source
with format `original` is served as PNG.