feat: optional S3-compatible storage for uploaded images, instead of the local upload directory
This commit is contained in:
fsylvain committed
1 parent
d752a61e52
commit
eb9bfabed5
23 files changed
+1962
-7
No files matched your search
@@ -0,0 +1,143 @@
|
||||
Feature: Uploaded images can be stored in an S3-compatible bucket
|
||||
|
||||
As an operator whose host disk is ephemeral or who runs several hosts
|
||||
I want to set STORAGE_DRIVER=s3 and bucket settings
|
||||
So that uploads live in the bucket while images are still served by the app
|
||||
at /static/uploadedImages/<name>, with unchanged URLs
|
||||
|
||||
Background:
|
||||
Given STORAGE_DRIVER defaults to "local" and local disk behaves exactly as today
|
||||
And an S3 storage implements image.Storage (Put, Open, Delete) in internal/infrastructure/storage
|
||||
And storage.Open has a case for the driver "s3" and openImageStorage selects it from the settings
|
||||
And no domain, use case, repository or staticimages handler file changes
|
||||
And the bucket is faked by an in-process httptest S3 server listening on 127.0.0.1
|
||||
And Open returns a Blob whose Content is an io.ReadSeeker, so http.ServeContent handles Range and conditional requests
|
||||
And the Blob ModTime and Size come from the S3 object metadata
|
||||
And no ETag is emitted and conditional requests are validated by Last-Modified only
|
||||
|
||||
Scenario Outline: Startup configuration validation
|
||||
Given STORAGE_DRIVER is <driver> and the S3 variables are <vars>
|
||||
When the storage settings are validated
|
||||
Then the result is <result>
|
||||
|
||||
Examples:
|
||||
| driver | vars | result |
|
||||
| (unset) | none | driver local, no S3 variable required |
|
||||
| local | S3_BUCKET set | driver local, S3 variables ignored |
|
||||
| s3 | S3_BUCKET and S3_REGION set | valid, default AWS credential chain |
|
||||
| s3 | S3_BUCKET set, S3_REGION unset | region defaults to us-east-1 |
|
||||
| s3 | S3_REGION set, S3_BUCKET unset | error "Environment variable S3_BUCKET is not set" |
|
||||
| s3 | key id set, secret unset | error naming S3_SECRET_ACCESS_KEY |
|
||||
| s3 | secret set, key id unset | error naming S3_ACCESS_KEY_ID |
|
||||
| s3 | UPLOAD_DIR set | valid, UPLOAD_DIR ignored without error |
|
||||
| gcs | none | error unsupported storage driver "gcs" |
|
||||
|
||||
Scenario: The registry opens the S3 storage after a bucket check
|
||||
Given the fake S3 answers HEAD bucket with 200
|
||||
When storage.Open is called with driver "s3"
|
||||
Then it returns a storage and no error
|
||||
|
||||
Scenario Outline: Startup fails fast with a clear secret-free error
|
||||
Given the fake S3 answers HEAD bucket with <status>
|
||||
When storage.Open is called with driver "s3" and secret access key "TOPSECRET"
|
||||
Then it returns an error mentioning the bucket name
|
||||
And the error text does not contain "TOPSECRET"
|
||||
And openImageStorage panics with that same text
|
||||
|
||||
Examples:
|
||||
| status |
|
||||
| 404 |
|
||||
| 403 |
|
||||
| 500 |
|
||||
|
||||
Scenario: Put stores the object under the prefix with its content type
|
||||
Given S3_KEY_PREFIX is "uploads/"
|
||||
When Put is called with stored name "a1b2.png", content type "image/png" and 3 bytes
|
||||
Then the fake S3 holds key "uploads/a1b2.png" with those bytes and Content-Type "image/png"
|
||||
|
||||
Scenario: Put failure is returned as an error
|
||||
Given the fake S3 answers PUT with 500
|
||||
When Put is called
|
||||
Then Put returns a non-nil error that is not image.ErrBlobNotFound
|
||||
|
||||
Scenario: Open returns content, size and modification time
|
||||
Given the fake S3 holds "uploads/a1b2.png" with 10 bytes and a known Last-Modified
|
||||
When Open("a1b2.png") is called
|
||||
Then the Blob has Size 10 and that ModTime
|
||||
And its Content implements io.ReadSeeker and reads the 10 bytes
|
||||
|
||||
Scenario: Open of a missing key is not found
|
||||
Given the fake S3 answers GET with 404 NoSuchKey
|
||||
When Open("missing.png") is called
|
||||
Then the result is image.ErrBlobNotFound, asserted with errors.Is
|
||||
|
||||
Scenario: Open with a backend failure is not a not-found
|
||||
Given the fake S3 answers GET with 500
|
||||
When Open is called
|
||||
Then the error is non-nil and is not image.ErrBlobNotFound
|
||||
|
||||
Scenario Outline: Delete is idempotent
|
||||
Given the fake S3 <state>
|
||||
When Delete("a1b2.png") is called
|
||||
Then it returns no error
|
||||
|
||||
Examples:
|
||||
| state |
|
||||
| holds the key |
|
||||
| does not hold the key (404) |
|
||||
|
||||
Scenario: A backend failure on Delete is returned
|
||||
Given the fake S3 answers DELETE with 500
|
||||
When Delete is called
|
||||
Then it returns a non-nil error
|
||||
|
||||
Scenario Outline: Serving /static/uploadedImages/<name> from the bucket
|
||||
Given the bucket holds "a1b2.png" with 10 bytes and a known Last-Modified
|
||||
When <method> /static/uploadedImages/a1b2.png is requested <with>
|
||||
Then the status is <status>
|
||||
And the response has <result>
|
||||
|
||||
Examples:
|
||||
| method | with | status | result |
|
||||
| GET | no extra header | 200 | Content-Type image/png, Content-Length 10, Last-Modified, Accept-Ranges bytes, Cache-Control "public, no-cache", X-Content-Type-Options nosniff |
|
||||
| HEAD | no extra header | 200 | the same headers and an empty body |
|
||||
| GET | If-Modified-Since = Last-Modified | 304 | an empty body |
|
||||
| GET | Range bytes=0-3 | 206 | Content-Range "bytes 0-3/10" and Content-Length 4 |
|
||||
| GET | Range bytes=50-60 | 416 | Content-Range "bytes */10" |
|
||||
|
||||
Scenario: SVG from the bucket keeps its security headers
|
||||
Given the bucket holds "logo.svg"
|
||||
When GET /static/uploadedImages/logo.svg is requested
|
||||
Then Content-Type is image/svg+xml
|
||||
And Content-Security-Policy is "default-src 'none'; style-src 'unsafe-inline'; sandbox"
|
||||
|
||||
Scenario: Missing object is 404 and a backend failure is 500
|
||||
Given the bucket does not hold "missing.png"
|
||||
When GET /static/uploadedImages/missing.png is requested
|
||||
Then the status is 404
|
||||
Given the fake S3 answers GET with 500
|
||||
When GET /static/uploadedImages/a1b2.png is requested
|
||||
Then the status is 500 and the plain text body is "Unable to read the file"
|
||||
|
||||
Scenario: Path traversal never reaches another key
|
||||
When GET /static/uploadedImages/../secret.png is requested
|
||||
Then the key requested from the fake S3 is the prefix plus "secret.png"
|
||||
|
||||
Scenario: Handler as-is for a non-seekable blob
|
||||
Given a storage fake returning a Blob whose Content is not an io.ReadSeeker, with Size 10 and a ModTime
|
||||
When GET and HEAD /static/uploadedImages/a1b2.png are requested
|
||||
Then GET returns 200 with Content-Length 10, Last-Modified and the body
|
||||
And HEAD returns 200 with the same headers and an empty body
|
||||
And no Range or ETag behaviour is specified for this path
|
||||
|
||||
Scenario: Boot backfill logs how many images it read from the bucket
|
||||
Given the driver is s3 and 3 stored images lack metadata
|
||||
When InitContainer runs the boot backfill
|
||||
Then a log line reports 3 images read from storage and contains no credential
|
||||
And backfill behaviour and errors are unchanged
|
||||
And no cap is applied and BackfillImageMetadataUseCase is not modified
|
||||
|
||||
Scenario: The local driver is unchanged
|
||||
Given STORAGE_DRIVER is unset and UPLOAD_DIR is a temp directory
|
||||
When an image is uploaded and requested
|
||||
Then it is stored in UPLOAD_DIR and served exactly as before
|
||||
Reference in new issue
Block a user