feat: PostgreSQL driver support

This commit is contained in:
fsylvain committed 2026-10-06 13:35:10 +02:00
1 parent eb9bfabed5
commit dc5d12348c
24 files changed
+1585 -75

No files matched your search

+153
View File
@@ -0,0 +1,153 @@
Feature: Optional PostgreSQL database driver
An operator who prefers PostgreSQL runs RenewCMS against it by setting
environment variables. SQLite stays the default, and MySQL/MariaDB behaves
exactly as before.
Background:
Given the database drivers are registered by name in database.drivers
And the database is chosen by the DB_DRIVER environment variable
And repositories, use cases and handlers are unchanged
Scenario Outline: Driver selection
Given DB_DRIVER is "<value>"
When the server starts
Then the outcome is "<outcome>"
Examples:
| value | outcome |
| unset | the sqlite driver opens DB_FILE |
| sqlite | the sqlite driver opens DB_FILE |
| mysql | the mysql driver opens the DB_HOST connection |
| postgres | the postgres driver opens the DB_HOST connection |
| postgresql | the postgres driver opens the DB_HOST connection |
| oracle | startup fails with: unsupported database driver "oracle" |
Scenario Outline: Missing PostgreSQL variable
Given DB_DRIVER is "postgres"
And "<missing>" is not set
When the environment is validated
Then startup fails with "Environment variable <missing> is not set"
Examples:
| missing |
| DB_HOST |
| DB_USER |
| DB_PASSWORD |
| DB_NAME |
Scenario: SQLite and MySQL do not require PostgreSQL-only variables
Given DB_DRIVER is "sqlite" or "mysql"
And DB_SSLMODE is "maybe"
When the settings are read
Then DB_SSLMODE is ignored and validation succeeds
Scenario Outline: Connection string built from variables
Given DB_HOST "<host>", DB_PORT "<port>", DB_USER "app", DB_NAME "cms", DB_SSLMODE "<sslmode>"
When the PostgreSQL DSN is built
Then the DSN targets "<host>:<expectedPort>" and database "cms"
And it sets sslmode "<expectedSslmode>"
And it sets the session time zone to UTC
And a password containing spaces, quotes, "@" or "/" is escaped so that parsing the DSN returns it unchanged
Examples:
| host | port | expectedPort | sslmode | expectedSslmode |
| db.local | | 5432 | | disable |
| db.local | 5433 | 5433 | require | require |
| pg | 5432 | 5432 | verify-full | verify-full |
Scenario: Invalid SSL mode is rejected
Given DB_DRIVER is "postgres" and DB_SSLMODE is "maybe"
When the settings are read
Then startup fails naming DB_SSLMODE and the accepted values
Scenario Outline: Fail fast when the server cannot be used
Given DB_DRIVER is "postgres" and "<fault>"
When the server starts
Then startup stops with "Unable to open the database"
And the message names the host and port
And the message does not contain the DB_PASSWORD value
Examples:
| fault |
| nothing listens on DB_PORT |
| the password is wrong |
| the database does not exist |
| the server requires TLS and DB_SSLMODE is disable |
Scenario: Data directory with PostgreSQL
Given DB_DRIVER is "postgres" and DATA_DIR is "/data"
Then update state, staging and backups live in "/data"
And DB_FILE is ignored
Scenario: Connection pool
Given DB_MAX_OPEN_CONNS 4, DB_MAX_IDLE_CONNS 2 and DB_CONN_MAX_LIFETIME_SECONDS 60
When the PostgreSQL connection is opened
Then the pool limits are 4 open, 2 idle and 60 seconds
Scenario: Migration on PostgreSQL
Given an empty PostgreSQL database
When database.Migrate runs
Then the tables articles, images, users, invites, api_keys and purge_settings exist in schema "public"
And the unique indexes on users and api_keys exist
And the first user lacking a role becomes "owner" and the others "admin"
And running database.Migrate a second time changes nothing
Scenario: Existing drivers are untouched by the PostgreSQL addition
Given a SQLite file holding rows, and a MySQL database holding rows
When database.Migrate runs on each
Then the schema and every row are unchanged on both
Scenario Outline: Repository behaviour is the same on every driver
Given the "<driver>" driver
Then the repository contract "<case>" holds
Examples:
| driver | case |
| postgres | create user and read it back with its id and timestamps |
| postgres | username "Bob" and "bob" are distinct users |
| postgres | email "A@x.io" and "a@x.io" follow the same rule as on sqlite |
| postgres | duplicate email returns an error and stores nothing |
| postgres | the failed duplicate insert leaves the connection usable |
| postgres | unknown id returns domain ErrUserNotFound |
| postgres | accept invite twice: the second returns ErrInviteNotFound |
| postgres | accept invite with a taken username leaves the invite unaccepted |
| postgres | renew invite with unchanged values still succeeds |
| postgres | duplicate image hashes are grouped and counted |
| postgres | media search treats % and _ literally |
| postgres | media search is case-sensitive exactly as on sqlite |
| postgres | article body of 1 MB round-trips |
| postgres | article title with emoji and accents round-trips |
| postgres | zero reset expiration round-trips as a zero time |
| postgres | stored times read back in UTC |
| postgres | purge settings save, then SaveLastRun with identical values succeeds |
| postgres | api key unique hashed_key, revoke and last_used |
| postgres | offline article reads as not found publicly, online article is listed |
| postgres | listing order and pagination match sqlite for the same data |
Scenario Outline: Backup before an update
Given the "<driver>" driver with a populated database
When the pre-update backup runs
Then a file "<pattern>" exists in DATA_DIR/backups
And only the 3 newest backups are kept
And a failure returns an error, so the update download fails instead of claiming a backup
Examples:
| driver | pattern |
| sqlite | renewcms-<time>.db |
| mysql | renewcms-<time>.sql |
| postgres | renewcms-<time>.sql |
Scenario: PostgreSQL backup content
Given a PostgreSQL database with 2 users and 1 article
When the pre-update backup runs
Then the dump contains the schema, indexes and the rows of every table
And it does not contain the DB_PASSWORD value
And replaying it into an empty database restores the same rows
And sequences are reset so the next insert gets an unused id
Scenario: Secrets are never exposed
Given DB_DRIVER is "postgres"
Then no log line, error message or file in the repository contains the DB_PASSWORD value
Scenario: Static builds
Then the module builds with CGO_ENABLED=0 for the 6 task release targets
+43
View File
@@ -0,0 +1,43 @@
# QA: PostgreSQL database driver
Backend only. Go tests in test/, black-box, no live server by default.
Decisions: DB_DRIVER=postgres (alias postgresql), gorm.io/driver/postgres (pgx, pure Go), shared DB_* vars plus DB_SSLMODE, pure-Go logical dump backup, `task test:postgres` against postgres:16 and postgres:17, SQLite or MySQL to PostgreSQL migration is a documented manual export/import only.
## Automated, no server (task test)
1. Registry: contains sqlite, mysql and postgres. Alias "postgresql" resolves to postgres. "oracle" returns `unsupported database driver "oracle"`.
2. Env validation: table-driven over Driver selection and Missing PostgreSQL variable. Invalid DB_SSLMODE or a non-numeric pool var returns an error naming the variable. DB_SSLMODE is ignored under sqlite and mysql. MySQL validation unchanged.
3. DSN builder: table-driven over the DSN scenario. Parse the output with the pgx config parser and assert host, port, database, sslmode and the UTC time zone. Round-trip a password such as `s3cr3t/Value@42:x 'q'`. The form used in errors never contains the password.
4. Settings.DataDir under postgres: DATA_DIR wins, DB_FILE ignored.
5. Fail fast without a server: closed 127.0.0.1 port, 1s connect timeout. Error has the "Unable to open the database" prefix, mentions host and port, not the password.
6. Pool settings via a stub opener or the real test below.
7. Regression: the existing SQLite and MySQL tests pass unchanged. A pre-change SQLite fixture is unchanged after Migrate.
8. Dump writer: reuse the existing formatter tests. Add PostgreSQL value formatting with a fake row source: bool as true/false, bytea hex escapes, timestamptz in UTC, NULL, identifier and string quoting.
9. Migrate dialect handling: no MySQL table options are applied under postgres (tableOptionsFor).
## Automated, real server (task test:postgres)
Setup: the task runs `docker run` for postgres:16 then postgres:17, waits with pg_isready, exports RENEWCMS_TEST_POSTGRES_DSN, runs `go test -tags nofrontend ./test/... -run Postgres -count=1`, then removes the container. Tests t.Skip when the variable is unset. Repository contract tests are parameterised over sqlite, mysql and postgres with a fresh database each.
10. Migrate on an empty database: tables and indexes via information_schema and pg_indexes, role backfill, idempotent second run.
11. User create/read, case-sensitive username and email, duplicate email, ErrUserNotFound. After the unique violation the next query on the same pool succeeds.
12. Invite accept twice, accept with a taken username (invite stays unaccepted), renew with unchanged values.
13. Image duplicates grouping and counts (GROUP BY strictness), search with % and _ and its case rule, filename backfill listing.
14. Article body 1 MB, unicode title, title lookup, online filter, listing order and pagination parity with sqlite.
15. API key unique hashed_key, revoke, last_used.
16. Zero-time round trip, UTC times.
17. Purge settings upsert.
18. Backup on a populated database: .sql file, 3-file retention, replay into a second empty database gives identical rows, no password in the file, next insert after replay gets a fresh id.
19. Wrong password, missing database and TLS-required-but-disabled fail with the "Unable to open the database" prefix and no password.
## Build and lint
20. `task lint:go` and `task test` pass. `task test:postgres` passes on 16 and 17 (record both; NOT RUN if Docker is unavailable).
21. `CGO_ENABLED=0 go build -tags nofrontend ./...` passes. Cross-compile the 6 `task release` targets.
22. `task mutate` impact: none (useCases only).
23. No comment lines in the added Go files.
24. Layering: `go list -deps` on useCases and domain shows no pgx or gorm. Only api/dependecyInjection.go builds the postgres config.
## Manual (needs Compose and real SMTP)
25. docker-compose.postgres.yml: renewcms plus a postgres service with a named volume and a pg_isready healthcheck, depends_on service_healthy, DB_DRIVER=postgres, DB_HOST=postgres, DATA_DIR=/data. `docker compose up`: /healthz 200, register, log in, create an article, restart the stack, data persists.
26. Stop the postgres container and start renewcms: clear error, no password in the logs.
27. In the Docker image (uid 10001), trigger the backup path and confirm the .sql file lands under /data/backups.
28. Docs: .env.example, docker.env.example, docs/configuration.md, docs/docker.md, docs/deployment.md, docs/development.md and the docs/architecture.md "Storage and database seams" section list DB_DRIVER=postgres, DB_SSLMODE and the manual migration note. DB_SSLMODE goes into cmd/server/main.go envDefaults.
29. SQLite default end to end with no new vars set behaves as before.