feat: PostgreSQL driver support
This commit is contained in:
1 parent
eb9bfabed5
commit
dc5d12348c
24 files changed
+1585
-75
No files matched your search
@@ -0,0 +1,153 @@
|
||||
Feature: Optional PostgreSQL database driver
|
||||
An operator who prefers PostgreSQL runs RenewCMS against it by setting
|
||||
environment variables. SQLite stays the default, and MySQL/MariaDB behaves
|
||||
exactly as before.
|
||||
|
||||
Background:
|
||||
Given the database drivers are registered by name in database.drivers
|
||||
And the database is chosen by the DB_DRIVER environment variable
|
||||
And repositories, use cases and handlers are unchanged
|
||||
|
||||
Scenario Outline: Driver selection
|
||||
Given DB_DRIVER is "<value>"
|
||||
When the server starts
|
||||
Then the outcome is "<outcome>"
|
||||
|
||||
Examples:
|
||||
| value | outcome |
|
||||
| unset | the sqlite driver opens DB_FILE |
|
||||
| sqlite | the sqlite driver opens DB_FILE |
|
||||
| mysql | the mysql driver opens the DB_HOST connection |
|
||||
| postgres | the postgres driver opens the DB_HOST connection |
|
||||
| postgresql | the postgres driver opens the DB_HOST connection |
|
||||
| oracle | startup fails with: unsupported database driver "oracle" |
|
||||
|
||||
Scenario Outline: Missing PostgreSQL variable
|
||||
Given DB_DRIVER is "postgres"
|
||||
And "<missing>" is not set
|
||||
When the environment is validated
|
||||
Then startup fails with "Environment variable <missing> is not set"
|
||||
|
||||
Examples:
|
||||
| missing |
|
||||
| DB_HOST |
|
||||
| DB_USER |
|
||||
| DB_PASSWORD |
|
||||
| DB_NAME |
|
||||
|
||||
Scenario: SQLite and MySQL do not require PostgreSQL-only variables
|
||||
Given DB_DRIVER is "sqlite" or "mysql"
|
||||
And DB_SSLMODE is "maybe"
|
||||
When the settings are read
|
||||
Then DB_SSLMODE is ignored and validation succeeds
|
||||
|
||||
Scenario Outline: Connection string built from variables
|
||||
Given DB_HOST "<host>", DB_PORT "<port>", DB_USER "app", DB_NAME "cms", DB_SSLMODE "<sslmode>"
|
||||
When the PostgreSQL DSN is built
|
||||
Then the DSN targets "<host>:<expectedPort>" and database "cms"
|
||||
And it sets sslmode "<expectedSslmode>"
|
||||
And it sets the session time zone to UTC
|
||||
And a password containing spaces, quotes, "@" or "/" is escaped so that parsing the DSN returns it unchanged
|
||||
|
||||
Examples:
|
||||
| host | port | expectedPort | sslmode | expectedSslmode |
|
||||
| db.local | | 5432 | | disable |
|
||||
| db.local | 5433 | 5433 | require | require |
|
||||
| pg | 5432 | 5432 | verify-full | verify-full |
|
||||
|
||||
Scenario: Invalid SSL mode is rejected
|
||||
Given DB_DRIVER is "postgres" and DB_SSLMODE is "maybe"
|
||||
When the settings are read
|
||||
Then startup fails naming DB_SSLMODE and the accepted values
|
||||
|
||||
Scenario Outline: Fail fast when the server cannot be used
|
||||
Given DB_DRIVER is "postgres" and "<fault>"
|
||||
When the server starts
|
||||
Then startup stops with "Unable to open the database"
|
||||
And the message names the host and port
|
||||
And the message does not contain the DB_PASSWORD value
|
||||
|
||||
Examples:
|
||||
| fault |
|
||||
| nothing listens on DB_PORT |
|
||||
| the password is wrong |
|
||||
| the database does not exist |
|
||||
| the server requires TLS and DB_SSLMODE is disable |
|
||||
|
||||
Scenario: Data directory with PostgreSQL
|
||||
Given DB_DRIVER is "postgres" and DATA_DIR is "/data"
|
||||
Then update state, staging and backups live in "/data"
|
||||
And DB_FILE is ignored
|
||||
|
||||
Scenario: Connection pool
|
||||
Given DB_MAX_OPEN_CONNS 4, DB_MAX_IDLE_CONNS 2 and DB_CONN_MAX_LIFETIME_SECONDS 60
|
||||
When the PostgreSQL connection is opened
|
||||
Then the pool limits are 4 open, 2 idle and 60 seconds
|
||||
|
||||
Scenario: Migration on PostgreSQL
|
||||
Given an empty PostgreSQL database
|
||||
When database.Migrate runs
|
||||
Then the tables articles, images, users, invites, api_keys and purge_settings exist in schema "public"
|
||||
And the unique indexes on users and api_keys exist
|
||||
And the first user lacking a role becomes "owner" and the others "admin"
|
||||
And running database.Migrate a second time changes nothing
|
||||
|
||||
Scenario: Existing drivers are untouched by the PostgreSQL addition
|
||||
Given a SQLite file holding rows, and a MySQL database holding rows
|
||||
When database.Migrate runs on each
|
||||
Then the schema and every row are unchanged on both
|
||||
|
||||
Scenario Outline: Repository behaviour is the same on every driver
|
||||
Given the "<driver>" driver
|
||||
Then the repository contract "<case>" holds
|
||||
|
||||
Examples:
|
||||
| driver | case |
|
||||
| postgres | create user and read it back with its id and timestamps |
|
||||
| postgres | username "Bob" and "bob" are distinct users |
|
||||
| postgres | email "A@x.io" and "a@x.io" follow the same rule as on sqlite |
|
||||
| postgres | duplicate email returns an error and stores nothing |
|
||||
| postgres | the failed duplicate insert leaves the connection usable |
|
||||
| postgres | unknown id returns domain ErrUserNotFound |
|
||||
| postgres | accept invite twice: the second returns ErrInviteNotFound |
|
||||
| postgres | accept invite with a taken username leaves the invite unaccepted |
|
||||
| postgres | renew invite with unchanged values still succeeds |
|
||||
| postgres | duplicate image hashes are grouped and counted |
|
||||
| postgres | media search treats % and _ literally |
|
||||
| postgres | media search is case-sensitive exactly as on sqlite |
|
||||
| postgres | article body of 1 MB round-trips |
|
||||
| postgres | article title with emoji and accents round-trips |
|
||||
| postgres | zero reset expiration round-trips as a zero time |
|
||||
| postgres | stored times read back in UTC |
|
||||
| postgres | purge settings save, then SaveLastRun with identical values succeeds |
|
||||
| postgres | api key unique hashed_key, revoke and last_used |
|
||||
| postgres | offline article reads as not found publicly, online article is listed |
|
||||
| postgres | listing order and pagination match sqlite for the same data |
|
||||
|
||||
Scenario Outline: Backup before an update
|
||||
Given the "<driver>" driver with a populated database
|
||||
When the pre-update backup runs
|
||||
Then a file "<pattern>" exists in DATA_DIR/backups
|
||||
And only the 3 newest backups are kept
|
||||
And a failure returns an error, so the update download fails instead of claiming a backup
|
||||
|
||||
Examples:
|
||||
| driver | pattern |
|
||||
| sqlite | renewcms-<time>.db |
|
||||
| mysql | renewcms-<time>.sql |
|
||||
| postgres | renewcms-<time>.sql |
|
||||
|
||||
Scenario: PostgreSQL backup content
|
||||
Given a PostgreSQL database with 2 users and 1 article
|
||||
When the pre-update backup runs
|
||||
Then the dump contains the schema, indexes and the rows of every table
|
||||
And it does not contain the DB_PASSWORD value
|
||||
And replaying it into an empty database restores the same rows
|
||||
And sequences are reset so the next insert gets an unused id
|
||||
|
||||
Scenario: Secrets are never exposed
|
||||
Given DB_DRIVER is "postgres"
|
||||
Then no log line, error message or file in the repository contains the DB_PASSWORD value
|
||||
|
||||
Scenario: Static builds
|
||||
Then the module builds with CGO_ENABLED=0 for the 6 task release targets
|
||||
@@ -0,0 +1,43 @@
|
||||
# QA: PostgreSQL database driver
|
||||
|
||||
Backend only. Go tests in test/, black-box, no live server by default.
|
||||
|
||||
Decisions: DB_DRIVER=postgres (alias postgresql), gorm.io/driver/postgres (pgx, pure Go), shared DB_* vars plus DB_SSLMODE, pure-Go logical dump backup, `task test:postgres` against postgres:16 and postgres:17, SQLite or MySQL to PostgreSQL migration is a documented manual export/import only.
|
||||
|
||||
## Automated, no server (task test)
|
||||
1. Registry: contains sqlite, mysql and postgres. Alias "postgresql" resolves to postgres. "oracle" returns `unsupported database driver "oracle"`.
|
||||
2. Env validation: table-driven over Driver selection and Missing PostgreSQL variable. Invalid DB_SSLMODE or a non-numeric pool var returns an error naming the variable. DB_SSLMODE is ignored under sqlite and mysql. MySQL validation unchanged.
|
||||
3. DSN builder: table-driven over the DSN scenario. Parse the output with the pgx config parser and assert host, port, database, sslmode and the UTC time zone. Round-trip a password such as `s3cr3t/Value@42:x 'q'`. The form used in errors never contains the password.
|
||||
4. Settings.DataDir under postgres: DATA_DIR wins, DB_FILE ignored.
|
||||
5. Fail fast without a server: closed 127.0.0.1 port, 1s connect timeout. Error has the "Unable to open the database" prefix, mentions host and port, not the password.
|
||||
6. Pool settings via a stub opener or the real test below.
|
||||
7. Regression: the existing SQLite and MySQL tests pass unchanged. A pre-change SQLite fixture is unchanged after Migrate.
|
||||
8. Dump writer: reuse the existing formatter tests. Add PostgreSQL value formatting with a fake row source: bool as true/false, bytea hex escapes, timestamptz in UTC, NULL, identifier and string quoting.
|
||||
9. Migrate dialect handling: no MySQL table options are applied under postgres (tableOptionsFor).
|
||||
|
||||
## Automated, real server (task test:postgres)
|
||||
Setup: the task runs `docker run` for postgres:16 then postgres:17, waits with pg_isready, exports RENEWCMS_TEST_POSTGRES_DSN, runs `go test -tags nofrontend ./test/... -run Postgres -count=1`, then removes the container. Tests t.Skip when the variable is unset. Repository contract tests are parameterised over sqlite, mysql and postgres with a fresh database each.
|
||||
10. Migrate on an empty database: tables and indexes via information_schema and pg_indexes, role backfill, idempotent second run.
|
||||
11. User create/read, case-sensitive username and email, duplicate email, ErrUserNotFound. After the unique violation the next query on the same pool succeeds.
|
||||
12. Invite accept twice, accept with a taken username (invite stays unaccepted), renew with unchanged values.
|
||||
13. Image duplicates grouping and counts (GROUP BY strictness), search with % and _ and its case rule, filename backfill listing.
|
||||
14. Article body 1 MB, unicode title, title lookup, online filter, listing order and pagination parity with sqlite.
|
||||
15. API key unique hashed_key, revoke, last_used.
|
||||
16. Zero-time round trip, UTC times.
|
||||
17. Purge settings upsert.
|
||||
18. Backup on a populated database: .sql file, 3-file retention, replay into a second empty database gives identical rows, no password in the file, next insert after replay gets a fresh id.
|
||||
19. Wrong password, missing database and TLS-required-but-disabled fail with the "Unable to open the database" prefix and no password.
|
||||
|
||||
## Build and lint
|
||||
20. `task lint:go` and `task test` pass. `task test:postgres` passes on 16 and 17 (record both; NOT RUN if Docker is unavailable).
|
||||
21. `CGO_ENABLED=0 go build -tags nofrontend ./...` passes. Cross-compile the 6 `task release` targets.
|
||||
22. `task mutate` impact: none (useCases only).
|
||||
23. No comment lines in the added Go files.
|
||||
24. Layering: `go list -deps` on useCases and domain shows no pgx or gorm. Only api/dependecyInjection.go builds the postgres config.
|
||||
|
||||
## Manual (needs Compose and real SMTP)
|
||||
25. docker-compose.postgres.yml: renewcms plus a postgres service with a named volume and a pg_isready healthcheck, depends_on service_healthy, DB_DRIVER=postgres, DB_HOST=postgres, DATA_DIR=/data. `docker compose up`: /healthz 200, register, log in, create an article, restart the stack, data persists.
|
||||
26. Stop the postgres container and start renewcms: clear error, no password in the logs.
|
||||
27. In the Docker image (uid 10001), trigger the backup path and confirm the .sql file lands under /data/backups.
|
||||
28. Docs: .env.example, docker.env.example, docs/configuration.md, docs/docker.md, docs/deployment.md, docs/development.md and the docs/architecture.md "Storage and database seams" section list DB_DRIVER=postgres, DB_SSLMODE and the manual migration note. DB_SSLMODE goes into cmd/server/main.go envDefaults.
|
||||
29. SQLite default end to end with no new vars set behaves as before.
|
||||
Reference in new issue
Block a user