feat: optional MySQL/MariaDB database driver for deployments that cannot use local SQLite

features/mysql_database_driver.feature

@@ -0,0 +1,160 @@
+Feature: Optional MySQL/MariaDB database driver
+  An operator who cannot use a local SQLite file runs RenewCMS against MySQL or
+  MariaDB by setting environment variables. SQLite remains the default and
+  behaves exactly as before.
+
+  Background:
+    Given the database drivers are registered by name in database.drivers
+    And the database is chosen by the DB_DRIVER environment variable
+    And repositories, use cases and handlers are unchanged
+
+  Scenario Outline: Driver selection
+    Given DB_DRIVER is "<value>"
+    When the server starts
+    Then the outcome is "<outcome>"
+
+    Examples:
+      | value  | outcome                                                  |
+      | unset  | the sqlite driver opens DB_FILE                          |
+      | sqlite | the sqlite driver opens DB_FILE                          |
+      | mysql  | the mysql driver opens the DB_HOST connection            |
+      | oracle | startup fails with: unsupported database driver "oracle" |
+
+  Scenario Outline: Missing MySQL variable
+    Given DB_DRIVER is "mysql"
+    And "<missing>" is not set
+    When the environment is validated
+    Then startup fails with "Environment variable <missing> is not set"
+
+    Examples:
+      | missing     |
+      | DB_HOST     |
+      | DB_USER     |
+      | DB_PASSWORD |
+      | DB_NAME     |
+
+  Scenario: SQLite does not require MySQL variables
+    Given DB_DRIVER is "sqlite"
+    And DB_HOST, DB_USER, DB_PASSWORD and DB_NAME are not set
+    When the environment is validated
+    Then validation succeeds
+
+  Scenario Outline: DSN built from variables
+    Given DB_HOST "<host>", DB_PORT "<port>", DB_USER "app", DB_NAME "cms", DB_TLS "<tls>"
+    When the MySQL DSN is built
+    Then the DSN targets "<host>:<expectedPort>" and database "cms"
+    And it contains parseTime=true, loc=UTC, charset=utf8mb4, clientFoundRows=true
+    And it contains tls="<tls>"
+
+    Examples:
+      | host     | port | expectedPort | tls         |
+      | db.local |      | 3306         | false       |
+      | db.local | 3307 | 3307         | true        |
+      | mysql    | 3306 | 3306         | skip-verify |
+
+  Scenario: Invalid TLS option is rejected
+    Given DB_DRIVER is "mysql" and DB_TLS is "maybe"
+    When the settings are read
+    Then startup fails naming DB_TLS and the accepted values
+
+  Scenario Outline: Fail fast when the server cannot be used
+    Given DB_DRIVER is "mysql" and "<fault>"
+    When the server starts
+    Then startup stops with "Unable to open the database"
+    And the message names the host and port
+    And the message does not contain the DB_PASSWORD value
+
+    Examples:
+      | fault                       |
+      | nothing listens on DB_PORT  |
+      | the password is wrong       |
+      | the database does not exist |
+
+  Scenario: Data directory with MySQL
+    Given DB_DRIVER is "mysql" and DATA_DIR is "/data"
+    Then update state, staging and backups live in "/data"
+    And DB_FILE is ignored
+
+  Scenario Outline: DataDir resolution
+    Given DB_FILE "<dbfile>" and DATA_DIR "<datadir>"
+    Then Settings.DataDir() is "<result>"
+
+    Examples:
+      | dbfile              | datadir | result       |
+      | ./data/renewcms.db  |         | ./data       |
+      | /var/lib/cms/cms.db |         | /var/lib/cms |
+      | ./data/renewcms.db  | /srv/x  | /srv/x       |
+
+  Scenario: Connection pool
+    Given DB_MAX_OPEN_CONNS 4, DB_MAX_IDLE_CONNS 2 and DB_CONN_MAX_LIFETIME_SECONDS 60
+    When the MySQL connection is opened
+    Then the pool limits are 4 open, 2 idle and 60 seconds
+
+  Scenario: Migration on MySQL
+    Given an empty MySQL database
+    When database.Migrate runs
+    Then the tables articles, images, users, invites, api_keys and purge_settings exist
+    And the unique indexes on users and api_keys exist
+    And the first user lacking a role becomes "owner" and the others "admin"
+
+  Scenario: SQLite schema is untouched by the model changes
+    Given a SQLite file created by the models before this change, holding rows
+    When database.Migrate runs
+    Then the schema and every row are unchanged
+
+  Scenario Outline: Repository behaviour is the same on both drivers
+    Given the "<driver>" driver
+    Then the repository contract "<case>" holds
+
+    Examples:
+      | driver | case                                                                 |
+      | sqlite | create user and read it back with its id and timestamps              |
+      | mysql  | create user and read it back with its id and timestamps              |
+      | sqlite | username "Bob" and "bob" are distinct users                          |
+      | mysql  | username "Bob" and "bob" are distinct users                          |
+      | sqlite | duplicate email returns an error and stores nothing                  |
+      | mysql  | duplicate email returns an error and stores nothing                  |
+      | sqlite | unknown id returns domain ErrUserNotFound                            |
+      | mysql  | unknown id returns domain ErrUserNotFound                            |
+      | sqlite | accept invite twice: the second returns ErrInviteNotFound            |
+      | mysql  | accept invite twice: the second returns ErrInviteNotFound            |
+      | sqlite | accept invite with a taken username leaves the invite unaccepted     |
+      | mysql  | accept invite with a taken username leaves the invite unaccepted     |
+      | sqlite | renew invite with unchanged values still succeeds                    |
+      | mysql  | renew invite with unchanged values still succeeds                    |
+      | sqlite | duplicate image hashes are grouped and counted                       |
+      | mysql  | duplicate image hashes are grouped and counted                       |
+      | sqlite | media search treats % and _ literally                                |
+      | mysql  | media search treats % and _ literally                                |
+      | sqlite | article body of 1 MB round-trips                                     |
+      | mysql  | article body of 1 MB round-trips                                     |
+      | sqlite | zero reset expiration round-trips as a zero time                     |
+      | mysql  | zero reset expiration round-trips as a zero time                     |
+      | sqlite | purge settings save, then SaveLastRun with identical values succeeds |
+      | mysql  | purge settings save, then SaveLastRun with identical values succeeds |
+
+  Scenario Outline: Backup before an update
+    Given the "<driver>" driver with a populated database
+    When the pre-update backup runs
+    Then a file "<pattern>" exists in DATA_DIR/backups
+    And only the 3 newest backups are kept
+    And a failure returns an error, so the update download fails instead of claiming a backup
+
+    Examples:
+      | driver | pattern             |
+      | sqlite | renewcms-<time>.db  |
+      | mysql  | renewcms-<time>.sql |
+
+  Scenario: MySQL backup content
+    Given a MySQL database with 2 users and 1 article
+    When the pre-update backup runs
+    Then the dump contains the schema and the rows of every table
+    And it does not contain the DB_PASSWORD value
+    And replaying it into an empty database restores the same rows
+
+  Scenario: Secrets are never exposed
+    Given DB_DRIVER is "mysql"
+    Then no log line, error message or file in the repository contains the DB_PASSWORD value
+
+  Scenario: Static builds
+    Then the module builds with CGO_ENABLED=0 for the 6 task release targets

features/mysql_database_driver.qa.md

@@ -0,0 +1,42 @@
+# QA: MySQL/MariaDB database driver
+
+Backend only. Go tests in test/, black-box, no live server by default.
+
+Decisions: separate DB_* vars, new DATA_DIR (DB_FILE ignored under mysql), pure-Go logical dump backup, `task test:mysql` against mysql:8.4 and mariadb:11, SQLite-to-MySQL data migration documented as manual export/import only.
+
+## Automated, no server (task test)
+1. Registry: Connect with Driver "sqlite" works against a temp file; Driver "oracle" returns `unsupported database driver "oracle"`; the registry contains sqlite and mysql.
+2. Env validation: table-driven over the Driver selection and Missing MySQL variable scenarios. SQLite needs no DB_* vars. DB_DRIVER defaults to sqlite. An invalid DB_TLS or a non-numeric pool var returns an error naming the variable.
+3. DSN builder: table-driven over the DSN scenario. Assert host:port, database, parseTime, loc=UTC, charset=utf8mb4, clientFoundRows, tls. Assert that the redacted form used in errors never contains the password.
+4. Settings.DataDir: table-driven (DB_FILE only, DATA_DIR overrides).
+5. Fail fast without a server: point mysql at a closed local port with a 1s timeout. Assert the error mentions the host and port, and not the password.
+6. Pool settings: assert sql.DB stats MaxOpenConnections via a stub opener, or via the real mysql test below.
+7. SQLite regression: build a database file using the pre-change schema (a fixture created with the old model definitions), run Migrate, and assert that the schema (sqlite_master) and the rows are unchanged. The existing suite must pass unchanged.
+8. Backup (SQLite): existing tests unchanged.
+9. Dump writer: unit-test the SQL dump formatter with a fake row source (quoting, NULL, time, bool, binary-safe strings, 3-file retention, existing-file handling, failure leaves no partial file).
+
+## Automated, real server (task test:mysql)
+Setup: the task runs `docker run` for mysql:8.4 and, in a second pass, mariadb:11, waits for readiness, exports RENEWCMS_TEST_MYSQL_DSN, runs `go test -tags nofrontend ./test/... -run MySQL -count=1`, then removes the container. Tests call t.Skip when the var is unset, so `task test` skips them. The repository tests are parameterised over both drivers (SQLite temp file and the MySQL DSN, each with a fresh database), and run the "Repository behaviour is the same on both drivers" Examples.
+10. Migrate on an empty database; table and index existence via information_schema; the backfill role rules.
+11. User create/read, case-sensitive username and email, duplicate email, ErrUserNotFound.
+12. Invite accept twice, accept with a taken username (invite stays unaccepted), renew with unchanged values.
+13. Image duplicates grouping and counts, search with % and _, filename backfill listing.
+14. Article body 1 MB, title lookup, online filter.
+15. API key unique hashed_key, revoke, last_used.
+16. Zero-time round trip, UTC times.
+17. Purge settings upsert.
+18. Backup: run on a populated MySQL database, assert the .sql file, the 3-file retention, a replay into a second empty database giving identical rows, and that the file contains no password.
+19. Wrong password and missing database fail with the "Unable to open the database" prefix and no password in the message.
+
+## Build and lint
+20. `task lint:go` and `task test` pass. `task test:mysql` passes on MySQL 8.4 and MariaDB 11 (record both; NOT RUN if Docker is unavailable).
+21. `CGO_ENABLED=0 go build -tags nofrontend ./...` passes. Cross-compile the 6 `task release` targets (e.g. `GOOS=linux GOARCH=arm64`) with CGO_ENABLED=0.
+22. `task mutate` impact: none, because it only targets useCases. No change.
+23. No comment lines in the added Go files (the hook enforces this).
+
+## Manual (needs Compose and real SMTP)
+24. docker-compose.mysql.yml example: renewcms plus a mysql service with a named volume and a healthcheck, `depends_on: condition: service_healthy`, DB_DRIVER=mysql, DB_HOST=mysql, DATA_DIR=/data. `docker compose up`: /healthz returns 200, register, log in, create an article, restart the stack, and the data persists.
+25. Stop the mysql container and start renewcms: it exits with a clear error and no password in the logs. docker.env.example holds placeholders only.
+26. In the Docker image (non-root uid 10001), trigger the backup path and confirm the .sql file lands under /data/backups.
+27. Docs: configuration.md, docker.md, deployment.md, `.env.example`, `docker.env.example` and the architecture.md "Storage and database seams" section list the new vars and the manual SQLite to MySQL export/import note.
+28. Verify the SQLite default end to end with no new vars set: same behaviour as before.