fsylvain/RenewCMS
1
1 Fork 0
Code Issues Pull Requests Packages Projects Releases 20 Wiki Activity

feat: setup logs retention by max file size

Browse Source
This commit is contained in:
fsylvain committed 2026-10-07 22:02:13 +02:00
1 parent 70c8492f4f
commit f914eae3c5
44 files changed
+2316 -161
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

features/activity_home_card_and_retention.feature
+269
View File
@@ -347,3 +347,272 @@ Feature: Recent activity card on Home and Retention card on Settings
Scenario: Existing behaviour is unchanged
Then every scenario of home.feature, settings_page.feature (except its exact-body statement, superseded above) and the Media cleanup scenarios still holds with GET /v1/activity?size=5 mocked and the settings mock widened with the four activity_* fields
Rule: Retention size limit, an optional maximum size as an alternative to the age limit
Background:
Given this rule supersedes the statements above where they conflict: the settings body, the next-cleanup rule, the PATCH validation, the card copy and the shortening rule
And this extension changes Go code in the audit settings, the audit repositories, the settings and cleanup use cases and the settings handler
And GET /v1/settings and the PATCH response carry exactly {"version", "activity_retention_days", "activity_retention_max_gb", "activity_log_size_bytes", "activity_last_cleanup_at", "activity_last_cleanup_removed", "activity_next_cleanup_at"}, snake_case
And activity_next_cleanup_at is null only when activity_retention_days is 0 and activity_retention_max_gb is 0, otherwise it is the next 03:00 (audit.CleanupHour) in the server zone strictly after now, so Forever with a size limit has a schedule
And the PATCH body is a partial object with activity_retention_days in {0, 30, 90, 365} and/or activity_retention_max_gb in {0, 1, 5, 10, 25, 50}, absent keys are left unchanged, and an empty object, a null, a wrong type, any other value or malformed JSON answers 400 with apperr.BodyErrorMessage and saves nothing
And a non-owner PATCH answers 403 "Only the owner can do this."
And validation goes through validate tags and validator.Validate.Struct
And the size options are one list in the audit domain, audit.SizeOptions(), mirroring RetentionOptions()
And activity_log_size_bytes is the payload size of the activity events table, measured by audit.Repository.SizeBytes: the sum of the OCTET_LENGTH of the text columns action, actor_name, target_type, target_id, target_name, failure_reason, ip, user_agent, params and changes plus a fixed per-row overhead constant defined in the domain
And it is not the database file size and excludes indexes, free pages and other tables, and repositories never touch files
And the audit Repository gains SizeBytes() (int64, error) and DeleteOldest(limit int, before time.Time) (int, error); DeleteOldest removes the oldest events by created_at then id and only those older than before
And the cleanup runs the age step when days is greater than 0, then removes the oldest events in batches through DeleteOldest, calling SizeBytes between batches, until the log is at or under the limit or a batch removes 0
And the cutoff of every removal is the 24 hour guard (audit.KeepRecentHours); when the limit cannot be met because of the guard, the cleanup stops without error, records the run and tries again at the next 03:00
And Tick returns early only when days is 0 and max_gb is 0
And the removed count is the total of the age and size removals, in the last-cleanup fields and in the system.activity_cleanup params {"removed": n}
And system.activity_retention_changed is recorded once per PATCH that changes something, with params {"from","to"} when the age changes and "max_gb_from" and "max_gb_to" when the size changes, and no event when nothing changes
And lib/activitySentence.ts has an English and a French sentence for a size change
And GET /v1/settings computes the size on every call and a SizeBytes failure answers 500, like any use case error; every verified user receives the number, only the owner sees the card
And the SPA shows the size in GB (1 GB is 1024^3 bytes) with one decimal, or "Less than 0.1 GB" below 0.1 GB
And the header description is "How long activity is kept, and how much log data to keep. Whichever limit is reached first removes the oldest events. Only owners can change it."
And the append-only alert reads "Events can't be edited or deleted from the app. Only the retention limits remove them."
And below the age select an "or" divider is followed by the label "Keep at most" over a USelectMenu 160 px wide (full width on mobile) with the items "No size limit", "1 GB", "5 GB", "10 GB", "25 GB", "50 GB" (values 0, 1, 5, 10, 25, 50)
And the age helper is "Older events are removed automatically." without a size limit, "Events older than this are removed." with one, "Nothing is removed automatically. The log grows over time." for Forever without a size limit and "No age limit. Only the size limit applies." for Forever with one
And the size helper is "Optional. Logs can grow without a size cap." without a limit and "When logs pass 5 GB, the oldest events are removed. Currently 3.2 GB." with one, the figure being activity_log_size_bytes and never a hard-coded value
And the size rank is 1 < 5 < 10 < 25 < 50 < No size limit and a lower rank than the saved one is "smaller"
And choosing a smaller size, including a size when there was none, sends nothing and shows the warning "Events over {n} GB will be removed" with "This happens at the next cleanup on {date}. It can't be undone.", "Apply {n} GB" and "Cancel"; choosing a larger size or "No size limit" saves at once
And each select sends only its own key, both share one status area, one error alert and one pending warning, and a warning concerns one control at a time: a longer or larger choice on either control drops it and another shortening replaces it
And the visual states are "Default, 90 days or 5 GB", "Age only (no size limit)", "Size only (forever)", the shortening warning with a 5 GB limit, Forever and Save error
Scenario Outline: Settings API, GET reports the size limit and the measured size
Given the saved retention is <days> days with a size limit of <gb> GB and the activity table holds <bytes> bytes of payload
When a verified user sends GET /v1/settings at the server time 2026-10-06T10:00:00+02:00
Then the status is 200 and the body has exactly the seven keys
And "activity_retention_max_gb" is <gb> and "activity_log_size_bytes" is <bytes>
And "activity_next_cleanup_at" is <next>
Examples:
| days | gb | bytes | next |
| 90 | 0 | 0 | "2026-10-07T03:00:00+02:00" |
| 90 | 5 | 3435973837 | "2026-10-07T03:00:00+02:00" |
| 0 | 5 | 3435973837 | "2026-10-07T03:00:00+02:00" |
| 0 | 0 | 1024 | null |
Scenario: Settings API, a fresh install defaults to no size limit
Given no settings row exists
When a verified user sends GET /v1/settings
Then activity_retention_days is 90, activity_retention_max_gb is 0 and activity_log_size_bytes is 0
Scenario Outline: Settings API, PATCH saves a size limit, an age, or both
Given the saved retention is 90 days and no size limit and the user is the owner
When the owner sends PATCH /v1/settings with <body>
Then the status is 200 and the body has activity_retention_days <days>, activity_retention_max_gb <gb> and activity_next_cleanup_at <next>
And the repository holds <days> and <gb>
Examples:
| body | days | gb | next |
| {"activity_retention_max_gb": 5} | 90 | 5 | the next 03:00 server time |
| {"activity_retention_days": 0, "activity_retention_max_gb": 5} | 0 | 5 | the next 03:00 server time |
| {"activity_retention_days": 30} | 30 | 0 | the next 03:00 server time |
| {"activity_retention_days": 0} | 0 | 0 | null |
Scenario: Settings API, leaving the size limit while Forever clears the schedule
Given the saved retention is 0 days with a size limit of 5 GB
When the owner sends PATCH /v1/settings with {"activity_retention_max_gb": 0}
Then activity_next_cleanup_at is null
Scenario Outline: Settings API, PATCH rejects invalid size input
Given the saved retention is 90 days with a size limit of 5 GB
When the owner sends PATCH /v1/settings with <body>
Then the status is 400 with the text of apperr.BodyErrorMessage and nothing is saved
Examples:
| body |
| {"activity_retention_max_gb": 2} |
| {"activity_retention_max_gb": -1} |
| {"activity_retention_max_gb": 100} |
| {"activity_retention_max_gb": 5.5} |
| {"activity_retention_max_gb": "5"} |
| {"activity_retention_max_gb": null} |
| {"activity_retention_days": 30, "activity_retention_max_gb": 2} |
| {} |
Scenario: Settings API, a PATCH changing nothing is a no-op
Given the saved retention is 90 days and 5 GB
When the owner sends PATCH /v1/settings with {"activity_retention_days": 90, "activity_retention_max_gb": 5}
Then the status is 200, nothing is saved and no audit event is recorded
Scenario: Settings API, a non-owner cannot set the size limit
When an admin sends PATCH /v1/settings with {"activity_retention_max_gb": 5}
Then the status is 403 with "Only the owner can do this." and nothing is saved
Scenario: Settings API, a failing size measurement answers 500
Given SizeBytes fails
When a verified user sends GET /v1/settings
Then the status is 500 with the generic internal error text
Scenario Outline: Settings API, the change is audited
Given the saved retention is 90 days and no size limit
When the owner sends PATCH /v1/settings with <body>
Then one event "system.activity_retention_changed" exists with params <params>
Examples:
| body | params |
| {"activity_retention_days": 30} | {"from": 90, "to": 30} |
| {"activity_retention_max_gb": 5} | {"max_gb_from": 0, "max_gb_to": 5} |
| {"activity_retention_days": 30, "activity_retention_max_gb": 5} | {"from": 90, "to": 30, "max_gb_from": 0, "max_gb_to": 5} |
Scenario Outline: Cleanup, the size limit removes the oldest events
Given the retention is <days> days, the size limit is 5 GB and the log is 7 GB
And the oldest events are older than the guard
When the cleanup runs at 2026-10-06T03:00:00
Then events are removed oldest first, by created_at then id, until SizeBytes is at or under 5 GB
And the removed count is the total of the age removals and the size removals
And the last cleanup time is the clock's time
Examples:
| days |
| 90 |
| 0 |
Scenario: Cleanup, age first then size
Given the retention is 30 days, the size limit is 5 GB, 100 events are older than 30 days and the log still exceeds 5 GB after they go
When the cleanup runs
Then the 100 events go first, then the oldest remaining events until the log is at or under 5 GB
Scenario: Cleanup, the age limit alone is enough
Given the log is under the size limit after the age removal
When the cleanup runs
Then no size removal happens
Scenario: Cleanup, Forever with no size limit never runs
Given the retention is 0 days and no size limit
When Tick runs at 2026-10-06T03:00:00
Then ran is false and neither DeleteOlderThan nor DeleteOldest is called
Scenario: Cleanup, the last 24 hours are protected even when the size cannot be met
Given the size limit is 1 GB and the log is 4 GB, all of it from the last 24 hours
When the cleanup runs
Then nothing is removed, no error is returned, the last cleanup records 0 removed and the next tick tries again
Scenario: Cleanup, a failing size removal is retried
Given DeleteOldest fails
When Tick runs
Then the error is returned and logged, the last cleanup fields are unchanged and the next tick runs again
Scenario: Cleanup, a size removal records one system event
Given the age step removed 10 events and the size step removed 40
When the cleanup runs
Then one "system.activity_cleanup" event exists with params {"removed": 50}
Scenario: Persistence measures the table payload portably
Given a real in-memory SQLite database and events with known text lengths, one with a multi-byte name
Then SizeBytes equals the sum of the byte lengths of the text columns plus the fixed per-row overhead
And it is 0 for an empty table
And deleting events lowers it
Scenario: Persistence deletes the oldest events up to a limit and a guard
Given 10 events, 3 of them newer than the guard
When DeleteOldest runs with limit 5
Then the 5 oldest events are removed
When DeleteOldest runs with limit 20
Then only the 2 remaining events older than the guard are removed and the 3 newest stay
Scenario: Settings retention, Default with 90 days or 5 GB
Given the role is "owner" and GET /v1/settings resolves 200 with activity_retention_days 90, activity_retention_max_gb 5 and activity_log_size_bytes 3435973837
When the owner navigates to /settings
Then the age select shows "90 days" with the helper "Events older than this are removed."
And an "or" divider and "Keep at most" show "5 GB" with the helper "When logs pass 5 GB, the oldest events are removed. Currently 3.2 GB."
And the lock alert reads "Events can't be edited or deleted from the app. Only the retention limits remove them."
And "Next cleanup" shows the date from activity_next_cleanup_at
Scenario: Settings retention, Age only (no size limit)
Given activity_retention_max_gb is 0
When the owner navigates to /settings
Then the size select reads "No size limit" with the helper "Optional. Logs can grow without a size cap." and the age helper is "Older events are removed automatically."
Scenario: Settings retention, Size only (forever)
Given activity_retention_days is 0, activity_retention_max_gb is 5 and activity_next_cleanup_at is set
When the owner navigates to /settings
Then the age helper reads "No age limit. Only the size limit applies." and "Next cleanup" shows the date, not "Not scheduled"
Scenario: Settings retention, Forever with no size limit
Given activity_retention_days is 0, activity_retention_max_gb is 0 and activity_next_cleanup_at is null
When the owner navigates to /settings
Then the age helper reads "Nothing is removed automatically. The log grows over time." and "Next cleanup" reads "Not scheduled"
Scenario Outline: Settings retention, the measured size is formatted
Given the size limit is 5 GB and activity_log_size_bytes is <bytes>
When the owner navigates to /settings
Then the size helper ends with "<shown>"
Examples:
| bytes | shown |
| 3435973837 | Currently 3.2 GB. |
| 0 | Currently Less than 0.1 GB. |
| 53687091 | Currently Less than 0.1 GB. |
| 5368709120 | Currently 5.0 GB. |
Scenario Outline: Settings retention, a larger size or no limit saves at once
Given the saved size is <from>
When the owner chooses "<to>"
Then PATCH is sent once with {"activity_retention_max_gb": <value>}, never the age key, and no warning shows
And the shared status reads "Saving…" then "Saved"
Examples:
| from | to | value |
| 5 GB | 10 GB | 10 |
| 5 GB | No size limit | 0 |
| 1 GB | 50 GB | 50 |
Scenario Outline: Settings retention, a smaller size asks first
Given the saved size is <from>
When the owner chooses "<to>"
Then no PATCH is sent and the size select shows "<to>"
And a warning alert shows "Events over <n> GB will be removed" and "This happens at the next cleanup on 7 Oct 2026, 03:00. It can't be undone." with "Apply <n> GB" and "Cancel"
Examples:
| from | to | n |
| 10 GB | 5 GB | 5 |
| No size limit | 5 GB | 5 |
| 50 GB | 1 GB | 1 |
Scenario: Settings retention, Apply and Cancel for a smaller size
Given the warning for 5 GB is shown on a saved 10 GB
When the owner activates "Apply 5 GB" and PATCH resolves 200 with activity_retention_max_gb 5
Then PATCH was sent once with {"activity_retention_max_gb": 5} and "Saved" shows
Given the warning is shown again
When the owner activates "Cancel"
Then the size select reads the saved value and no PATCH was sent
Scenario: Settings retention, only one warning at a time
Given the age warning for 30 days is open
When the owner chooses "5 GB" as a smaller size
Then the age warning is replaced by the size warning and the age select reverts to the saved value
When the owner chooses a longer age or a larger size
Then the warning is dropped and PATCH is sent once for that control
Scenario: Settings retention, switching age to Forever with a size limit
Given the saved period is 90 days and 5 GB
When the owner chooses "Forever"
Then PATCH is sent with {"activity_retention_days": 0} and the age helper becomes "No age limit. Only the size limit applies."
And "Next cleanup" still shows the returned date
Scenario Outline: Settings retention, a size save error reverts only the size select
Given the saved size is 5 GB and the owner chooses "10 GB"
When PATCH /v1/settings <failure>
Then the error alert "Couldn't save the retention period" with "Your previous setting is still in place." and "Try again" shows
And the size select reads "5 GB" again and the age select is unchanged
When the owner activates "Try again" and PATCH resolves 200
Then PATCH was sent a second time with {"activity_retention_max_gb": 10} and "Saved" replaces the alert
Examples:
| failure |
| resolves with a 500 response |
| resolves with a 400 response |
| resolves with a 403 response |
| never reaches the server |
Scenario: Settings retention, offline disables the size select too
Given the owner is on /settings and the browser goes offline
Then both selects, "Apply" and "Cancel" are disabled
Scenario: The French interface covers the size strings
Given the language is French
Then every new string has a French entry in messages.ts, and the Activity sentence for a size change is French