libretechandClaude Opus 4.7
bcccba92f7
CI deployment (.gitea/workflows/deploy.yml):
- Two jobs (build, deploy) gated on the repo variable
DEPLOY_ENABLED=true so the workflow exists but does nothing
until secrets and host are configured.
- Build pushes two image tags per run: rolling :main + the short
SHA on main, or vX.Y.Z + :latest on tag pushes. Immutable per
commit/tag tags make rollback trivial.
- Deploy SSHes to DEPLOY_HOST, runs docker compose pull && up -d
in DEPLOY_PATH, then polls HEALTH_URL for up to a minute. A
failed health check fails the workflow, which is the alert.
- Required secrets and the rollback procedure are documented in
docs/operations.md.
Backup tooling (scripts/):
- backup.sh: SQLite online .backup snapshot + tarball of the
per-tenant data dir + info.txt header, all wrapped into a
single librenotes-YYYYMMDD-HHMMSS.tar.gz. Optional BACKUP_REMOTE
triggers an rclone copy for off-site storage.
- backup-prune.sh: enforces retention "30 daily + 12 monthly".
Sorts archives by filename (date is in the name so lex order
matches chronological) and keeps the newest 30 plus the newest
archive for each of the most recent 12 months.
- backup-restore-test.sh: extracts the most recent archive into
a tmpdir, runs sqlite3 .schema (proves DB readability), and
asserts the notes tar has at least one entry. Failure is the
alert. Wired into a separate weekly timer.
- librenotes-backup.{service,timer}: systemd units for the daily
03:17 UTC run with 5min jitter; ProtectSystem=strict, only
/var/backups/librenotes is writable.
- librenotes-backup-verify.{service,timer}: weekly Monday
04:00 UTC restore test.
Closes #26 and #27.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
118 lines
3.8 KiB
Markdown
118 lines
3.8 KiB
Markdown
# Operations
|
|
|
|
This document covers deployment, backups, and rollback for the
|
|
librenot.es production environment. For developer / contributor
|
|
docs see the top-level [README](../README.md).
|
|
|
|
## Deployment
|
|
|
|
The repository ships a CI workflow at `.gitea/workflows/deploy.yml`
|
|
that builds and pushes a Docker image on every push to `main` and
|
|
on every `vX.Y.Z` tag, then SSHes to the deployment host and runs
|
|
`docker compose pull && up -d`.
|
|
|
|
The workflow is gated on the repository variable
|
|
`DEPLOY_ENABLED=true`. Deployment is opt-in; flipping the variable
|
|
disables the workflow without removing the file.
|
|
|
|
### Required secrets
|
|
|
|
The workflow expects these secrets and variables on the repo:
|
|
|
|
| Name | Type | Purpose |
|
|
| ---- | ---- | ------- |
|
|
| `REGISTRY` | secret | hostname of the OCI registry |
|
|
| `REGISTRY_USER` | secret | robot account |
|
|
| `REGISTRY_PASS` | secret | robot token |
|
|
| `DEPLOY_HOST` | secret | `user@host` SSH target |
|
|
| `DEPLOY_KEY` | secret | passphrase-less private key |
|
|
| `DEPLOY_PATH` | secret | absolute path on host with `docker-compose.*.yml` |
|
|
| `HEALTH_URL` | secret | e.g. `https://librenot.es/healthz` |
|
|
| `DEPLOY_ENABLED` | variable | `true` to enable the workflow |
|
|
|
|
### Production compose stack
|
|
|
|
On the deployment host, place `docker-compose.yml` and
|
|
`docker-compose.prod.yml` from this repo at `$DEPLOY_PATH`,
|
|
together with an `.env` file containing the runtime configuration
|
|
(JWT secret, SMTP credentials, public base URL, image tag).
|
|
|
|
Bring it up with:
|
|
|
|
```sh
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
```
|
|
|
|
### Rollback
|
|
|
|
Image tags are immutable per commit / version. To roll back, set
|
|
`LIBRENOTES_IMAGE` to the previous tag in `.env` and run the same
|
|
`up -d` command. The deployment workflow does not auto-rollback
|
|
on health-check failure — failed health alerts the operator via
|
|
the workflow itself, who can then redeploy the prior tag manually.
|
|
|
|
```sh
|
|
# Example: roll back to v0.1.2
|
|
sed -i 's/^LIBRENOTES_IMAGE=.*/LIBRENOTES_IMAGE=registry.librete.ch\/librenotes:v0.1.2/' .env
|
|
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
|
```
|
|
|
|
## Backups
|
|
|
|
`scripts/backup.sh` is a self-contained backup driver suitable for
|
|
running from cron, a systemd timer, or the supplied
|
|
`scripts/librenotes-backup.{service,timer}` units.
|
|
|
|
Required tools on the backup host: `sqlite3`, `tar`, `gzip`, and
|
|
optionally `rclone` for off-site copy. On Debian/Ubuntu:
|
|
`apt install sqlite3 rclone`.
|
|
|
|
### What is backed up
|
|
|
|
- The SQLite database (`$LIBRENOTES_DB`) via the SQLite
|
|
`.backup` command, which produces a consistent online snapshot
|
|
without needing to stop the application.
|
|
- The per-tenant note directory (`$LIBRENOTES_DATA_DIR`) as a
|
|
gzipped tar.
|
|
- An `info.txt` in each archive recording the backup timestamp,
|
|
hostname, and version.
|
|
|
|
### Off-site copy
|
|
|
|
Set `BACKUP_REMOTE` to an `rclone` destination (e.g.
|
|
`s3:librenotes-backups`). When set, the script invokes
|
|
`rclone copy` to upload the archive after creation. Without it,
|
|
backups stay on the local disk only.
|
|
|
|
### Retention
|
|
|
|
`scripts/backup-prune.sh` enforces the policy "keep 30 daily
|
|
+ 12 monthly". Run it from the same timer as the backup. Files
|
|
keep the form `librenotes-YYYYMMDD-HHMMSS.tar.gz` so the prune
|
|
script can sort and select by name alone.
|
|
|
|
### Restore test
|
|
|
|
`scripts/backup-restore-test.sh` picks the most recent archive,
|
|
extracts it into a scratch directory, runs `sqlite3 .schema` on
|
|
the database to confirm it is readable, and verifies that the
|
|
note tar lists at least one entry. It is wired into a separate
|
|
weekly timer so a silent backup-corruption regression cannot hide
|
|
indefinitely.
|
|
|
|
### Systemd
|
|
|
|
Drop `scripts/librenotes-backup.{service,timer}` into
|
|
`/etc/systemd/system/`, then:
|
|
|
|
```sh
|
|
systemctl daemon-reload
|
|
systemctl enable --now librenotes-backup.timer
|
|
```
|
|
|
|
Confirm with:
|
|
|
|
```sh
|
|
systemctl list-timers librenotes-backup.timer
|
|
```
|