Borgmatic Backup Container & Tools

A wrapper around borgmatic to simplify setting up backups, specifically designed to run as a container. It also has installed a bunch of helpful utilities to run my backup scripts.

Based on Arch Linux rolling release, it gets built daily.

Environment Variables

Core Borg Settings

• BORG_REPO_NAME: (Required for borgmatic) The name of the repository. It will be used to construct the full path: ssh://cyborg@brickyard.home.arpa:19198//volume1/backups/k8s.borg/{BORG_REPO_NAME} with the label brickyard.
• BORG_PASSPHRASE: (Required) The passphrase for the Borg repository encryption.
• BORG_PATTERNS_FILE: Path to a file containing a list of patterns to include/exclude (passed to patterns_from).
• BORG_ENCRYPTION: Encryption mode for repository initialization. Defaults to repokey-blake2.

SSH Configuration

• SSH_KNOWN_HOSTS: A space or comma-separated list of hostnames/IPs to scan and add to ~/.ssh/known_hosts at startup. Defaults to brickyard.home.arpa.
• SSH_PRIVATE_KEY: Content of the SSH private key to be used for authentication. It will be saved to ~/.ssh/id_container_key and used by borgmatic.

Scripts

• Any executable scripts found in /scripts will be added to the before_backup hook and executed before the backup runs.

PostgreSQL Backup

If PGHOST or PGDATABASE are set, a postgresql_databases hook is added to the borgmatic configuration.

• PGHOST: Database hostname.
• PGPORT: Database port (default 5432).
• PGUSER: Database username.
• PGDATABASE: Database name (defaults to all which triggers pg_dumpall).
• PGPASSWORD: Password should generally be provided via ~/.pgpass or this environment variable (standard libpq behavior).
• PGSSLMODE: SSL mode for the database connection (e.g., require, verify-full).

SQLite S3 Backup

If SQLITE_S3_PATHS is set, the container will automatically download SQLite databases from S3 using Litestream's restore functionality before starting the backup. These databases are then added to borgmatic's sqlite_databases section to be dumped and included in the archive.

• SQLITE_S3_PATHS: A space-separated list of databases in the format namespace/app/db_name.
• S3_ENDPOINT: The endpoint for the S3-compatible server. Defaults to http://brickyard.home.arpa:3900.
• LITESTREAM_FORCE_PATH_STYLE: Whether to force path-style S3 access. Defaults to true.
• LITESTREAM_SKIP_VERIFY: Set to true to skip TLS certificate verification for the S3 endpoint.
• AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY: Required credentials for accessing the S3 bucket.

TLS Certificates

• TLS_CUSTOM_CERTS_DIR: Path to a directory containing additional TLS certificates (.crt or .pem files) to be added to the trust store.

Monitoring

• HEALTHCHECKS_URL: The URL for a check on Healthchecks.io. Borgmatic will ping this URL on start, success, and failure.

S3 Configuration

The container is pre-configured to interact with the local Garage S3 endpoint at http://brickyard.home.arpa:3900.

• rclone: A remote named garage is pre-configured via environment variables. You can use it like: rclone ls garage:my-bucket. It uses standard AWS environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) for authentication.
• s3cmd: Pre-configured via /home/backup/.s3cfg to point to the same endpoint.

Usage

Default Behavior

Running the container without arguments will execute borgmatic (which defaults to performing a backup + prune + check based on the generated config).

docker run -d \
  -e BORG_REPO_NAME=app-backups \
  -e BORG_PASSPHRASE=secret123 \
  -e SSH_PRIVATE_KEY="-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\n...\n-----END OPENSSH PRIVATE KEY-----" \
  -v /my/data:/backups \
  c.code.nogweii.net/ops/containers/backup-tools:latest

There's a special mode if you pass the command "sh", it will drop you into a bash shell instead of running borgmatic:

docker run --rm -it c.code.nogweii.net/ops/containers/backup-tools:latest sh

Tools

Here are all the tools installed in the container:

• psql and pg_dump for PostgreSQL databases
• borg and borgmatic for generating backup archives
• ssh and specifically sftp for moving stuff around
• rclone for interacting with remote services (Pre-configured with a garage remote pointing to my local S3 endpoint.)
• s3cmd for interacting with S3 buckets (Pre-configured to talk to garage on my NAS.)
• curl for pinging Healthchecks.io
• rsync for smartly copying files to remote servers
• kubectl for interacting with the Kubernetes control plane
• sqlite3 for interacting with SQLite databases
• download-litestream for restoring SQLite snapshots from S3-compatible storage
• jq and yq for manipulating JSON and YAML text
• zstd for better compression
• redis-cli (via valkey) for Valkey/Redis databases