Skip to content

Backups

Current state

There is no backup of the application database.

This was verified directly on the server rather than assumed. There is no scheduled job, no periodic dump, and no copy of the data anywhere other than the running database.

Coolify does run nightly backups, which is easy to mistake for coverage. Those backups are of Coolify's own internal database, which records what applications exist and how they are configured. They contain none of this project's data.

The database is small, about 12 MB. Nothing about the size makes backups difficult; this gap is simply unclosed.

What that means today

The only copy of every profile, announcement, role assignment, and subscriber is the one running PostgreSQL instance on one server. There is nothing to restore from if the disk fails, the server is deleted, or a migration damages data in a way that is not caught immediately.

Photos are in the same position. They live in the object store on the same server.

What a backup should look like

The standard shape is three copies, on two kinds of storage, with one of them somewhere else entirely. Applied here:

CopyWhereProtects against
The live databaseOn the serverNothing. This is the thing being protected
A dump written to diskOn the serverA bad migration, an accidental delete
That dump, shipped off the serverElsewhereLosing the server itself

The third copy is the one that matters most and the one that does not exist. A backup stored only on the server dies with the server.

The mechanism

pg_dump is PostgreSQL's own export tool and is already present inside the database container. It writes everything needed to rebuild the database exactly: tables, rows, constraints, policies, functions, triggers.

The custom format is compressed and allows selective restores, which is useful when you need one table back rather than all of them. pg_restore reads it.

Remember that the database is named admin. A dump command that omits the database name will quietly back up the wrong one.

A plan sized to this project

  1. A nightly dump, compressed, written to the server.
  2. Each dump uploaded to storage that is not this server. A storage box at the same hosting provider is the least friction; an S3-compatible bucket elsewhere is the more independent choice.
  3. A rolling retention window rather than keeping everything forever. Two weeks of daily copies plus a few weekly ones is more than adequate at this size.
  4. Object storage included, not only the database. Photos are member data too.
  5. A periodic restore into a throwaway database, to prove the backups work.

Point five is the one people skip. A backup nobody has ever restored from is an assumption, not a safeguard. The same scratch-database approach used for testing migrations works here.

Restoring

The procedure should be written down before it is needed, and tested at least once, so that nobody is reading documentation for the first time during an outage.

When the backup system is built, this section should describe the exact restore steps, including how long a full restore takes.

Internal engineering documentation.