Skip to content

Backups

Overview

The Cloudron backup system creates a portable snapshot of the platform data and application data. Each app is backed up independently allowing them to be restored, cloned or migrated independently.

Unlike VM snapshots, these backups contain only the necessary information to complete re-instate the Cloudron or app. For example, application code and system libraries are not part of a backup because Cloudron packages are read-only and can never change. Runtime files (lock files, logs) and temporary files generated by apps are not backed up either. Only the database and app user data is backed up. This design significantly reduces the size of backups.

Storage providers

Amazon S3

To get started:

  • Create a bucket in S3.

Lifecycle rules

S3 buckets can have lifecycle rules to automatically remove objects after a certain age. When using the rsync format, these lifecycle rules may remove files from the snapshot directory and will cause the backups to be corrupt. For this reason, we recommend not setting any lifecycle rules that delete objects after a certain age. Cloudron will periodically clean up old backups based on the retention period.

  • AWS has two forms of security credentials - root and IAM. When using root credentials, follow the instructions here to create access keys. When using IAM, follow the instructions here to create a user and use the following policy to give the user access to the bucket:
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "s3:*",
                "Resource": [
                    "arn:aws:s3:::<your bucket name>",
                    "arn:aws:s3:::<your bucket name>/*"
                ]
            }
        ]
    }
  • In the Cloudron dashboard, choose Amazon S3 from the drop down.

DigitalOcean Spaces

To get started:

  • Create a DigitalOcean Spaces bucket in your preferred region following this guide.

  • Create DigitalOcean Spaces access key and secret key following this guide.

  • In the Cloudron dashboard, choose DigitalOcean Spaces from the drop down.

Rate limits

In our tests, we hit a few issues including missing implementation for copying large files (> 5GB), severe rate limits and poor performance when deleting objects. If you plan on using this provider, keep an eye on your backups. Cloudron will notify admins by email when backups fail.

Exoscale SOS

To get started:

  • Create a Exoscale SOS bucket

  • Create Access key and Secret Access Key from the Exoscale dashboard

  • In the Cloudron dashboard, choose Exoscale SOS from the drop down.

Filesystem

To get started:

External Disk

Having backups reside in the same physical disk as the Cloudron server is dangerous. For this reason, Cloudron will show a warning until the Backup directory is an external EXT4 Disk option is checked.

  • In the Cloudron dashboard, choose Filesystem from the drop down.

The Use hardlinks option can be checked to make the Cloudron use hardlinks 'same' files across backups to conserve space. If you happen to use a file system that does not support hardlinks (like NFS), just turn off hardlinks. This option has little to no effect when using the tgz format.

Samba/CIFS

Hosting providers like Hetzner and OVH provider storage boxes that be mounted using Samba/CIFS. To use a Samba/CIFS store, mount it into the file system using the iocharset=utf8 option.

For example, mount the samba share into your server by putting the following in /etc/fstab:

//<samba url>  /backups  cifs  uid=yellowtent,gid=yellowtent,user=<user>,pass=<pass>,iocharset=utf8  0  0

Google Cloud Storage

To get started:

Minio

To get started:

Be sure to install Minio on another server

Do not setup Minio on the same server as the Cloudron! Using the same server will inevitably result in data loss if something goes wrong with the server's disk. The minio app on Cloudron is meant for storing assets and not backups.

  • Create a bucket on Minio using the Minio CLI or the web interface.

  • Once setup, minio will print the necessary information, like login credentials, region and endpoints in it's logs.

$ ./minio server ./storage

Endpoint:  http://192.168.10.113:9000  http://127.0.0.1:9000
AccessKey: GFAWYNJEY7PUSLTHYHT6
SecretKey: /fEWk66E7GsPnzE1gohqKDovaytLcxhr0tNWnv3U
Region:    us-east-1
  • In the Cloudron dashboard, choose Minio from the drop down.

For HTTPS installations using a self-signed certificate, select the Accept Self-Signed certificate option.

Scaleway Object Storage

To get started:

No-op

This storage backend disables backups. When backups are disabled, updates to apps cannot be rolled back and result in data loss. This backend only exists for testing purposes.

Backup formats

Cloudron supports two backup formats - tgz (default) and rsync. The tgz format stores all the backup information in a single tarball whereas the rsync format stores all backup information as files inside a directory.

Both formats have the same content

The contents of the tgz file when extracted to disk will be the exact same as the contents of the rsync directory. Both the formats are complete and portable.

tgz format

The tgz format uploads an app's backup as a gzipped tarball. This format is very efficient when having a large number of small number files.

This format has the following caveats:

  • Most Cloud storage API require the content length to be known in advance before uploading data. For this reason, Cloudron uploads big backups in chunks. However, chunked (multi-part) uploads cannot be parallelized and also take up as much RAM as the chunk size.

  • tgz backup uploads are not incremental. This means that if an app generated 10GB of data, Cloudron has to upload 10GB every time it makes a new backup.

rsync format

The rsync format uploads individual files to the backup storage. It keeps track of what was copied the last time around, detects what changed locally and uploads only the changed files on every backup. Note that despite uploading 'incrementally', tgz format can be significantly faster when uploading a large number of small files (like source code repositories) because of the large number HTTP requests that need to be made for each file.

This format has the following caveats:

  • By tracking the files that were uploaded the last time around, Cloudron minimizes uploads when using the rsync format. To make sure that each backup directory is "self contained" (i.e can be simply copied without additional tools), Cloudron issues a 'remote copy' request for each file.

  • File uploads and remote copies are parallelized.

  • When using the file system backend, the rsync format can hardlink 'same' files across backups to conserve space.

  • When encryption is enabled, file names are encrypted as well.

App Snapshot

To take a backup of a single, click the Backups button in the app grid:

Then click Create Backup:

Full Snapshot

To take a backup of Cloudron and all the apps, click the Backup now button in the Settings page:

Warning

When using the no-op backend, no backups are taken. If backups are stored on the same server, be sure to download them before making changes in the server.

Disable automatic backups

An app can be excluded from automatic backups from the 'Advanced settings' in the Configure UI:

Note that the Cloudron will still create backups before an app or Cloudron update. This is required so that it can be reverted to a sane state should the update fail.

Warning

Disabling automatic backup for an app puts the onus on the Cloudron adminstrator to backup the app's files regularly. This can be done using the Cloudron CLI tool's cloudron backup create command.

Cloning an app

To clone an app i.e an exact replica of the app onto another domain, first create an app backup and then use the Clone UI to create a clone:

Restoring an app

Apps can be restored to a previous backup by clicking on the Backups button.

Select the backup you want to restore to:

Both data and code are reverted

Restoring will also revert the code to the version that was running when the backup was created. This is because the current version of the app may not be able to handle old data.

Restoring uninstalled app

If an app has been uninstalled but you still have the app backups, then you can use the CLI tool to restore it. The commands below must be run on your laptop/PC and not on the server.

First, prepare the CLI tool for use:

npm install -g cloudron
cloudron login my.<cloudron domain>

Next, determine the backup id to restore from. This can be done as follows:

  • Go to the Event Log in Cloudron dashboard and locate the uninstall event of the app. Clicking on the event will show app's id.

  • Go to your backup storage (Filesystem or S3) and browse to the latest storage. Backup directories are timestamped. Use the timestamp to locate the latest backup. Find the latest backup of the app. Backup will have the name app_<appid>_<version>. Depending on the backup format (tgz or rsync) this will be a single file or a directory. Make a note of the file name or the directory name including the timestamp.

  • Determine the appstore id of the app. You can find this id by browsing the appstore and looking at the URL bar. An appstore ID is like com.nextcloud.cloudronapp or org.wordpress.cloudronapp.

  • We are now ready to restore the app using the CLI tool. Replace the appstore id, backup id and format based on your setup. Note that the backup id does not contain the .tar.gz extension and contains the timestamp prefix (see the example below).

    cloudron install --appstore-id <apps appstore id>@<specific version if required> --backup <backupId> --backup-format <rsync|tgz>

Example:

~$ cloudron install --backup 2019-01-29-024014-522/app_a65bb5da-29a2-41cd-9631-b30e6f3f9afb_2019-01-29-024015-838_v2.0.4 --backup-format tgz --appstore-id org.wordpress.cloudronapp
Location (subdomain): test2
App is being installed.

 => Waiting to start installation
 => Registering subdomain .
 => Download backup and restoring addons
 => Waiting for DNS propagation ...................

App is installed.

Encryption key

If your backup has a secret/encryption key, then the key is picked from your current backup settings automatically. If the current key is different from the backup's key, just change the current backup configuration to have the backup's key temporarily.

Migrate Apps from one Cloudron to another

Migrating apps or moving apps from one Cloudron to another works by first creating a new backup of the app on the old Cloudron, copying the backup tarball onto the new Cloudron's backup storage and then installing a new app, based on the backup on the new Cloudron.

Migrating an app across Cloudrons has the following caveats:

  • When moving an app from one Cloudron to another, the user ids on one Cloudron will be different from the user ids on the other Cloudron. Cloudron does not support automatic user id mapping at this point. For this reason, you might have to manually fix up the database of the new app in the new Cloudron. A workaround is to install apps without using Cloudron Single Sign-On (for apps that support it).

  • Both Cloudrons have to use the same backup encryption key.

The following steps will migrate an app:

  • Against the old Cloudron
cloudron backup create --app <subdomain/appid>
  • Copy the new backup from the old Cloudron's backup storage to the new one. This can be done via the s3 webinterface or scp if the filesystem provider is used. The backup can be located by its backup ID, which can be seen with:
cloudron backup list --app <subdomain/appid>
  • Make note of the app's appstore id and version from:
cloudron list
  • Then login to the new Cloudron and install the new app based on the created backup:
cloudron login my.<new Cloudron domain>
cloudron install --appstore-id <apps appstore id>@<specific version if required> --backup <backupId> --backup-format <rsync|tgz>

Note

The backupId usually also includes a path prefix and looks like and must not contain the file extension (eg. .tar.gz): 2017-07-17-121412-248/app_2d7f2a6a-4c17-43a6-80bc-0bd47a99727f_2017-07-17-121412-269_v4.1.1

Restoring Cloudron

To restore from a backup:

  • Install Cloudron on a new server with Ubuntu 18.04:
  wget https://cloudron.io/cloudron-setup
  chmod +x cloudron-setup
  ./cloudron-setup --provider digitalocean --version x.y.z # version must match your backup version

Backup & Cloudron version

The Cloudron version and backup version must match for the restore to work. If you installed a wrong version by mistake, it's easiest to just start over with a fresh Ubuntu installation and re-install Cloudron.

  • Complete the DNS setup

  • Click on Looking to restore located at the bottom:

  • Provide the backup information to restore from:

    Warning

    When using the filesystem provider, ensure the backups are owned by the yellowtent user. Also, ensure that the backups are in the same file system location as the old Cloudron.

        root@intranet:~# ls -l /var/backups/
        total 8
        drwxr-xr-x 2 yellowtent yellowtent 4096 Sep 25 21:02 2017-09-25-210206-153
        drwxr-xr-x 2 yellowtent yellowtent 4096 Sep 25 21:02 2017-09-25-210210-192
  • Cloudron will download the backup and start restoring:

The new Cloudron server is an exact clone of the old one - all your users, groups, email, apps, DNS settings, backup settings, certificates, will be exactly as-is before.

Migrating Cloudron to another server

If the existing server's cpu/disk/memory can be resized (as is the case with most server providers), then simply resize it and reboot the server. Cloudron will automatically adapt to the available resources after a server resize.

To migrate to a different server or move Cloudron to a different server provider:

  • Take a complete backup of the existing Cloudron. Click the Backup now button in the Settings page and make a note of the backup id.

Backup location

We recommend backing up to an external service like S3 or Digital Ocean Spaces. This is because the backups become immediately available for the new server to restore from. If you use the filesystem for backups, you have to copy the backup files manually to the new server using rsync or scp.

  • Follow the steps to restore from the latest backup on the new server. It is recommended to not delete the old server until migration to new server is complete and you have verified that all data is intact.

Backup & Cloudron version

The Cloudron version and backup version must match for the restore to work. To install a specific version of Cloudron, pass the version option to cloudron-setup. For example, cloudron-setup --version 3.3.1.