Skip to content

Backups

How Cloudron backups work

Cloudron automatically creates a complete backup of the server every day. Each app is backed up independently to files with prefix app_<appid>. The platform data (users, groups, email, cloudron configuration) is backed up independently to files with the prefix box_.

Unlike VM snapshots, these backups contain only the necessary information to complete re-instate the Cloudron or app. For example, application code or libraries are not part of a backup since these are read-only and can never change. Runtime files (lock files, logs) or temporary files generated by apps are not backed up either. Only the database and app user data is backed up. This design significantly reduces the size of backups.

Backups are designed for portability. App backups can be used to create a clone of the app or even move the app to another Cloudron. Platform and app backups together can also be used to move the Cloudron in it's entirety to another server easily.

By default, backups reside in /var/backups. Please note that having backups reside in the same physical machine as the Cloudron server instance is dangerous and it must be changed to an external storage location like S3 as soon as possible.

Backup formats

Cloudron supports two backup formats - tgz (default) and rsync. The tgz format stores all the backup information in a single tarball whereas the rsync format stores all backup information as files inside a directory.

Both formats have the same content

The contents of the tgz file when extracted to disk will be the exact same as the contents of the rsync directory.

Both the formats are complete and portable. They can be cloned and restored within the same cloudron or across cloudrons. If you are looking to archive backups, simply keep a copy of the .tar.gz file in the case of tgz backups or the entire directory in the case of rsync backups.

The backup format is set globally and applies to all apps and the Cloudron data.

The formats only differ on how the final backup blobs are created and uploaded. We recommend using tgz format unless you have app(s) that create large amounts of data (say > 5GB).

tgz format

The tgz format uploads an app's backup as a gzipped tarball. This format is very efficient when having a small number of files. This is the case for most apps that use the database to store all information and the database dump file is just a single file on the filesystem. This format also works well when there are a very large number of small files.

This format has the following caveats:

  • Most Cloud storage API require the content length to be known in advance before uploading data. This means that the tar.gz has to be buffered completely in disk (doubling the disk space requirement). Cloudron does upload big tgz backups in chunks. However, chunked (multi-part) uploads cannot be parallelized and also take up as much RAM as the chunk size.

  • tgz backup uploads are not incremental. This means that if an app generated 10GB of data, Cloudron has to upload 10GB every time it makes a new backup.

  • Encrpytion is currently only supported with the tgz format.

rsync format

The rsync format uploads individual files to the backup storage. It keeps track of what was copied the last time around, detects what changed locally and uploads only the changed files on every backup. Note that despite uploading 'incrementally', tgz format can be significantly faster when uploading a large number of small files (like source code repositories) because of the large number HTTP requests that need to be made for each file.

This format has the following caveats:

  • By tracking the files that were uploaded the last time around, Cloudron minimizes uploads when using the rsync format. To make sure that each backup directory is "self contained" (i.e can be simply copied without additional tools), Cloudron issues a 'remote copy' request for each file.

  • File uploads and remote copies are parallelized.

  • When using the file system backend, the rsync format can hardlink 'same' files across backups to conserve space. If you happen to use a file system that does not support hardlinks, just turn off the hardlinks option.

  • When encryption is enabled, file names are encrypted as well.

Making an app backup

To take a backup of a single, click the Backups button in the app grid:

Then click Create Backup:

Making a complete backup

To take a backup of Cloudron and all the apps, click the Backup now button in the Settings page:

Warning

When using the no-op backend, no backups are taken. If backups are stored on the same server, be sure to download them before making changes in the server.

Disabling automatic backups for a specific app

Cloudron makes a complete backup every day. When using apps that contain a large number of files (like NextCloud, ownCloud) the backup storage may quickly add up. An app can be excluded from daily automatic backups from the 'Advanced settings' in the Configure UI:

Note that the Cloudron will still create backups before an app or Cloudron update. This is required so that it can be reverted to a sane state should the update fail.

Warning

Disabling automatic backup for an app puts the onus on the Cloudron adminstrator to backup the app's files regularly. This can be done using the Cloudron CLI tool's cloudron backup create command.

Storage provider

Amazon S3

To store backups on Amazon S3, use the S3 provider.

When creating the bucket in S3, the bucket can be setup to periodically delete old backups by adding a lifecycle rule using the AWS console. S3 supports both permanent deletion or moving objects to the cheaper Glacier storage class based on an age attribute. With the current daily backup schedule a setting of two days should be sufficient for most use-cases.

When using root credentials on AWS, follow the instructions here to create access keys.

When using IAM, follow the instructions here to create a user and use the following policy to give the user access to the bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::<your bucket name>",
                "arn:aws:s3:::<your bucket name>/*"
            ]
        }
    ]
}

The Encryption key is an arbitrary passphrase used to encrypt the backups. Keep the passphrase safe; it is required to decrypt the backups when restoring the Cloudron.

DigitalOcean Spaces

To store backups on DigitalOcean Spaces, use the DigitalOcean Spaces provider.



A heads up about using DO Spaces. In our tests, we hit a few issues including missing implementation for copying large files (> 5GB), severe rate limits and very poor performance when deleting objects. If you plan on using this provider, keep an eye on your backups. In any case, Cloudron will notify the admins by email when backups fail.

Exoscale SOS

Exoscale SOS is a simple, scalable and safe S3-compatible object store based on pithos.



Filesystem

To store backups on the filesystem, use the Filesystem provider.

The Use hardlinks option can be checked to make the Cloudron use hardlinks 'same' files across backups to conserve space. If you happen to use a file system that does not support hardlinks (like NFS), just turn off hardlinks.

Warning

Having backups reside in the same physical machine as the Cloudron server instance is dangerous. For this reason, Cloudron will show a warning until the Backup directory is an external EXT4 Disk option is checked.

Google Cloud Storage

Provide GCS backup credentials in the Backups section of the Settings page.

Minio

Minio is a distributed object storage server, providing the same API as Amazon S3.

Minio can be setup, by following the installation instructions on any server, which is reachable by the Cloudron.

Be sure to install Minio on another server

Do not setup Minio on the same server as the Cloudron! Using the same server will inevitably result in data loss if something goes wrong with the server's disk.

Once setup, minio will print the necessary information, like login credentials, region and endpoints in it's logs.

$ ./minio server ./storage

Endpoint:  http://192.168.10.113:9000  http://127.0.0.1:9000
AccessKey: GFAWYNJEY7PUSLTHYHT6
SecretKey: /fEWk66E7GsPnzE1gohqKDovaytLcxhr0tNWnv3U
Region:    us-east-1

The credentials can now be used to setup Minio backups:

The Bucket must exist and can be created using Minio CLI or the web interface. For HTTPS installations using a self-signed certificate, select the Accept Self-Signed certificate option.

The Encryption key is an arbitrary passphrase used to encrypt the backups. Keep the passphrase safe; it is required to decrypt the backups when restoring the Cloudron.

Encrypting backups

Backups can be encrypted by providing an encyption key in the backup settings.

Please note that when using encryption with the rsync format, the file names are encrypted as well.

Do not lose encryption key

Please keep the encryption key safe. Without the key, there is no way to decrypt the backups.

Decrypting backups

When encryption is used, backups can be downloaded and decrypted locally with the following command:

cat backupfile.tar.gz.enc | openssl aes-256-cbc -d -nosalt -pass "pass:secretkey" > backupfile.tar.gz

secretkey is the encryption key that was used for creating backups

Backup interval

Cloudron automatically creates a backup every night (in the timezone of your Cloudron).

To instantly make a complete backup, click the Backup now button in the Backups section of the Settings page. Alternately, use the CLI command cloudron machine backup create.

Debugging backups

The backup logs can be viewed and downloaded using the Show Logs button in the Backups view.

Migrate Apps from one Cloudron to another

User Identifiers

When moving an app from one Cloudron to another, the user ids on one Cloudron will be different from the user ids on the other Cloudron. Cloudron does not support automatic user id mapping at this point. For this reason, you might have to manually fix up the database of the new app in the new Cloudron. A workaround is to install apps without using Cloudron Single Sign-On (for apps that support it).

Migrating apps or moving apps from one Cloudron to another works by first creating a new backup of the app on the old Cloudron, copying the backup tarball onto the new Cloudron's backup storage and then installing a new app, based on the backup on the new Cloudron.

Both Cloudrons have to use the same backup encryption key!

The following step will migrate an app:

  • Against the old Cloudron
cloudron backup create --app <subdomain/appid>
  • Copy the new backup from the old Cloudron's backup storage to the new one. This can be done via the s3 webinterface or scp if the filesystem provider is used. The backup can be located by its backup ID, which can be seen with:
cloudron backup list --app <subdomain/appid>
  • Make note of the app's appstore id and version from:
cloudron list
  • Then login to the new Cloudron and install the new app based on the created backup:
cloudron login my.<new Cloudron domain>
cloudron install --appstore-id <apps appstore id>@<specific version if required> --backup <backupId>

Note

The backupId usually also includes a path prefix and looks like and must not contain the file extension (eg. .tar.gz): 2017-07-17-121412-248/app_2d7f2a6a-4c17-43a6-80bc-0bd47a99727f_2017-07-17-121412-269_v4.1.1

Cloning a Cloudron app in same Cloudron

To clone an app i.e an exact replica of the app onto another domain, first create an app backup and then use the Clone UI to create a clone:

Cloning a Cloudron app into another Cloudron

Cloudron app backups are essentially 'snapshots' of the app and are portable across Cloudrons. To move an app from one Cloudron to another or to instantiate an app from an arbitrary backup perform the following steps:

  1. Make the app backup available in the destination Cloudron's backup directory.

  2. Install the app using the following command:

    cloudron install --backup <id> --appstore-id org.wordpress.cloudronapp@0.9.1

Restoring an app from existing backup

Apps can be restored to a previous backup by clicking on the Backups button.

Select the backup you want to restore to:

Both data and code are reverted

Restoring will also revert the code to the version that was running when the backup was created. This is because the current version of the app may not be able to handle old data.

Restoring Cloudron from a backup

Cloudron 1.9 introduced a UI to restore Cloudron. To restore from a backup:

  • Install Cloudron on a new server with Ubuntu 16.04:
  wget https://cloudron.io/cloudron-setup
  chmod +x cloudron-setup
  ./cloudron-setup --provider digitalocean # change this to your new server provider
  • Complete the DNS setup

  • Click on Looking to restore located at the bottom:

  • Provide the backup information to restore from:

    Warning

    When using the filesystem provider, ensure the backups are owned by the yellowtent user. Also, ensure that the backups are in the same file system location as the old Cloudron.

        root@intranet:~# ls -l /var/backups/
        total 8
        drwxr-xr-x 2 yellowtent yellowtent 4096 Sep 25 21:02 2017-09-25-210206-153
        drwxr-xr-x 2 yellowtent yellowtent 4096 Sep 25 21:02 2017-09-25-210210-192
  • Cloudron will download the backup and start restoring:

The new Cloudron server is an exact clone of the old one - all your users, groups, email, apps, DNS settings, backup settings, certificates, will be exactly as-is before.

Migrating Cloudron to another server

If the existing server's cpu/disk/memory can be resized (as is the case with most server providers), then simply resize it and reboot the server. Cloudron will automatically adapt to the available resources after a server resize.

To migrate to a different server or move Cloudron to a different server provider:

  • Take a complete backup of the existing Cloudron. Click the Backup now button in the Settings page and make a note of the backup id.