Store backups in Amazon S3

This is how you set up a bucket at Amazon S3 as a target for your backups: from the access key with the necessary permissions through the connection test to the saved target.

What you need

• a bucket at Amazon S3 that already exists: the plugin does not create one,
• an access key and the secret key that belongs to it,
• the permissions for this key, see the next section.

For every other service that speaks the S3 interface there is a separate target type: Store backups in S3-compatible storage.

The permissions of the access key

Use a key that was created only for this backup and may reach only this bucket. For the user of the access key the policy at Amazon can look like this. Replace mein-backup-bucket with the name of your bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": "arn:aws:s3:::mein-backup-bucket"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": "arn:aws:s3:::mein-backup-bucket/*"
    }
  ]
}

If one of the permissions is missing, the connection test names it. Without the two permissions that list unfinished uploads and their parts (s3:ListBucketMultipartUploads and s3:ListMultipartUploadParts), backups still work; the connection test then warns.

Step 1: Choose the target type

Open the Targets page and click Add target. Choose Amazon S3.

1. The target types that exist. Amazon S3 carries the sign “aws”.
2. Dismiss closes the selection.

The plugin talks to Amazon S3 through the PHP extension cURL and reads the responses with the PHP extension DOM. If the web server lacks one of them, Amazon S3 is greyed out and carries the label Not available on this server. The reason is shown below the selection.

Step 2: Fill in the form

1. Name is up to you. The plugin suggests “Amazon S3 1”; any name that is not in use yet works.
2. Bucket is the name of the bucket in which the backups are to be stored.
3. Access key is visible in the form, like a user name.
4. Secret key is stored encrypted and never goes back to the browser.
5. Folder in the bucket may stay empty: then the folder of this website lies directly in the bucket.
6. Test connection checks the entered values without saving.

Fields with a red star are required.

Step 3: Test the connection

Click Test connection. The test writes a small file into the bucket, reads it back, lists it and deletes it again. In addition it uploads a test file of just over 5 MiB in two parts and measures the upload rate while doing so. It deletes this file again as well.

1. The result of the test: the connection works, a test file was written, read back and deleted.
2. A warning of the test. In the picture the bucket keeps versions.
3. The region of the bucket, the upload method and the measured upload rate.

If the test succeeded, the form shows these lines:

Line in the result	Meaning
Connection works: a test file was written, read back and deleted.	Writing, reading back, listing and deleting work.
Region of the bucket: …	The region that Amazon S3 names for the bucket. The plugin remembers it at the target.
Upload method: in parts (multipart upload); an upload continues with the next part.	S3 cannot continue a file. That is why the plugin uploads every file in parts.
Measured upload rate: … per second.	Based on the rate the plugin chooses the size of the parts.

Below the first line, warnings are shown in yellow if the test noticed something. The section “Warnings after a successful test” explains them.

Step 4: Save

Click Save. The plugin saves the target, tests the connection once more and reports the result: “…” saved and connected. Plans and backups use a target only after a successful connection test.

1. All targets are connected.
2. The message names the target: saved and connected.
3. The new target is in the list. The green dot means: connected. A target of the type Amazon S3 is preceded by the sign “aws”.

If the plugin finds backups in the bucket that are not in your list, the tile Backups found at “…” appears. More about this: Find backups again after a total loss.

Step 5: Choose the target in the backup plan

• Your first target: The plugin creates the plan “Daily backup”: daily at a random time, everything, this target, Max. backups 7. The plan is paused until you switch it on.
• Another target: Choose it on the Backup page in the tile Backup plans in a plan. The free version allows two plans and one target per plan.

More about this: Schedule backups automatically.

The fields of the Extended view

1. Region may stay empty: then the connection test asks Amazon S3 for the region of the bucket. An example of a region is eu-central-1.
2. Connection timeout (seconds) takes 3 to 60, the default is 10.
3. Upload rate (bytes per second) may stay empty: then what the connection test measured applies.

An empty field whose value the test determines shows automatic. Every connection test determines region and rate anew. An upload rate you enter yourself takes precedence. If Amazon S3 names a different region than the one you entered, the plugin works with the region of the service, and the connection test warns.

Warnings after a successful test

Warning	What you can do
The service names another region than the one you entered. The plugin uses the region of the service; correct the field Region.	In the Extended view, enter the region that the result names, or clear the field Region.
This bucket keeps versions: a deleted backup stays there as an old version and keeps taking space. Set a lifecycle rule at the service that removes old versions.	Set up the rule at Amazon S3. Deleting by the plugin only hides a backup in such a bucket.
The service does not list unfinished uploads, so the plugin cannot clean them up there. Set a lifecycle rule at the service that removes unfinished uploads.	Give the key the permission s3:ListBucketMultipartUploads, or set up the rule.
The service does not let the plugin list the parts of an upload. Uploads work, but the plugin cannot check them before it continues.	Give the key the permission s3:ListMultipartUploadParts.
The clock of this web server differs from the clock of the service by about … minutes. The plugin works with the time of the service; have the clock of the web server corrected.	Ask your hosting provider to correct the clock of the web server. Backups keep running until then.

When the test fails

If the test fails, the result is shown below the fields of the form. In the picture the access key lacks the permission to write objects.

1. The cause in plain language. The link at the end, Show log, opens the general log. The details of the test are there.
2. The permission that the access key lacks.
3. Message from the target is the original error message of the service.

Line 2 is shown only below some causes and names the reason more precisely, for example the missing permission. If cURL reported something too, the line Message from cURL is shown below Message from the target. The error messages in these two lines are not translated. The bucket, the access key and the server name of the service are masked in them; the secret key is never sent.

Message	What you can do
The target cannot be reached. Check the server name and the port; the host of this website may block the port.	The plugin builds the server name of the service from the region. Check the field Region in the Extended view, or clear it. If the region is right, ask your hosting provider whether the web server may open outgoing connections.
The service refused the keys.	Below it you see which key is wrong: The service does not know this access key. Or: The signature of the request was refused: the secret key does not belong to this access key. Enter the key again.
The access key lacks a permission that the plugin needs.	Below it the permission is named: Missing permission: …. Add it to the policy of the key.
The bucket does not exist at this service. The plugin does not create buckets.	Check the name in the field Bucket, or create the bucket at Amazon S3.
The bucket lies in another region than the one that was used.	Below it the region of the bucket is named. Enter it in the field Region, or clear the field.
The service refuses the request because the clock of this web server is off by more than the service allows.	Ask your hosting provider to correct the clock of the web server.
An upload did not arrive before the time of a step ran out: the connection to the target is too slow for it.	Below it the reason is named: This service takes an upload only in parts of at least 5 MiB, and such a part has to arrive within one step. How long a step takes at most is set by the setting Maximum time per step: Adapt the plugin to your server.
This value is not valid. at the field Bucket	The name of a bucket at Amazon S3 has 3 to 63 characters: lowercase letters, digits, dots and hyphens. It begins and ends with a letter or a digit. The form does not accept two dots in a row or the form of an IP address.

Unfinished uploads and versions

• Unfinished uploads. An upload that was never completed keeps taking space at the service. At Amazon S3 it never expires on its own. The plugin aborts its own unfinished uploads, and cleaning up leftovers aborts the ones it finds. Nevertheless, set a lifecycle rule at Amazon S3 that removes unfinished uploads after a few days.
• Versions. If the bucket keeps versions, deleting only hides a backup: the old versions stay and keep taking space. Set a lifecycle rule that removes old versions.

Good to know

• The backups are stored in the bucket under <folder>/cloneworx-backup-<code>/<backup>/.
• A part of an upload is at least 5 MiB in size, only the last one may be smaller. A file appears in the bucket only when its upload is complete.
• The plugin places no protection files in a bucket: a bucket is not served by a web server that would read them.
• The plugin never follows a redirect.
• Backups that are stored only in a bucket you do not download through the plugin, but with a program for S3 or in the web interface of the service. Restore, check and finding backups read directly from the service.
• The plugin also uses a proxy from the wp-config.php for Amazon S3.
• Below the selection of the target types the note reads: Names and logos of third parties are the property of their respective owners. This does not imply any affiliation or partnership.

See also

• Which target suits you?
• Store backups in S3-compatible storage
• The “Targets” page
• Renew the credentials of a target
• Check backups at a target and clean up leftovers

On the command line

WP-CLI not set up yet? How to install WP-CLI.

The command line does not accept the secret key; that is why you set up a target of the type Amazon S3 on the Targets page. Everything else also works with WP-CLI:

# Show targets with ID and state
wp cloneworx-backup target list

# Test the connection
wp cloneworx-backup target test amazon-s3-1

# Create a backup plan that backs up to this target
wp cloneworx-backup plan add --targets=amazon-s3-1 --rhythm=daily --time=03:15 --name="Nachts"

# Check the backups at this target
wp cloneworx-backup target check amazon-s3-1

# Show leftovers and unfinished uploads (--dry-run) and clean up
wp cloneworx-backup target cleanup amazon-s3-1 --dry-run
wp cloneworx-backup target cleanup amazon-s3-1