Migrating Between Amazon Cloud Drives

This is a work in progress and may be updated.

I want to transfer a rather large amount of data between two different Amazon Cloud Drives for, ahem, reasons.

Get the rclone beta, I’m using v 1.36.

Authorize with the proxy server. Enter your creds on this site at your own risk.

Do this locally on your Mac/PC, once for each account, then copy the resulting config file ~/.rclone.conf to a server in the cloud.

ACD doesn’t support cloud-to-cloud copying, so you’ll have to download and re-upload everything. I’ll be doing this on a box with a 1 Gbit/s line.

These command options appear to work for me:

./rclone -vv --buffer-size=10G --acd-templink-threshold 0 --transfers 10 sync acd1:folder/ acd2:folder/

Some of the files appear to be in cold storage and need a couple of minutes to unfreeze, but rclone handles this just fine. In this first batch of transfers, rclone uses about 3GB of memory.

Larger files naturally transfer with less overhead and saturate the line at 700-900 Mbit/s. Smaller files slow down the process significantly, but running a copy of rclone for every top level folder with –transfers with a value of 10-30 each doesn’t appear to get rate limited too much, and with over 100 connections, I also get around 700Mbit/s for loads and loads of smaller files.

Update: Finished after a total of 3 days. Did a second and third run to make sure at least all file sizes match. Number of files and storage space reported by “Manage Storage” match 100%.