How do I Move or Migrate files from one storage provider to another?

How do I move of migrate my file from one storage provider to another?

3 Likes

Currently odrive cannot facilitate moving files between remote storage, directly, but it can still make movement/migration easy.

Once you have logged in and linked your desired storage providers at https://www.odrive.com/ you can download and install the desktop client.

In this example I will copy files from OneDrive to Amazon Cloud Drive using the odrive desktop client. For step-by-step example using the CLI/agent (for example, to use cloud infrastructure to facilitate a storage migration), take a look at this post here:

To copy your files and/or folders from one cloud storage provider to another you must first synchronize all the files from the source.

  • To do this right click on the source cloud folder, select Sync, then be sure to set the slider to Everything, and check Include subfolders then click on the arrow button.

  • Ensure all your files are downloaded to your computer (identified by a blue badge on all files and folders). If you are moving large files or folders, you can just do one at a time depending on available hard drive space.

  • Now simply copy all the files and/or folders from source

  • and paste them in the new target storage location (Amazon Cloud drive for example)

  • Once again when all the newly copied files and folders have the blue badging that indicates the files have been successfully uploaded to your alternate cloud storage host.

  • You can now unsync (right click > Unsync) the source folder, e.g. OneDrive as well as any other files or folders in any provider you no longer need stored locally to recover any necessary disk space, and if necessary once you are comfortable that all files have been migrated you can delete the file(s) and/or folders from the source if desired.

1 Like

Can you guys create a function to move files between drives online?
If I have a TB of data to transfer that is going to kill my download for the month.

There is a feature request for this here: Moving, Copying and Auto-Syncing Files Between Services (Clouds)

One possible workaround for files not synced on your local machine is to use a cloud instance, which usually has screaming download and upload speeds (in the order of 100’s MB/s). In AWS, Google, Azure, etc. create a Windows instance, which you can access via RDP. Once spun up, you can install the desktop client, move files from storage provider A to storage provider B without burdening your machine or bandwidth. Once done make sure to shutdown and delete the instance, and you’re done.

For files that are locally synced, you will have to re-download them from the new provider to your machine.

In terms of specific Cloud instances:

  • I have found Google cloud to require the least upfront overhead, and it does offer a $300 starting credit, which I cannot imagine will not support a full transfer.
  • AWS is pretty powerful, but requires one to go through the full IAM exercise, including RSS keys and what not, so probably too much overhead if you are going to use it for transfers only.
3 Likes

I migrated from Amazon Cloud Drive (now branded as Amazon Photos) to Google Drive today. Even with relatively fast home internet bandwidth downloading and uploading 1TB of data is time consuming. Here is how I did it

  1. Spin up a bare metal Windows Server on your favourite hosting service. I used packet.net
  2. Login to Amazon Cloud, download and install their photos App.
  3. Use the app to restore all your Amazon cloud folders to the local drive.
  4. Login to Google Drive and upload your data
  5. Unsync your odrive folders from Amazon Drive
  6. Resync your odrive folders to Google Drive

This worked great for me and I migrated the data in hours instead of days at a cost of about $30.

2 Likes

These are some good pointers.

I am having trouble downloading all my files on to my limited-storage local drive to switch cloud providers.

If all my files were unsynced on my local machine, would I be able to sync all my files onto a windows machine in the cloud using the odrive client and switch cloud providers? Can the odrive employees or users provide support with step by step procedures?

Thanks.

Hi @digiform,
Yes you could do this. The procedure would be similar to what @paj posted above.

I’ve done this a few times, as well, and can provide the steps I took.

  • I’ve used a few different types of services, including Amazon Lightsail, Amazon EC2, and DigitalOcean. They are all going to be the same process, so it is just going to depend on your needs in terms of storage and pricing. The storage will be the most important, as you will want enough storage to perform the migration.

  • On the service you choose, spin-up a Linux instance. I usually go for the latest Ubuntu LTS instances they offer (probably 18.04 right now).

  • Install the Linux odrive sync agent (64 bit) and CLI: https://docs.odrive.com/docs/odrive-sync-agent

  • Setup odrive and create the mount: https://docs.odrive.com/docs/odrive-sync-agent#section-using-odrive-sync-agent

  • Sync (expand) the root-level folder for the storage you want to download. For example:

    "$HOME/.odrive-agent/bin/odrive" sync "$HOME/odrive-agent-mount/Dropbox.cloudf" This will expand the Dropbox folder in my odrive.

  • Recursively sync the expanded Dropbox folder, downloading everything in it until there are no more placeholder files. The command for this example would be:

    exec 6>&1;num_procs=4;output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -print0 | xargs -0 -n 1 -P $num_procs "$HOME/.odrive-agent/bin/odrive" sync | tee /dev/fd/6); done

    More details on what this command is doing here, if you are curious: Odrive Sync Agent: A CLI/scriptable interface for odrive's Progressive Sync Engine for Linux, OS X, and Windows

  • Once the download has completed, you will want to expand the destination storage folder to the appropriate location and then move the downloaded data from the source folder into the destination folder. For example:

    • "$HOME/.odrive-agent/bin/odrive" sync "$HOME/odrive-agent-mount/Google Drive.cloudf"

    • "$HOME/.odrive-agent/bin/odrive" sync "$HOME/odrive-agent-mount/Google Drive/Destination folder for migration.cloudf"

    • mv "$HOME/odrive-agent-mount/Dropbox/*" "$HOME/odrive-agent-mount/Google Drive/Destination folder for migration/"

  • Once the move is complete, perform a refresh command on that folder to kick-off an immediate odrive scan in that location, which will start the upload process. For example:

    "$HOME/.odrive-agent/bin/odrive" refresh "$HOME/odrive-agent-mount/Google Drive/Destination folder for migration"

  • At this point the data will be uploading to your new destination. You then need to decide what to do with the data in your old storage. I would suggest leaving it there until all of the data has been uploaded to the new location and you are satisfied with the migration. If you plan to delete the data, it will probably be more efficient to do so from the storage’s web client, directly, than through the odrive client’s “emptytrash” comand.