How can I force a recursive download with the CLI?

faq

#1

How can a recursive download be forced to continue, despite the storage exceptions that odrive may encounter?


All Amazon Drive files have suddenly been unsynced
CLI Syncing Subfolders
"Error updating the file: Size Mismatch" syncing images from Slack
Syncing all files
Synch stops with blue icon but folders/files not downloaded from onedrive
#2

For bulk downloads, if odrive encounters numerous exceptions, or certain types of “critical” errors, it will abort the action instead of just blindly retrying to infinity. This can be frustrating if a user wants to download lots and lots of content, but the storage service is returning frequent errors.

For scenarios like these, it is possible to use the CLI to essentially perform a brute-force “download until done” action. It is not elegant, but it can be useful for some folks.

To use the CLI commands on MacOS:

  1. Open a terminal session (type “terminal” in Spotlight search):

  2. Run the following command in the terminal session (copy & paste + Enter):

exec 6>&1;num_procs=3;output="go"; while [ "$output" ]; do output=$(find "[path to folder here]" -name "*.cloud*" -print0 | xargs -0 -n 1 -P $num_procs python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py sync | tee /dev/fd/6); done

For the above command, be sure you replace “[path to folder here]” with the path of the folder you want to start the download from.
It is an ugly one-liner, but the above command will download everything in the folder using 3 concurrent workers. It won’t stop until everything has been downloaded.
This text will be hidden

To use the CLI commands on Windows:

Open up a command prompt by clicking on the Windows icon in the taskbar, typing “cmd”, and then clicking on “Command Prompt”.

Once the command prompt is open, copy and paste the following command in (all one line) and hit enter. This will install the CLI for us to use in the next command:

powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";(New-Object System.Net.WebClient).DownloadFile(\"https://dl.odrive.com/odrivecli-win\", \"$comm_bin\oc.zip\");$shl=new-object -com shell.application; $shl.namespace(\"$comm_bin\").copyhere($shl.namespace(\"$comm_bin\oc.zip\").items(),0x10);del \"$comm_bin\oc.zip\";}"

Once the CLI is finished installing (it could take a minute or two), copy and paste this next command to perform a recursive sync of a folder. This command will download one file at a time until everything is downloaded. This is another long one-liner, so make sure you copy the whole thing:

powershell -command "& {$syncpath=\"[path to folder here]\";$syncbin=\"$HOME\.odrive\common\bin\odrive.exe\";while ((Get-ChildItem $syncpath -Filter \"*.cloud*\" -Recurse | Measure-Object).Count){Get-ChildItem -Path \"$syncpath\" -Filter \"*.cloud*\" -Recurse | % { & \"$syncbin\" \"sync\" \"$($_.FullName)\";}}}"

For the above command, be sure you replace “[path to folder here]” with the path of the folder you want to start the download from.

You can actually run multiple instances of this script to create parallelism by just opening additional cmd prompts and pasting the command in. You may see some messages about files no longer being available, and this is just because another script instance got to it first, but its not a problem. You can essentially “ramp up” or “ramp down” the download speed in this manner, at least to a certain extent.


Files & Folders in Amazon Drive not appearing on local folder
CLI Syncing Subfolders
I want to stop using Amazon Cloud Drive, how do I download all .cloud files
Auto Download not working?
Amazon Cloud Cam Clips
"Amazon Cloud Drive internal server error." for more than 50% of my syncs
Intermittent unexpected errors when syncing