Issue syncing files within structured folder on external drive

Troubleshooting
,
1

I was using odrive + amazon to store my files on my hard drive alongside video files from work on an external drive. To save space on the drive due to the high volume of video files, I was unsyncing video files on the drive…I am moving over to Dropbox Business thanks to Amazon’s elimination of unlimited data and need assistance because I need to download over a thousand videos within these various folders and can’t seem to do that by using SYNC. I can attach a screenshot. But an example:

Folder being synced: @footage

@footage > clients > joe.smith > 7.1.2017 > movie1.mp4.cloud movie2.mp4.cloud movie3.mp4.cloud movie4.mp4.cloud

As you can imagine, having over 50 clients and hundreds of videos makes it extremely tedious to click each file and have it re download - odrive has crashed several times. Someone please advise, my Amazon account expires in about a month and I’ve moved everything else over from amazon and I can’t seem to resolve this. Yes, I’ve tried syncing @footage and setting it to download EVERYTHING, have tried this for CLIENTS folder and even the individual user’s folder…and even doing so with, for example, the folder 7.1.2017 and all the others. This is a real problem,

2

Hi @ryan1,
The issue is that you need to download all of the data in @footage, but odrive is aborting or closing before it can finish? Is that correct?

If so, we can try a different approach and use the CLI to manage the downloads. It will essentially sync that folder until it is done.

To do this follow these steps:

Open up a command prompt by clicking on the Windows icon in the taskbar, typing “cmd”, and then clicking on “Command Prompt”.

389×622 9.3 KB

Once the command prompt is open, copy and paste the following command in (all one line) and hit enter. This will install the CLI for us to use in the next command:

powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";(New-Object System.Net.WebClient).DownloadFile(\"https://dl.odrive.com/odrivecli-win\", \"$comm_bin\oc.zip\");$shl=new-object -com shell.application; $shl.namespace(\"$comm_bin\").copyhere($shl.namespace(\"$comm_bin\oc.zip\").items(),0x10);del \"$comm_bin\oc.zip\";}"

Once the CLI is finished installing (it could take a minute or two), copy and paste this next command to perform a recursive sync of a folder. This command will continue retrying any downloads that fail until everything is downloaded. This is another long one-liner, so make sure you copy the whole thing:

powershell -command "& {$syncpath=\"$HOME\odrive\[Folder to Sync Here]\";$syncbin=\"$HOME\.odrive\common\bin\odrive.exe\";while ((Get-ChildItem $syncpath -Filter \"*.cloud*\" -Recurse | Measure-Object).Count){Get-ChildItem -Path \"$syncpath\" -Filter \"*.cloud*\" -Recurse | % { & \"$syncbin\" \"sync\" \"$($_.FullName)\";}}}"

For the above command, be sure you replace [Folder to Sync Here] with the folder you want to recursively sync (Amazon Drive@footage in your case, for example).

You can actually run multiple instances of this script to create parallelism by just opening additional cmd prompts and pasting the command in. You may see some messages about files not longer being available, and this is just because another script instance got to it first, but its not a problem.

3

Using a mac - can I get alternative instructions? Again, need to download all this to an external drive as it’s too much data for my local hard drive (just in case this isn’t factored in).

4

Hi @ryan1,
Sure. It sounds like you are a premium subscriber of odrive, correct? This means you can setup a “sync to odrive” folder on the external drive and point it to the @footage folder on the remote storage. So the steps would be like this:

Create a folder called “@footage” on your external drive. Right-click on that empty folder and select “sync to odrive”. A browser window will pop-up and then you can navigate to the “@footage” folder on Amazon and select it for the sync relationship. After setting this up you should see the cloud files appear in the “@footage” folder on the external drive.

Note: It is important that the local and remote folder names match.

5

To use the CLI commands from Mac:

  1. Open a terminal session (type “terminal” in Spotlight search):

  2. Run the following command in the terminal session (copy & paste + Enter):

exec 6>&1;num_procs=2;output="go"; while [ "$output" ]; do output=$(find "/Volumes/ExtDrive/@footage" -name "*.cloud*" -print0 | xargs -0 -n 1 -P $num_procs python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py sync | tee /dev/fd/6); done

Change “/Volumes/ExtDrive/@footage” to the proper path of your “sync to odrive” folder.
It is an ugly one-liner, but the above command will download everything in the remote folder using 2 concurrent workers. It won’t stop until everything has been downloaded.

6

I was - my subscription just expired and I have no reason to continue using odrive + amazon as dropbox has its own sync tool. Will this still be viable with the first set of instructions you gave me regarding creating a folder, syncing, etc?

7

It appears that once I hit sync and set it to sync all subfolders and files it’s still pulling files down as .cloud

8

Ran command - getting “this file is not inside an odrive folder” - this is just after I followed these instructions:

"Sure. It sounds like you are a premium subscriber of odrive, correct? This means you can setup a “sync to odrive” folder on the external drive and point it to the @footage folder on the remote storage. So the steps would be like this:

Create a folder called “@footage” on your external drive. Right-click on that empty folder and select “sync to odrive”. A browser window will pop-up and then you can navigate to the “@footage” folder on Amazon and select it for the sync relationship. After setting this up you should see the cloud files appear in the “@footage” folder on the external drive.

Note: It is important that the local and remote folder names match."

and then…

exec 6>&1;num_procs=2;output=“go”; while [ “$output” ]; do output=$(find “//Volumes/FOOTAGE/@FOOTAGE” -name “.cloud” -print0 | xargs -0 -n 1 -P $num_procs python $(ls -d “$HOME/.odrive/bin/”*/ | tail -1)odrive.py sync | tee /dev/fd/6); done

9

Hi @ryan1,
If you do not have a Premium subscription then you will not be able to use the “sync to odrive” feature to sync an external drive folder.

You can still use the command, but it will have to target a folder inside the default odrive folder and not an external folder. So, for example:
"$HOME/odrive/Amazon Cloud Drive/@footage"

10

Is there anything that can be done? I really can’t justify $99 to pull down some files.

11

Hi @ryan1,
I suppose you could close your current odrive account and create a new one, which would put you into the trial.

12

Did that - getting the same message when it runs through the folder and everything is .cloud

13

Did you deathorize the desktop client (odrive menu->authorized user->deauthorize) and then login with the new account? If so, can you send a diagnostic from the menu please?

14

Yes and done, please advise.

15

The diagnostic looks okay to me. It appears that things are downloading.

Are you using the script or the right-click->sync functionality?

In either case you will see initial placeholders until odrive can get to them to download. You can take a look at the sync status in the odrive tray menu to view the current downloads that are in progress.

16

The windows script is working for me to bring down my ACD, and unstuck some folders where the context menu was missing.

1 Like