Recursive folder sync

[subhashishkar]

Hi,

I have a folder which has several sub folders and sub-sub folders and so on. The total number of sub-directories (all inclusive) inside that main parent folder is close to 7,000 with nearly 50,000 files. I am having a hard time syncing this directory. First off it takes a very long time (even though the size of the directory is not that big and I have a 40 upload/download speed) but what is more annoying is that in the end it shows the folder is synced (blue check on the folder icon). However, on opening the folder, I see tons of unsynced folders and files.

Is there a ply depth where odrive stops syncing after a fixed number of sub folder levels?

Why does odrive show a check mark (indicating all sub folders and files in it are synced) when there are unsyced files and folders inside?

[Tony]

Hi @subhashishkar,
Currently long-running, bulk download operations, like the one you are attempting, can hit cloud service exceptions (api rate limit, network connection issue, etc.) that can abort the operation. As long as there are no “dirty” files (files that need to be uploaded) in the folder structure, the folder will be badged with a green checkmark. This behavior is a byproduct of odrive’s ability to have both downloaded files and placeholder files in the same folder. A folder can be considered “in sync” even if not all of the files inside have been downloaded.

This behavior can certainly seem confusing if you don’t expect it. We are planning on adjusting this in an upcoming release to try to avoid this type of confusion and make it more clear what the state of the folder is.

Additionally, we are working on better tolerating transient download errors so that the bulk operations are not aborted when encountering an exception. The ideal behavior is that the action you specified (a recursive download of an entire folder) is completed if at all possible.

In the meantime we can work around this behavior by using the CLI.

If you are running Windows you can take a look at this post here:

Big Folder not syncing - The file doesn't exist

Hi @jmtudor, I took a look and there are several requests within that structure that are getting network timeouts. Basically we are asking OneDrive for the file and OneDrive is not responding. I noticed that every error is to the same server (5wpnsw-bn1306.files.1drv.com), so it is possible that requests to this particular node in their infrastructure was/is having problems. We are currently working on ways to better handle exceptions during a bulk operation, like a recursive sync-all on a lar…

If you are on a Mac, take a look here:

Keep content on local drive and in the cloud with Free odrive

Hi @gmreeg62610, How far did it get? It is possible it started hitting exceptions that caused it to abort. This was discussed a bit in this thread here, which I think parallels what you may be hitting: https://forum.odrive.com/t/big-folder-not-syncing-the-file-doesnt-exist/1616/27 I think you are using OS X, so we can apply a similar tactic and use the CLI to essentially “download until done” using this command: exec 6>&1;num_procs=4;output=“go”; while [ “$output” ]; do output=$(find "$HOME/…

[subhashishkar]

I tried using the script your provided. It launches a powershell window for a second and instantly closes. I do not get the dialog prompt either as shown in that thread. Am I missing something?

I am using win 10.

[Tony]

Hmm, not sure. I just ran it on Windows 10 and it ran as expected.

Try opening a command prompt and running it from there. It must be hitting an error somewhere on your machine.