This is relation to the Amazon crash and how Odrive is no longer syncing.
I am Mac OS 10.11.6, with all of the files associated with Odrive on an eSATA RAID. After the Amazon hickup, all of my local files are now placeholders. I can sync the files one by one with “Download nothing” option picked, and they show up again not no problem. So at least I know all of my files are here an don’t need to be downloaded again. However, every attempt to have odrive sync the files in a folder fails. It says it’s synced the folder, but none of the files inside are showing up, just the placeholder icon. I have thousands of files like this and obviosly can’t spend several days fixing this one by one. This is keeping Lightroom from working properly, and is seriously slowing down my business. Please advise.
Hi @r_speth,
Can you walk me through how you are trying to perform the recursive sync on the folder? Here is a reference from our documentation: https://docs.odrive.com/docs/sync-your-odrive#section--sync-all-files-and-subfolders-
If the recursive sync is not working, can you send a diagnostic from the odrive menu?
Diagnostic file just sent.
I’m clickon on a folder, telling it to sync, includding all subfolder, do not download. This does nothing to the files inside the folder, but the folder itself has a chack mark next to it. If I do the same thing, but to a single file in a folder, it syncs and returns the files to its original state, no longer a placeholder file. Selecting all the files in a folder, then tell them to sync, only syncs the firt file and does nothing to the rest. Odrive was left on overnight hoping it would figure out that it was only confirming the folder was synced, but not the contents of the foloders. Basicly I have 30-50K files that are placholders and oDrive thinks everything is synced up.
Hi @r_speth,
Can you clarify this? You would want to set it to “Everything”, by sliding the cursor all the way to the right. This will download all of the files, replacing the placeholders with ‘real’ files.
Can you try that?
Tried that already and just tried it again as instructed.
I’m going to let it run a bit, not sure what it’s doing.
Will update you in two hours.
We are talking about more than 3TB of photo and video files, let hope it’s not downloading everything!
Hi @r_speth,
Unfortunately that is the case. Placeholders are just representations of what is in the cloud. They take up no space on the drive. To convert a placeholder to a ‘real’ file, they will need to be downloaded again.
If all of the files are showing as placeholders then each file you want local will need to be downloaded again.
This does not make me very happy. Both for having to do this, and the reliability of this software to do what I need it to. Makes me very nervous about using this software at all.
Hi @r_speth,
The root cause was a failure of Amazon Drive’s storage service, unfortunately.
We can try to speed up the download with a CLI script that will “power through” the downloads.
To use the CLI commands from Mac:
-
Open a terminal session (type “terminal” in Spotlight search):
-
Run the following command in the terminal session (copy & paste + Enter):
exec 6>&1;num_procs=3;output="go"; while [ "$output" ]; do output=$(find "[folder to sync here]" -name "*.cloud*" -print0 | xargs -0 -n 1 -P $num_procs python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py sync | tee /dev/fd/6); done
Change “[folder to sync here]” to the proper path of the folder you want to sync.
It is an ugly one-liner, but the above command will download everything in the remote folder using 3 concurrent workers. It won’t stop until everything has been downloaded.
Yeah, so it’s downloading everything.
I’m going to let it run over the weekend and see where it is on Monday.
I do have a lot of this backed up to a offsite drive so maybe I can speed things up by stopping odrive and moving files over from the second hard copy. Thanks for the code, but I’m not sure if that will help much, I’m stuck with the basic Spectrum business class connection until we get Fios in our building. I only have 65mbps download, and 5mbps upload. I’ll see where it’s at on Monday, it tells me it’s at 4% now.
Thanks for your help.
Okay, thanks for the update @r_speth. One thought: you can prioritize folders that you need right away, and sync those first instead of syncing everything.