I am using odrive CLI on a linux machine and it recently finished syncing my 515gb of files to Google Drive. Once it completed that first round I’ve added another 615gb of files to upload. Since adding those files to the folder, it has not synced.
Checking the status, it says nothing is waiting and nothing is uploading.
When I try to unsync the entire google drive from odrive, it says it can’t because it’s still syncing a directory.
I’ve tried shutting down the client and restarting it, same issue.
I’ve tried unmounting the mount point and remounting it again, same issue.
Is there a way to get it restarted in syncing, or wiping out the databases and start again or some other way to force to start syncing?
I just added a small file to the root directory and it uploaded right away, no waiting. Other files nested deep in my folder structure do not sync. Others have at the same folder level/depth have during the first major sync, but nothing I’ve moved to there new has been synced(sunk? lol) yet.
Hi @james4,
When a new set of data is added to odrive’s scope, it will scan through the entire set and try to determine the most efficient way to upload. If the number of objects is very large this can take a very long time, and it can increase processing overhead to the point that you start getting some resource exhaustion.
To avoid this, I always recommend importing data in segments, instead of all in one go. This reduces the overhead and allows odrive to process the subset of data faster. If you can break the set up into smaller chunk and wait until a chunk has finished before adding another you should see better results.
Additionally, if there is already a lot of data that odrive is tracking, the new data is additive. odrive still tracks all of the existing data, on both side, to determine if any changes have been made. This will also increase overhead as the dataset grows and grows. If you haven’t already done so, you can unsync portions of the data that have already synced up and that you do not need immediate access to to reduce the amount of work odrive has to perform continually. This will also help move things along for new imports.
When I finished the first round of sync and it was complete, I completely unsynced the entire directory, then only synced the directories I would be working in.
Mentioning the overhead, I went to check and see the processes that were running. I had odrive agent running under my user and under root, so twice. It should only be running under root on my system. I killed it for the user and I am letting it go to see what happens. I’ll report back after a while if I don’t see any changes.
Thank you for your help. Going forward I will add things in small batches.