Google Drive (G Suite) on macOS High Sierra

[timsilva]

Hi there, I’m on the latest version of masOS, and I have a G Suite account and I am uploading to Google Drive through odrive with a Premium account.

I am having a number of issues. I am uploading several terabytes of data from an external hard drive, so I expected it to be a slow process. Unfortunately, I am running into issues with the desktop application.

First, the drop-down menu from the “Status menus” part of macOS (the top-right corner) randomly stopped working. Normally, when you click on the pink infinity logo, the menu pops open and I can see “Syncing changes (#)”, “Open odrive folder”, “Manage storage”, etc… HOWEVER, after random amounts of time (sometimes 10 minutes, sometimes 2 hours), this menu simply DOESN’T open anymore. I click on it, and it looks like its trying to open, but it doesn’t. If I click it a few times, then sometimes it will open up, but usually not. Annoyingly, the only way to close the app now is through the Activity Monitor app. It’s become a huge pain to have to constantly end the process and restart the app all the time. I need to wait a few weeks for this to upload, so I’d like to have confidence in leaving my computer alone to upload for 12 hours without having to worry that it froze after the first 15 minutes. I have tried the three different upload speed options under “Set bandwidth throttling”, but I can’t tell if they actually do anything different. Also, can the usage guide for odrive gives NO meaningful information on the differences between “Unlimited”, “Normal”, and “Limited”. What do these actually mean? Does leaving it on “Unlimited” trigger issues with bandwidth caps more frequently?

Second, I have noticed that the list of files being upload under the “Syncing changes (#)” sub-sub menu doesn’t seem to reflect ALL of the files that I am actually uploading. It’s shows some of them, but not all of them. Is this because smaller files (under 100kb, for example) aren’t there long enough to warrant appearing in this list? I mean, I have a wide variety of file sizes; terabytes of data, millions of items (files/folders). I don’t expect everything to show up. I just want to confirm that this is expected behavior. When I sync massive folders with many sub-folders and files, I can see that it appears everything did in fact sync. But I don’t have 100% confidence.

Let me know.

Thanks!
Tim

[Tony]

Hi @timsilva,
I think most of what you are seeing is just overhead, as odrive is trying to index and track millions of objects. This sounds like a pretty massive data set you are bulk uploading, so I highly recommend trying to break up the set.

odrive needs to scan and track that entire set in real-time, both on the local and remote sides. Since odrive is full sync rather than traditional backup, the processing is heavier. Breaking up the dataset and letting the “segments” sync, before adding the next can help a lot.

Another thing that can be extremely helpful is to unsync sections that have already completed syncing to the cloud. This reduces the terrain that odrive needs to cover and relieves overhead processing.

The larger the dataset, the more work is required, and that work is constant. I expanded on this a bit in this post: Backup (one-way sync) - #11 by Tony

The odrive menu tends to get bogged down with this amount of activity, so you can also try using the CLI to help with odrive status. Check out this post for information on that:

How can I use the CLI to check odrive status?

To use the CLI commands from Mac: Open a terminal session (type “terminal” in Spotlight search): [image] Copy and paste the following command into the terminal and hit enter: python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status This will return a summary of odrive status. To get more detailed you can add parameters to the status command like this: python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --uploads python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)od…

@timsilva

the differences between “Unlimited”, “Normal”, and “Limited”.

These settings are more to compensate for the local environment. The logic tries to be adaptive, with the intention to leave some amount of bandwidth for other systems tasks (web browsing, for example). Unlimited doesn’t impose any limits, Normal tries to allow for about 10-15% network for other things. Limited tries for about 40-50%.

[timsilva]

Thanks for the detailed and thoughtful response Tony!

The scanning of the local & remote files makes sense. Glad to hear that this behavior isn’t unexpected. Great tip on unsyncing folders that have already been uploaded. I will experiment with that a little to see if it helps.

From my limited experience over several days of using this software, I have found that the context menu options (Stop, Remove Sync, Sync, and Refresh), especially in relationship to the folder icons states (the pink syncing and the blue checkmark) behave a bit wonky. I’m never sure if something that I did has gone into effect yet. I find myself waiting a few minutes, relaunching the odrive app, and relaunching finder. Again, this is probably related to the fact that my folders are massive amounts of files in them. I did a quick count, and there are in fact several million items. For now, things are going somewhat smoothly, so I will leave it to do its thing. If I run into any other specific bugs that I am able to reproduce, I will report them again here.

The main outstanding issue I am having is around the uploading randomly coming to a halt, and the drop-down menu freezing. I am currently just closing and relaunching the app every 15 minutes or 30 minutes depending on how busy I am. Not ideal, but it works for now. I suspect that once the data set is uploading, the ongoing syncing shouldn’t be a real problem.

Thank you for explaining the upload bandwidth settings. I will keep it on Unlimited and Normal depending on where I am. Others may find that useful on the guide page. I’m sure the goal is the keep it a light and clean as possible, but that info might be worth squeezing in.

[Tony]

Thanks for the follow-up @timsilva! The context menus acting a bit weird is not abnormal when odrive is cranking through things. I usually recommend just leaving it alone to do its thing, if possible.

If you notice that things halt again, I would be curious what the CLI shows. I have seen where the odrive menu isn’t responsive, but there is still activity in the engine, when under heavy load.