Syncing nearly full Google Drive

I have several Google Accounts that i manage with odrive.
Recently I shuffled a lot of files around to consolidate space - I then left odrive running for a weekend and expected to see the syncing finished on return.
Instead, what appears to have happened is that odrive attempts to sync the NEW files to Google, without deleting the removed files. The result is thousands of “Not Allowed” messages. My recycle bin is empty - it isn’t clearing up the deleted files at all, so there is never any room to store the “new” files.

How do I resolve this? I’ve tried restarting odrive, but it just starts the doomed uploading process all over again.

Hi @tangotonyb,
Moves can be tricky on the desktop client because odrive has to infer what has happened on the local filesystem (file is no longer at place x and the file at place y is the same file that was at place x). It tries to optimize these changes into a move, but sometimes it is unable to and sees them as a delete and an add of a file.

In these cases odrive will hold the deletes in the odrive trash, so you would need to empty the odrive trash to make sure those files/folders are deleted in the cloud. https://docs.odrive.com/docs/sync-changes#section--delete-existing-files-

If you need to perform a lot of reorganization, I usually recommend using the service’s web client to do so (Google Drive web client, in this case). This way the move operations are direct, rather than inferred and synced.

Hi,

  1. you cannot use the Google clients to love files BETWEEN Google accounts.
  2. as already stated my recycle bin is empty.

The problem it sppears is that odrive , when faced with a large number of changes, leaves the deletes until last instead if prioritising them to make space for new files.

A further issue I’m encountering

To resolve the above problem I decided to simply empty my google account drives and start again.
I deleted all files locally and then emptied my recycle bin (in odrive) - it took a few goes as it errored out a few times, but eventually, odrive showed an empty recycle bin.

But when I look at my Google Drive online, the recycle bin there is full.

Surely this is a bug or is this a limitation of odrive, that it basically cannot handle deletes?

Yet further investigation has revealed that in an account where I only use a fraction of the available space, despite odrive indicating sync is complete for this folder, and despite the recycle bin being empty, when I log into Google and look in the recycle bin online, I find hundreds of files there.

So what exactly does emptying the recycle bin in odrive do? It appears not to affect the Google account, which eventually runs out of space and sync fails.

I am using odrive version 6284

Hi @tangotonyb,
odrive doesn’t touch the Google Drive trash. The only way to empty that is to do it from the Google Drive web client. This is a safety precaution to ensure that files are not accidentally deleted from cloud storage, where they would no longer be recoverable.

The odrive trash is just a secondary safety measure, to try to further prevent unwanted deletes. You can read more about it here: https://docs.odrive.com/docs/sync-changes#section--delete-existing-files-

The odrive trash becomes especially useful when you are using storage that does not have its own trash (S3, WebDAV, FTP, SFTP, etc…)

That is the central issue here then, and this certainly affects how GDrive needs to be used.
If I’m backing up data that involves file deletes, then I need to ensure I leave enough space free in my Google Account in order to handle this.

Running at 95% capacity is a sure-fire way to break the system.

Hi @tangotonyb,
If your backup depends on deletes taking place before additions then you will definitely need to empty the Google Drive trash after any deletes have been sent to the cloud.

odrive Premium has an option to automatically empty the odrive trash, although we recommend not enabling that, if possible.

It’s an issue regardless of the order of the operations!

I have found, and you have agreed that odrive does not delete from the Google Trash folder. This means that if you store 14.9gb of files with Google, and then delete .1Gb of files, Google Drive will be full, regardless of what happens with odrive trash, until I have manually logged into the Google Account and THERE deleted the item from the trash. Until this manual delete takes place, odrive will be unable to sync any NEW files, or replace any files with larger versions.

It seems to me that there needs to be better flagging of this process to the user - at least odrive can monitor the Google Drive trash folder size and alert the user when space is low that they can go and empty google trash.

And unless you are prepared to continually go to Google to emtpy the trash, I would suggest that storing anything more than 14Gb of data on Google is problematic as you need to leave space for the trash folder so that backups are not prevented.

Hi @tangotonyb,
So you are seeing that when you have 14.9GB stored and the only operation you take is to delete .1GB of that 14.9GB, Google Drive will say it is full? That is definitely unexpected, and sounds like a a Google Drive bug, if that is the case.

What I would expect, and have seen through previous testing, is that you would have 14.8GB of “live” data and .1GB of space taken in the Google Drive trash for a total of 14.9GB.

The next generation of odrive will have a much more robust interface to see status, and it will be much easier to identify an out-of-space scenario like the one you are seeing. We don’t have any plans to touch a storage’s trash area, however, so that will always require an explicit user action unless the storage offers auto-emptying of their own trash.

No - I am talking about what would constitute normal usage of a backup system.

Let’s take an idealized example.

I have a folder with 14 files in it labelled A to N, each is 1gb in size.
Backed up with GDrive, that leaves me 1Gb of space in my GDrive

I then replace “A” with a file named A2 (also 1Gb) - as far as Google is concerned A is deleted and a new file A2 uploaded.

Google now has 14 Gb of files in my folder and 1Gb in trash - it is now FULL.

I then replace B with B2 - this will now fail to sync.
Not just that however - EVERY additional file, or increased size file will now also fail to sync, and emptying the trash in odrive will not change this.

This is what occurred with me, except I had thousands of smaller files, many of which were being replaced by different named ones over a period of months - I wasn’t even aware the sync was no longer working for a while, and then it took me a while to figure out why emptying the odrive trash had no effect.

Given how odrive and google work, this situation must arise quite a lot - certainly enough to warrant a message from the tray telling you to go empty your trash bin ON GOOGLE! Otherwise, you could be left with

Hi @tangotonyb,
I agree. odrive should do a better job of informing the user that the target storage is full. The current interface doesn’t put that information front and center, so it can be missed. This is improved in the next generation product.