Duplicate Files Issue with Amazon Drive

Recently the duplicate files in the root folder issue that happens with amazon drive has gotten exponentially worse. Instead of just having to delete the duplicate files once a month, they now show up everyday, sometimes twice in a day. When looking on the forum to see if there was an answer I noticed that odrive just received an update around the same time the issue became a daily problem. I wonder if the intended fix has actually made the problem worse? Has anyone noticed this?

Hi @acatalfamo,
This is the first report we’ve seen of an issue after our new version.

This was the change made:
Previously odrive would issue a “send to Amazon trash” command for each item that ended up in the odrive trash. This means you could end up in a situation where files in a folder were put in the odrive trash, and then the parent folder was added, after that. When the odrive trash is emptied, it issues the commands to send the individual files to the Amazon trash and then the parent folder.

Amazon doesn’t seem to perform any consolidation of their cloud trash, so our theory was that an out of order sequencing when Amazon auto-purges their trash (every 30 days) could lead to orphaned files, somehow.

The improved method was to optimize the odrive trash prior to sending to Amazon. This means that when you empty the odrive trash, it goes through the list of items and removes “duplicate” trash commands if they will be covered by a parent folder.

As an example:

You delete the 10 files in a folder and then, later, end up deleting the entire folder.

  • In the old method, odrive would send 11 “trash” requests: 10 for each file and one for the parent folder. When Amazon purges its cloud trash, it seems to purge them in the same way, with 11 operations.
  • In the new method, odrive would only send 1 “trash” request for the folder. When Amazon purges its cloud trash, it only has to act on the 1 folder.

The actual Amazon orphan files problem happens when Amazon purges its cloud trash, or the user initiates the Amazon trash purge from the Amazon Drive web client (https://www.amazon.com/clouddrive/trash). It may be that you are seeing the results of deletes from 30 days ago that are now being emptied automatically by Amazon. Is it possible that you had several deletes on subsequent days that are now being purged and causing orphans?

The new odrive version with the optimization has only been out for 14 days (pushed to the autoupdater 12 days ago), so its impact on Amazon’s auto-purge shouldn’t be seen for another 2 weeks or so.

I don’t often sign into Amazon Drive through their website but I did yesterday and saw that all the duplicate files were in Amazon. First time I’ve run into that issue but deleting them in Amazon solved the issue. I just assumed the was the orphan files problem since that’s what it’s been for over a year. Thanks!

1 Like

Thanks for the update @acatalfamo! I’m glad you’ve got it sorted now.

I am anxious to see if the changes we’ve made improve the Amazon trash situation in the coming weeks.

Hi Tony,

Yesterday I had hundreds and hundreds of duplicates in my main folder again. Wanted to mention it since I know the odrive team was trying to solve this problem. Reading through your explanation of the issue again, I want to mention that the duplicate files are not ones that I have deleted or even opened recently.

Thanks!

Hi @acatalfamo,
The only issue dealing with random files in Amazon’s root that I know of are due to the way Amazon empties its trash. Is sounds like your issue is different than this, so I’ll need some additional info to figure out what is going on.

Was it only files, or were there folders too?
Can you describe your typical workflow as it pertains to Amazon Drive and odrive?

It is files and folders. I work off the odrive/amazon cloud folder on my desktop. I also have an external drive synced up that I use infrequently while traveling. I am a graphic designer for a clothing company so I have a massive amount of image files. The files that get duplicated are always a random mix… some files are years old and haven’t been touched in that long, others are recent files. Sometimes it duplicates hundreds of files, sometimes thousands. It happens once a month.

Hi @acatalfamo,
Since it is happening every 30 days, it sounds very similar to the trash issue.

Perhaps there are files that are ending up there as a result of reorganization, like moves. Sometimes odrive cannot optimize the move, so it ends up being an add of the file in the new location and a delete of the file in the old location. Those old location deletes would end up in the trash.

Do you empty the odrive trash often, or have an auto-trash rule setup on the odrive client?
When you go into the Amazon Cloud Drive trash (https://www.amazon.com/clouddrive/trash), are the files/folders in there what you would expect? Do you notice any odd behavior when you empty the Amazon Cloud Drive trash?

I can say with confidence the old files are not being moved or deleted. They are in folders I never touch. The new files are definitely being moved around.

I do not have auto-delete set up for trash in odrive. Not sure what to say about amazon trash because right now it’s full of the duplicate files from last week. I’m deleting the 3000+ files in there now and will keep an eye on it.

1 Like

Thanks for the update @acatalfamo