Unexplained issues syncing a folder with Amazon Drive

I’m trying to systematically rename my files in my Amazon Drive. Everything was going well at first. Then I got to one folder that has essentially become completely unresponsive. I can make changes locally without issue but oDrive simply will not sync them. I can make changes to the Amazon Drive folder itself without issue, and if those changes affect other folders the change to the other folder will sync. But nothing changes in the other version of the problem folder no matter what I do.

How do I force oDrive to deal with a particular folder that it’s ignoring?

Hi @las1817,
Can you tell me the name and path of the folder in question? Can you also send a diagnostic from the odrive menu and I will take a look?

Do you see that folder listed in the odrive menu under “not allowed” or “waiting”?
Also make sure that the new name given does not follow into one of our ignored patterns: https://docs.odrive.com/docs/sync-changes#section--ignore-list-

None of the new names had ignored patterns, and none of them were “Not Allowed” on Amazon Drive. The weirdest thing about the past few days issues is that it took a long time (sometimes even one or a few restarts) to get updated files to even show up in oDrives waiting list.

Looks like things might be going smoothly again today though. I will send diagnostics and ping this thread again if things take another turn for the worse.

I guess it might be worth asking more generally though: are there any ‘bad practices’ that are likely to lead to issues when batch renaming files in a synced folder? I’m wondering if I could have done something that triggered an anti-robot trip wire or something? I also sent a query to Tech Support at Amazon yesterday around the same time that I posted here. They haven’t responded, but today I’m not having any issues. Maybe just coincidence, but still I’m curious now.

Hi @las1817,
I’m glad to hear things are back on track for you.

There were a few reports yesterday of odd Amazon behavior. I believe Amazon was having some general issues, internally, that were manifesting in high API request exception rates, unexpected errors, and incomplete responses. Things seemed to settle later in the day (PST) and users that reported issues previously noted that their issues had largely gone away.

Much worse issue today: I programatically renamed all files in a folder (there were quite a lot of them) and oDrive responded by apparently deleting most of them. The amount of (renamed) files remaining in the folder on my Mac is exactly equal to the amount of files that have been renamed in my Amazon Cloud Drive if I go there directly. And there is a massive list of files in my trash bin according to oDrive but not according to Amazon Drive!

Hi @las1817,
It sounds like the renames were picked up as deletes and adds instead of renames/moves. This can sometimes happen in certain rename scenarios, depending on the number of renames and the current odrive activity. For bulk renaming/moving it can help to close odrive, perform the bulk renames/moves, and then start odrive again to help odrive optimize the bulk action.

The files in the odrive trash won’t register as deletes on Amazon until you empty the trash. On the Amazon Drive web client, do you see both the old named files and the new ones, currently?

I will make sure to close oDrive prior to performing any bulk renaming operations in the future. I have noticed behavior along the line of what you describe previously. What worries me this time is it’s behaving differently than in those cases. On other instances, the files with the old names have occasionally shown up as deletes, as they have this time, and eventually things would sort themselves out. But this is the first time I can recall seeing an instance where, in addition to this, the renamed files disappeared entirely.

On the Amazon Drive web client, there are 88 files of the problem folder that have been renamed. It appears that one of these is duplicated in that folder, with a file of the older name also present. All the rest of the files in that folder, however, retain their old names only.

Hi @las1817,
I just want to make sure I am clear on what you are seeing.

You renamed files locally.
The new file names are showing both on the local and remote side, as expected
The old file names show in the local odrive trash
The old file names do not show on the remote side

Is the above correct? If not, can you clarify?

Not quite.

I renamed (several thousand) files locally.
A small fraction (88) of the new file names are showing both on the local and remote side, as expected.

On the local side, the folder which had contained several thousand files now contains only that small fraction of 88 files with new names.
On the local side, oDrive’s Trash Bin now contains 4478 old file names.
On the local side, I can find no definite evidence that the remaining several. thousand files with new names exist anymore.

On the remote side, the folder continues to contain 4566 files.
On the remote side, 88 of those 4566 files have new names.
On the remote side, 4478 files still have old names.

Strangely, given the way the maths work out above, one of these 4478 is an old name of one of the 88 that are also present with new names.

I understand that this makes for a confusing situation. I am scratching my head as well. At least when I am remaining calm enough not to be tearing hair out instead! :confused:

Hi @las1817,
Thanks for the details. This is definitely strange.

First, let’s get you back to where you were before. It looks like you have grandfathered unsync, so you can unsync the folder with those files in it to cancel out any pending deletes held in the odrive trash and return that folder back into a placeholder. When you expand that placeholder folder again (double-click into it). The files will be as they were, minus the 88 files that were renamed. This should let you get back to a “good” state, so you can move forward.

Can you tell me how you renamed these files? I am trying to reproduce this behavior, but I haven’t been able to. I would like to try using whatever method you used.

Ok, I will do that.

As for how, I’m a bit embarrassed to admit it: I am very new to automating things programmatically outside of Excel macros, so I’ve only taken the path of least resistance so far. The renames were performed with an applescript that checks the old file name against a condition and either renames it or else moves it to a holding folder if the condition is not met.

Thanks @las1817,
No need to be embarrassed at all!

Were any of the 4566 files moved to the holding folder, vs renamed? That conditional behavior is interesting in that there is a possibility for the file to be moved elsewhere. It would help to explain some of this behavior, if so.

In any case, I am hesitant to take up more of your time with this. If you are able to unsync that folder then resync, you can exit odrive, run your script again, and verify the results locally before they start syncing. If you are satisfied with the local output, start odrive again and we can observe the sync results.

Hi @Tony,

Unsyncing and resyncing worked like a charm. I haven’t had any issues running the script while oDrive was closed, and right now it and Amazon are chugging away syncing the folders I’ve renamed without any apparent issue. So thanks!!

As for whether any of the files had been relocated: No, not for this folder. All of it’s file’s names were well-behaved, and the holding folder didn’t receive anything new. If that conditional might be relevant, I suppose I’ll mention that there was one more: each file name is also checked to see if it already had the new naming pattern so that it can be skipped over if so.

Here are the script’s other idiosyncrasies for thoroughness. I can’t see why they’d be relevant to why the glitch might have occurred, but then I’m also a newb, so I can’t see why my not being to able see it should be particularly relevant either:

(1) the full script was set up to loop through all subfolders of a particular folder. It is the files of each subfolder that are getting renamed. This outer loop worked well at first, but for reasons that I have for now stopped trying to fathom, once it got to the folders with massive amounts of files, it started terminating itself upon completing a folder.

(2) when i first started the project, having noticed oDrive occasionally get confused for awhile after bulk renames using a 3rd party app, I incorporated a delay after each rename or file move action. I guess I was hypothesizing that it would help oDrive keep track of things by breaking it up. I was definitely operating under a false hypothesis that oDrive would respond to the changes most efficiently if it was active and connected to the web while the changes were being made. Then I started experimenting by reducing the delay time to see how well the syncing operations would keep pace. Once the issues on Amazon’s end that started this string had been resolved, things were going smoothly enough that I decided to remove the delay to see what would happen. The first time it all went well: it took awhile for all the syncs to carry through to the web client, but I didn’t have any issues. The glitch that we’ve been talking about the past few days happened the second time I ran the script without a delay. Well, not quite without a delay: the script still contained the delay command, but its duration was set to zero.

(3) Given that I was experimenting with delays and working over a lot of files, each folder was added to a list in a txt once complete. This let me terminate the script as needed and restart it later: the outer loop checks each subfolder against the list so that it can skip it if it’s already been processed.

I don’t know if any that is helpful. If you’ve read through it all and concluded that it’s not, sorry for making you wade through it…

And, once again, thanks a lot for the advice! It saved the day.

Hi @las1817,
I really appreciate you writing out all the details here. I don’t see anything in the explanation that stands out, so the behavior seen is a bit of a mystery still, unfortunately. We have actually revamped our whole move tracking process in the next generation of odrive, so if it was an odrive bug, it has likely been removed as part of that thorough overhaul.

I am really glad to hear that things are back on track. Thanks for your patience and help!