Recently I started having a peculiar sync issue. Only some random selection of the new folder uploads gets synced to my computer(s). Here are the details:
I have the following folder D:\odrive\Google Drive\Photos\Android
I sync all the new photos from my phone to the Android folder on Google Drive via IFTTT
All the photos from the smartphone were successfully sent by IFTTT to the destination, Android folder (checked IFTTT logs)
all the photos from the smartphone are mirrored on GD and visible in the web interface (both GD and Odrive)
Only only a fraction of all the photos reaches the Odrive folder on my computers, not even the placeholders.
Problem occurs both on my desktop and laptop
Both computers run on Windows 10
Hi Igor,
Usually the cause of this is illegal characters in the filenames. Since you can see the items in the odrive web client, that means that odrive is capable of seeing the items (logic is the same between clients). I cannot see the full file names of the missing items. Is it possible they have “:” characters in the times?
Can you send a diagnostic from the odrive tray menu so I can take a deeper look?
Hi @idesignwebsites,
I took a look at the diagnostic. I see many instances of Google Drive rate limiting the API calls. Google Drive is probably the most strict when it comes to enforcing API limits.
I noticed that you have over 13,000 folders synced in odrive and the vast majority seem to be inside “1 IDW Google Drive/3-Toolkit/”. When odrive is refreshing the contents of that account, it is hitting many rate limit exceptions because of the number of items in that location.
Here is what I would recommend trying:
Unsync the D:/odrive/1 IDW Google Drive/3-Toolkit/ folder, or at least the “D:/odrive/1 IDW Google Drive/3-Toolkit/FONTS” folder
Right-click->“∞ refresh” on the “D:/odrive/1 IDW Google Drive/Photos/Android” folder
It may take some time after the unsync in step 1 before the API limits are relaxed, so it would be good to wait several minutes.
If you are still seeing a discrepancy after that, send another diagnostic. Also let me know if you see any errors when performing the right-click->“∞ refresh” on the “D:/odrive/1 IDW Google Drive/Photos/Android” folder
It’s the second time I needed help with odrive and it’s the second time I’m extremely impressed. Thank you! The quality of the help you provide here is really exceptional.
Now that you explained what is the issue, I think I even have an Idea what exactly happened.
Recently I installed Extensis Suitcase Fusion - a font manager. I placed its library in the D:/odrive/1 IDW Google Drive/3-Toolkit/FONTS
Apparently, Suitcase Fusion has its own cloud sync. But I thought that it wouldn’t hurt to have an extra copy of the library, just in case. Well… looks like this time it did hurt:)
I imagine that Suitcase Fusion must have been making all these calls to its own cloud sync and these must have been done on some insane rates.
Now it seems simple but I really wouldn’t be able to figure it out myself.
Hi @idesignwebsites,
Thanks for the follow-up. Let me know if removing that folder (with unsync, or just transplanting it elsewhere), clears up the issue you are seeing. It sounds like the likely culprit in a few ways ;).
I followed all the steps that you suggested, plus I tried some additional things like renaming the folder, moving the folder, removing files in the folder, but my Photos/Android still is not in sync. Not only older photos are missing (still) but also new uploads are not appearing there. I’ m just pinging diagnostic again, hoping that you’ll be able to help.
Hi @idesignwebsites,
I took a look. I see some activity in the Photos/Android folder, so it seems to be doing some syncing in there. It is hard to gauge it, though, because there is still a lot of noise from the odrive/1 IDW Google Drive/3-Toolkit folder.
If possible, can you do the following?
Unsync odrive/1 IDW Google Drive/3-Toolkit to remove the noise
Restart odrive (this will ensure we have a clean diagnostic)
Wait about 10 minutes
Add a file, locally, to “D:/odrive/1 IDW Google Drive/Photos/Android” and make sure it syncs
Add a file remotely to 1 “IDW Google Drive/Photos/Android” from the Google web client
right-click->refresh “D:/odrive/1 IDW Google Drive/Photos/Android”
Send another diagnostic
After step #7, let me know if the files added in #4 and #5 synced as expected. Please also let me know the names of those files.
Once again, thank you for your time. I completed steps 1-7 as listed above.
Test files synced both ways instantly. Names of these files are: “test-web” and “test-desktop”
I was playing a bit with this stuff yesterday by the way. I tried renaming the folder, moving it, etc. What worked was renaming the files. Like this one: January 19, 2017, at 0133AMxx - after I renamed it in the GD web client it appeared on PC. New files are still syncing to PC randomly.
Hi @idesignwebsites,
The behavior you are describing sounds like something in the name is preventing sync. The names you are giving as examples seem fine. It is possible they have any whitespace at the beginning or end of the file name?
Are you consistently seeing that renaming a file allows it to show up? Can you give me an example of the rename (from and to)?
Look at the file names I listed above:) Aren’t they… the same?
I’ve selected “Date Taken” ingredient in the IFTTT applet builder to be used as the file name. The problem is that IFTTT engineers didn’t think through the date format too well. And that actual file name is not created based on date taken, but based on upload (I guess) date. (I noticed it because there were two photos I know they couldn’t be taken within the same minute)
So - the date format on IFTTT lacks precision. Within the same minute, there could be several pictures taken - and they all will have the same file name. GD and Odrive don’t care - I guess there must be some checksum that is taken into account? But Windows - Windows can’t handle two files with the same name in one folder. And that’s the reason for the patchy sync.
So obvious, right? But I noticed it only because you asked me to provide file name samples and linked to the applet. I went to IFTTT to double check whether I didn’t have any extra spaces (as you suggested it could be an issue) and then it hit me
I didn’t realize that there were multiple files with the same name. Indeed, that would cause a problem when pulling them down to the local system. It’s too bad that the timestamp doesn’t provide granularity to the second. It also doesn’t appear that there are any other ingredients that could be used, aside from possibly the URLs, which would make the file name very ugly.
What happens if you do not enter an ingredient for the file name?
Yes - certainly the file naming options are not the best at the moment. There are easy fixes - like you mentioned - with seconds, but I wonder if they would care to implement it. And as I said - it looks like they are faking “Taken Date” which in reality looks more like a “Pull Date” - so seconds might not fix this issue. So maybe simply n+1 option is lacking.
These are the alternatives by the name:
public link: httpslocker.ifttt.comf30d58817-17f1-4364-a760-c2dd80df8987.jpg
when you leave the file name field empty: 22ec365f-b490-49b8-a0c1-85aed0c7a691
Yes, not pretty:)
Luckily batch file renaming is not a big issue and the date is saved in metadata anyway
@Tony once again, thanks for all your help. I really appreciate. The level of service I’m getting here is top notch (consistently) and odrive itself, well - just an epic bit of software. I think it may be time to get the paid premium. Despite the fact, that feature-wise there is nothing really, that would tempt me …you made the free plan just too good
Thanks for helping me work thought this issue and for the swift follow-up. It is much appreciated. We are working on some alternate purchase plans that you may be interested in. I expect them to be available within the next 2 months or so (although we are not very good at estimates… )
Hi,
I’m having the same issue with filenames that contain colons (in Google Drive in this case). They show up in the web client, but not in windows client. Most files are created via the web interface, but that is slow from South Africa, so I’m trying odrive - which is much quicker - but is losing these files:
Hi @g.hillebrand,
Unfortunately there isn’t a better way to handle this, at the moment.
We will list these items in the “not allowed” section of the menu so that they can at least be identified, but we don’t currently have a way of converting these names into something Windows will accept.
It is something we would like to support in the future, however. We just need a reliable, bulletproof method of doing this. Our previous attempt ended up creating more problems than it solved…
Hi,
Thanks for the quick response. I figured out the “not allowed” list after I had posted this.
I don’t know what you guys tried, but hashing the not-alloweds and dropping the illegal characters (which is what Google seem to do), or escaping using ~'s (ala the DOS 8.3 mapping) come to mind. I’m sure you have smart people on it. It was just awkward that the first file I looked for had a colon!