Amazon is failing on some of these uploads with a “Cannot complete upload” error. That is all it is sending back, so I can’t say why they can’t complete the upload, but it sounds like an internal error occurred.
Network exceptions (broken pipe). This can happen for a number of reasons, but most likely indicates that the connection is being severed somewhere upstream.
Are you copying some of these files in from a network volume? I see some files are increasing in size after sync starts. Usually this happens when the file is picked up by odrive before it has finished its copy (when users are copying files from some other place, for example). If you are copying items in, it may be best to stop odrive until that copy operation is complete.
They used to eventually make it but recent behaviour shows that they never get uploaded.
They can sit at anything between 93-98% for a few seconds to about a minute, before moving to the next file. Obviously in terms of bandwidth I can check after an hour and about 10gb was sent but nothing “actually” transferred. Very frustrating!
Since you had better luck before, I am hoping that perhaps this is a temporary thing with Amazon Drive. I also wish they would release their upload resume API…
Thanks for such a great reply. I think it’s the fact I was transferring files to my Mac from an external drive whilst it was still syncing. I’ve been stopping and starting since last night and large files have synced fine. Fingers crossed.
@n1ppy,
Is this still with only large files? You could send another diagnostic and I can see if it is still the same issue.
We have our IFS feature (split files) that will split files up into smaller chunks that odrive can still access as if it was a single file, but I am guessing you want these files to all be “whole” on the destination storage.
So I created a new Odrive folder on the same volume and then manually moved all the folders and files. Lo & behold the synching kicked off again, everything seems to be working again and I’ve had constant uploads for the past 3 hours. Albeit only one file at a time even though average size is only about 8mb, I believe this a limitation of service right and now and you will be introducing controls for threads later?
A little bit frustrating that only a tiny bit of total bandwidth is being used but I’ll take that over nothing!
So things definitely improved but it was only short lived. The problem reared its ugly head again!
But good news, after further investigation I finally found the offending software. I use software to monitor the network throughput on my machine called NetWorx (was freeware till this month) along with NetLimter 4 to throttle usage.
It so happened as soon as I killed Networx app everything was perfect. So for some reason this was killing uploads towards the end of transmission.
Thanks for your help and hopefully this may be useful for someone down the line!
Hello again, everything had been running so smoothly. I now always ensure odrive is turned off before moving large files over and then starting it up. And it’s been great. But I’ve been uploading a huge file over the past 24 hours. And really frustratingly it’s gone back to uploading it all over again. I’m going to send a diagnostic right now.
Repeated uploading is a common issue with large file uploads to Amazon Drive. I think I see the file you are talking about and Amazon Drive issued a 504 (Gateway timeout) exception on it after uploading. This indicates that Amazon had some issue when processing the file on their side and then returned an error to us that the upload had failed because of it. I typically see either a 504 or a 408 error from Amazon Drive on these large uploads.
We are going to try to combat this as much as possible in future revs, but there are limits to what we can do when Amazon is the one producing the errors. I think if Amazon Drive was to release an upload resume API, that would also help tremendously.
Well that’s very helpful to know. Thank you.
So I stopped the transfer and a short while later logged onto the amazon drive iOS app to discover the file is there. I therefore checked back in odrive to discover it actually had the blue tick next to it. I was also able to unsync it. Curious.
What seems to happen is that Amazon throws the upload error, but ends up processing the file, eventually. odrive will query Amazon for the file before starting the upload again, but if the file hasn’t “registered” yet then the file is not found and odrive will re-attempt the upload. If this happens multiple times then you will get this recurring upload cycle…
When you stop upload for a bit, the file can magically “show up” and then all is well. It ends up being a timing issue, though there doesn’t seem to be a predictable delay, at least that I have been able to figure out.
Unfortunately I had the same issue reoccur but with a smaller, albeit still large, file. This time, a miracle hasn’t happened. I can only but hope amazon develop the API to further needs. I’m sure they will eventually. Thank you for your time.
Sorry to drag it up again but it’s happening frequently this past week with numerous file upon file ceasing completion of upload just as it is meant to. I’ll send you a diagnostic report, if you wouldn’t mind just checking. Thank you.
Sorry for tardy reply. Thank you for your observations. I do find that particular files can be stubborn for some reason or another. Also, it can happen upon a fresh restart of the Mac… once it has been operating a while, things seem to balance out.