"Amazon Cloud Drive internal server error." for more than 50% of my syncs

Troubleshooting
1

Hi folks !

I’m trying to get my data back because of amazon’s new pricing plans that give me 180 days to remove everything before they do it for me…

The problem is that almost 80% of the syncs return “Amazon Cloud Drive internal server error.” without anymore further information.

How can I even troubleshoot this ? What kind of error is it ? What can I do.

The other threads on this forum are 1 year old and do not give any solution. My files doesn’t have any special character and are between 10MB to 4-5GB.

I am also opening this thread so that the team can accept my diagnostics from the “Send diagnostic” menu.

I am using v6413 on iMac late 2012 on a 4 TB drive.

Any useful help would be appreciated.

Thanks.

2

Hi @salmi.charaf,
I am seeing a lot of network timeout errors when getting content from Amazon’s servers. It may be best to use the CLI to perform this bulk export. The CLI method is more “brute force” and won’t stop after hitting several of these errors.

Take a look at this post here:

Try running that for a while and see if you have better success. You should also be able to get a bit more visibility into what is failing.

3

Thanks.

How do you see the timeout errors ?

I can’t find any verbose log file.

I am already using CLI.

$ for file in /[PATH]/**/*.cloud; do echo $file && ~/.odrive/bin/6413/odrive.py sync $file; done;

4

Hi @salmi.charaf,
You won’t be able to see those underlying network errors, as they are not surfaced in the UI. For the CLI, I would suggest using the script in the post I listed above, as it will continue to cycle through until all files have been synced, even if they previously hit an exception.

5

Ok, I am using this script since my last message. I still get a few ACD Errors, but I can’t see the exact proportion as I can’t count how many file are currently downloading. I am sending another diagnostic now so you can learn from it and, perhaps, give me an hint.

Thanks !

6

Hi @salmi.charaf,
Things seem to be moving along. You are syncing 4 files at a time.

You are going to get ACD errors as you go, either network exceptions, rate limiting, etc., so that is expected. The CLI script will just keep chugging along.

You may eventually get down to just a set of files that won’t download (illegal characters, for example). Those will cycle endlessly at that point, so just keep an eye on things every once in a while.

7

Hi !
There are still a looot of ACD errors, like I am getting 1 success download every 6 hours, the rest is failure.
What happens if I download the more important files from the web browser and manually replace the .cloud files ? Will odrive let me do that ?
Thanks.

8

Hi @salmi.charaf,
Can you send another diagnostic from the odrive menu and let me know when that has been sent? I would like to see what things currently look like. The version we just released yesterday should have helped some, actually.

9

I’ve sent one yesterday night and I just sent another one right now. Thanks.

10

Hi @salmi.charaf,
The latest diagnostic is not showing too many sync errors. There are a few network timeouts to Amazon Cloud Drive, but I am not seeing a ton of errors.

Can you provide a few screenshots of the most recent errors you are seeing and then send one more diagnostic, so I can be more precise in what I am looking for?

11

image
image.png802×462 65.3 KB

12

I just sent another one.

13

Hi @salmi.charaf,
The error shown above was this one:
host='content-eu.drive.amazonaws.com', port=443): Read timed out. caused by ReadTimeoutError(HTTPSConnectionPool(host='content-eu.drive.amazonaws.com', port=443): Read timed out.

This indicates a network error, where Amazon stopped sending data for long enough that the connection timed out.

It looks like the files being downloaded are all pretty large. I would suggest changing the num_procs value to 2 or even 1 instead of 4 so that you are not downloading so many large files at the same time.

I am looking at what we could do on our end to try to mitigate these, as well. We already have a fairly generous timeout, but its possible that Amazon needs a larger one for downloads for some of their nodes.

14

Hi Tony,
I’m here to give you some news. Yesterday, my last ACD file has been downloaded (a 30GB file that has been retried around 30 times). I’ve almost downloaded 5 times the ACD source because of these errors, but now it’s ok.
Today, I decided to check the files and suprisingly found almost 2 thousand 0 byte files. Astonishing ! I’ll open another topic for this.
Thanks a lot for the answers you gave already.

1 Like