seems there is an issue with the odrive windows client and the encryption of large files stored in spaces.
What I did:
added amazon drive to odrive
created space ‘software’ in odrive
create encrypted directory in space’s folder: ‘spaces/folder’ (odrive path!)
setup encryption phrase in most current windows client
synced ‘odrive/Encryptor/Software/SomeFolder’ to windows client
added file (~4GB) with name ‘SW_DVD9_Exchange_Svr_2013w_SP1_MultiLang_Std_Ent_MLF_X19-35118.ISO’ to this path
it get’s stuck in some sort of upload loop.
it finishes the upload. amazon drive already reports the filesize is entirely present
in it’s space consumption info-graph.
once reached 100% on the windows client it starts over at 0%. now looped about 3 or 4 times.
it works for small files though, so I started from 1. - 5. again, then activated IFS, set
to 100MB, then copied the ISO file again to ‘odrive/Encryptor/SomeFolder/Software’. Same issue, but a little different,
because of IFS. Amazon Drive keeps counting up in 100MB steps, until 4GB is used. Once reached 100% in odrive client,
it just restarts from 0%.
2nd approach: tried without a any subfolder just ‘odrive/Encryptor/Software/’
now it finishes the sync of the file (/w 100MB IFS). but there are too few files on amazon drive. about 2/3 of the original file. but on the windows client it says sync is OK. after unsyncing and trying to resync the file, the client tells me: “Can’t sync de_windows_10_multip…vd_7223737.iso.cloud.
The file could not be decrypted. It may have been modified after it was encrypted.”
I just tried the same file with IFS (set to 100MB) with a plain folder on Amazon Drive and an encrypted folder directly placed on Amazon Drive.
Only the unencrypted upload worked. So it seems unrelated to spaces.
After unsyncing the encrypted file and syncing it again, it gives me:
“Can’t sync de_windows_10_multip…vd_7223737.iso.cloud.
The file could not be decrypted. It may have been modified after it was encrypted.”
This is a known issue with downloading IFS files in encrypted folders that will be fixed in our next desktop release (which should hopefully be soon).
For the encrypted folders in Spaces, the issue you are seeing may be unrelated to encryption. Can you repro that again (not using IFS) and send a diagnostic and the path to the file that is in an upload loop?
I just uploaded a file to odrive/Encryptor/Software/W.D.X/W D X D1.iso. (8.5GB)
without IFS, encrypted folder on space root directory.
The encrypted folder itself resolves to odrive/spaces/software. (which is a odrive Space called “Software”).
The space is itself is located at Amazon Drive path AD/encrypted/spaces/software.
The file just got uploaded to 100% 13.August 2018 12.00pm (GMT+0).
between 12.00pm and 12.02pm it showed the file as syncing, but without any progress (percent) indicator.
at 12.02pm it started with 1% progress
Diagnostic report send at 12.06pm (to wait for about 3% of sync progress on the next loop)
I am going to try an unencrypted space with the same file now. Will report back in a few hours.
Thanks for the information. I see in both cases there were instances where “read timeouts” occurred. This generally means that odrive sent data to Amazon and is waiting for them to reply, but they do not reply within the timeout period (60 seconds).
Unfortunately, when this happens, at any point in the transfer, we need to start the transfer over from the beginning, unless IFS is being used. We’ve seen this happen quite a bit with Amazon, which is actually what prompted us to develop IFS
What I’ve seen quite a bit is that Amazon can timeout after receiving the whole file.
Amazon Drive is actually pushing the data into S3, and there is a hand-off during the upload transmission process, including a hashing process. On large files Amazon can not respond for an extended period of time after we have transmitted all of the data. We tried increasing our timeout to accommodate this and found that Amazon would timeout the connection themselves because we hadn’t sent anything within 60 seconds (lol). It is possible they have removed their own timeout now, so I will put this on the list for revisiting. The current solution, though, is IFS.
@pkoenighofer if you are still seeing this, can you send over another diagnostic? I am going to investigate the timeouts a bit more and I need a good example to take a look at.
I uploaded a folder consisting of two 8.1GB sized files. It looped them after each other! FILE1 reached 100%, then FILE2 started and FILE1 was placed as “Waiting” in the odrive queue again.
After FILE2 reached 100% it started re-uploading FILE1 again, placed FILE2 in the queue and so on.
Since it took already all the spaced need on amazon drive. I decided to quit the odrive client and deleted the local odrive folder. (even while uploading the file again at 37% already)
after restarting the odrive client I was able to download both FILE1 and FILE2 again. I checked the hash values. They matched.
This problem is re-produceable with large files on unencrypted and encrypted spaces. Just upload a folder with two large files to an amazon backed folder.
It seems the client does not mark the file as finished and re-adds it to queue forever. So I think there is no timeout problem with Amazon, since the data got stored. It is a problem with odrive.
Even if there is a timeout. Amazon seems to do it’s part correctly. Since I was able to retrieve both files, after they started looping. Just find a way to properly detect the upload finished. As far as I noticed Amazon only reports a filesize via API, when the file is stored entirely. So there should be a way to recheck the state of a file after a timeout.
Another bug report:
After deleting the local odrive folder, it recreated it.
but it trashed “Amazon Cloud Drive”, “Encryptor” and “Spaces”.
I was able to untrash them via the odrive client. But still, I don’t think they should be trashed since they are system-defined folders and present on the web client anyways.
Thanks. Did you send a diagnostic during this time so I can take a look?
We are supposed to account for this scenario. What happens is Amazon times out, but ends up processing the file. We are supposed to query Amazon for another listing before starting the file from waiting again. The hope is that Amazon will now list the file as there and we can mark it as synced. Either this is not working as expected, or it is taking Amazon longer to list the file than it is for us to restart syncing it.
The diagnostic may be able to give us a clue. My own testing tonight is not hitting this issue, so I’m hoping to look at yours. Can you also tell me what region/country your Amazon account is in?
all times are CEST. I am located in Austria. I noticed some connections to fls-eu.amazon.com.
09:49 Upload start. odrive/Spaces/TestSpace/W.D
At first it showed the entire directory as “Syncing” item with percentage count.
10:17 4%
10:43 9%
11:21 17%
11:51 23%
15:23 Reports WD1.iso in “Syncing Queue” with 33%
WD2.iso is visible on AMAZON DRIVE
15:51 Reports WD1.iso 44%
18:44 odrive finished uploading both files.
send diagnostic report
18:45 started same folder with same two files to sync to encrypted space odrive/encryptor/software/W.D
show W.D directory with percentage count
19:19 WD2.iso uploading 1%
WD1.iso not present on amazon yet.
19:45 WD2.iso 12%
22:58 WD1.iso 2%
WD2.iso is on amazon drive now. but already in the waiting queue again.
22:59
send diagnostic report
Will send another one as soon as the already present file, start uploading again.
Any ETA on the release of the windows client?
If IFS is working properly with encryption, that would be sufficient for my use case, no need to upload large files directly
What’s the hold on? Maybe you can compile an intermediate release?
I would also be willing to try out a new build and report back.
23:02 WD1.iso 4%
00:05 26%
01:15 Lost internet connection. ISP modem firmware flash/reboot. ordrive goes idle
01:20 internet connection restored. odrive start syncing again
REstarts with WD2.iso (which is already present on amazon drive!)
does not show any percentage
01:27 start with 1% of the already present file. (maybe is unable to detect presence with encryption enabled?)
WD1.iso which is NOT on amazon drive, is not even in “Waiting” queue (yet!)
send diagnostic report
23:02 WD1.iso 4%
00:05 26%
01:15 Lost internet connection. ISP modem firmware flash/reboot. ordrive goes idle
01:20 internet connection restored. odrive start syncing again
REstarts with WD2.iso (which is already present on amazon drive!)
does not show any percentage
01:27 start with 1% of the already present file. (maybe is unable to detect presence with encryption enabled?)
02:05 13%
06:26 WD2.iso at 38% (should have looped once. i guess)
06:29 send diagnostic
as far as i see it. first it started alternating again, then after loosing internet connection it completly forgot about the other file.
now it just loops on the file present at amazon drive.
i would have understood if it now tries to loop the missing file and fail, but not the other way around.
so detection may be broken in two ways. 1. cannot detected finished encrypted files 2. after loss of connection forgets about other file.
