Hi, noticed something odd with my synced folders. Cloud placement files are appearing inside folders where they do not exist. So in my Videos folder which I have synced using the premium version I can also see cloud placement folders for Music folder and 6 others, as highlighted in attached screen snip… (except Backups which I have navigated to but is also not in this folder physically) . All these folders live in Amazon Cloud Drive and have synced correctly from their parent folders.
Hi @andy
Take a look at this thread. I think you may be running into scenario:
We actually just implemented some new behavior for this that will prevent this from happening. The new behavior will seek out an existing folder if there is one that matches the name of the local folder (in your case that would be Videos), otherwise it will create a new folder. That should hopefully be pushed out next week.
Ah my bad for not finding that on a search… I do seem to remember the contentsof my videos folder being copied to the route of Amazon cloud drive which would explain why the other folders now appear there… Am I right in thinking if I move them the entire lot would need to resync? Ah I see that as long as I move on one computer within the same container, Amazon cloud drive in this case, it should sort itself out in the cloud and other clients.What is the simplest way to move files and folders around?
Removed the other questions, bold part answers them.
Hi @andy,
I’m hoping this is already all buttoned up, but I wanted to make sure.
My suggestion for sorting out the issue you have would be to remove sync on the folder you selected for “sync to odrive” (right->click->remove sync). Then move the folders to the proper location via the Amazon Drive webclient, or from inside the odrive\Amazon Cloud Drive link in Explorer.
After the files are in their proper locations in the cloud (presumably under /Videos), you would right-click->sync to odrive your local folder again, pointing it to the “Videos” folder in Amazon.
Well, I have already moved most of the folders within the odrive\amazon… Folder on my server which has successfully moved them on amazon drive. However I was assuming that when sync is complete (I have so far only moved folders that were fully uploaded) I would need to do as you suggest and stop and stop the sync relationship to point to the new folder. I was nervous that simply moving as I have might do something to the physical location of the files on my server but it doesn’t seem to have done. It makes logical sense that the folders I have moved are at present simply a copy in their new Amazon location until I reestablish the sync link. I will drop a test file in one of them just to test that out too, just to see what happens. I will post back here.
Unless you think I should not have done it this way?
Edit: I should have definitely done it your way, since on checking several hours later I have effectively unsynced most of the Videos folder by moving as I did whilst it was still linked. I’m going to look at how to undelete all the files which will be quicker than downloading them all back down, probably after breaking the link which I will re-establish afterwards. Ok I won’t be unlinking first since this would clear the trash as I understand it after doing some searching… So at present I could recover all the files one by one, all 320 of them, which would still take a lot less time than downloading 1TB of files again. Any news on when multiple recovery is on the cards? I now have a greater understanding of how the product works, and much more space on my server!
Hi @andy,
Once a file is in the odrive trash, the local contents are no longer retained by odrive. The odrive trash is basically just a place where the delete “commands” are held until the user empties the trash, at which point the commands are sent to the cloud.
There are certain cases where the files may be in your OS trash, though, so you can take a look to see if they can be grabbed from there and put into the proper place, locally. If the files are no longer found anywhere locally then you will, unfortunately, have to re-download them.
Unlinking and unsyncing won’t commit the deletes to the cloud. They will essentially discard any pending deletes. When you link again or re-sync, the previously deletes items will be visible and available again as placeholders.
The good news is that once you have your data in the cloud, you are safe from any catastrophic/unrecoverable problems (aside from perma-deleting items out of the cloud storage trash). This is why I always stress just getting all of your stuff up, in its entirety, as the main priority before doing a lot of reorganizing or pruning. The other things can always be worked out later , once everything is safe and sound.
Ok thanks @Tony for your help and clarification. Some of the files are in the system recycle bin but most aren’t. Fortunately have a decent download speed so it won’t take too long to get the files and in reality unsynching some of them is probably worthwhile anyway. As you say once up they are safe enough.
@Tony just adding to this thread again because it is doing my head in a bit. My uploads had slowed to a crawl and I wanted to reorganise my folders, only one was left to sync and was doing one file at a time which was painful. So I removed sync and it just will not stop… The one file uploading I selected stop sync, it just keeps going as does the folder sync. How long is the remove sync process supposed to take? Until the file currently pending is processed or just everything? I was uploading two folders to one location on Amazon Cloud so it was also downloading placeholders for the files/folders from the second upload folder. I have removed sync on both these folders. Is the only way to stop this by removing the amazon account from odrive and reorganising before relinking?
Hi @andy,
In this particular case, can you try stopping global sync from the odrive tray menu? Click on “Stop automatic sync”, under “Syncing changes” (the first entry in the odrive tray menu).
Once you do that, do you notice any files in the folders that you removed sync on queuing up in “waiting”?. If not, try starting sync again. If that doesn’t work you can also try restarting odrive.
Just as an FYI, I always recommend not messing with sync until it is finished, just to make sure you don’t create more issues. It sounds like things are mostly finished, though.
Amazon Drive can be pretty slow with single file uploads. Currently odrive will limit uploads to a single file if that file is 100MB or more, so it may seem like upload is slow when it reaches a file like that. We are working on refining this behavior.
Hmmm, stopping auto sync I had already tried and with a restart but items simply kept restarting or re-queuing so I unlinked the account and that has stopped all activity for now. I will reorganise my folders and then set up the link to Amazon drive and resync folders which should hopefully avoid huge amounts of upload/download activity other than the files yet to sync up.
The main issue with files over 100MB then is how slow they go. Some of my files are 15GB or so and use a mere 10% of my upload bandwidth. Using the Amazon client (which I notice has a syncfolder these days which fills up with any folder you happen to upload!) it maxes out my upload capacity but of course is very unreliable for large files, constant duplicate file names, failed uploads etc. Hopefully odrive will be that happy medium but Amazon themselves don’t seem that helpful to developers.
I will update here when I have sorted my file life out
Quick progress report @Tony . So all seems to be going well, I have split the one big folder remaining into 6 sub folders and synced each individually so I am getting about 75% upload utilisation, which is a big improvement speed wise. Once the big upload is done I aim to stop sync and then move the files back to one big folder and resync without having to reupload.
Just a waiting game now for the next few days. I will then reconnect the other folders that had already uploaded, again in more logical folder structure to avoid the mess I got myself in before.
Hi again @Tony, slight issue with the 6 sub folders, one is reporting invalid on the popup and removing sync and retrying makes no difference. Looking at file dates it has reported invalid more or less since I added it three days ago. Interestingly when I tried to add back another folder, Backups, which exists on the root of my Amazon Drive to start the resynching process after stopping them all the other day, this also reported as invalid. I just tried my music folder and this too reported invalid, so currently it seems any new additions fail. Time for a diagnostics send?
Diagnostic on its way. As soon as the sync is tried it shows as invalid. to my surprise the last attempt resulted in the folder starting to sync! Will be interesting to see what the diagnostic shows…
Well, initially I had set six folders syncing, all with video files in them, over two TB of data. It was only when I noticed the upload rate had dropped sone that I saw that folder3 was reported as invalid under the sync to odrive menu option on the tray icon.
I removed the sync and restarted with no joy several times. I then tried my music folder just to reestablish sync after my initial problems. This failed for the same reason several times.
Maybe it is as you say and the client was too busy, but maybe it was some issue at the Amazon end too? At one point yesterday about 12 files were listed as not allowed, this is now down to two.
Everything else has now resynced to my original uploads which is great and I only have about 1.2TB to go for full sync.
Just one of those things I guess but odd that nothing showed up. It doesn’t log the invalid file/folder status?
Currently the diagnostics do not capture as much as I would like. This will be changing in the next few months as we flesh out the logging capabilities, including exposing certain pieces to the user. Additional changes will be made to give more visibility into what it going on, too.
On top of this, the concurrency and throttling capabilities are being tweaked. Service like Amazon Drive as especially susceptible to slow down if concurrent transfers are not being done. This is because Amazon Drive shows a pretty slow single-stream upload speed for most folks. When a very large file is uploading, odrive tried to limit the amount of concurrency, to prevent multiple streams of very large size, as there is more probability of interruption. I see that you are using IFS, so that helps the resume capabilities, at least, but you can still find yourself in a single-stream upload situation to Amazon Drive, which will stand out as being “very slow”.
As for the issue you are seeing, I have a feeling that there are times where odrive is busy enough that it falsely detects a problem with one of the “sync to odrive folders”, or, as you said, an Amazon exception triggers a response that indicates that the target is having an issue/not available. Generally you can just let it sit and odrive will auto-correct. This seems to be the case for you, according to what you are observing
