@Tony, thanks for the quick reply. No, it’s a folder but not one that got moved to the root directory. It’s not uniquely named either so I can’t tell which “Original Media” folder it is in all of my ACDrive.
I clicked odrive’s Send Diagnostics, but there was no confirmation it was sent… I’ll assume it actually went through. Please let me know what the next step is.
Hi @cliffordmpaulick,
I split this off into another thread to track it better.
I took a look at the diagnostic and I can see lots of items in the trash, awaiting delete. I also see that Amazon is throwing frequent exceptions, which is halting bulk operations, like empty trash. I think this is the issue you are seeing.
We can try to clear up the empty trash operation, first, then see if there are still issues to address. In this case we can use the CLI to better view what we are dealing with and to automatically loop the “emptytrash” operation.
To use the CLI commands from Mac:
Open a terminal session (type “terminal” in Spotlight search):
Copy and paste the following command into the terminal and hit enter:
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status
This will return a summary of odrive status. To get more detailed you can add parameters to the status command like this:
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --uploads
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --downloads
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --waiting
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --trash
python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --not_allowed
The “status --trash” command will show you all of the items that are being held in the odrive trash. You can look through these, just to make sure nothing stands out.
To empty the trash, continuing regardless of how many errors Amazon spits back at us, you can run this command:
while [ "$(python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py status --trash)" != "No trash." ]; do echo "There are items in the odrive trash. Processing the deletes ..."; python $(ls -d "$HOME/.odrive/bin/"*/ | tail -1)odrive.py emptytrash; sleep 10; done; echo "All items in the odrive trash have been processed."
To use the CLI commands on Windows:
Open up a command prompt by clicking on the Windows icon in the taskbar, typing “cmd”, and then clicking on “Command Prompt”.
Once the command prompt is open, copy and paste the following command in (all one line) and hit enter. This will install the CLI for us to use in the next commands. It could take a minute or two to complete:
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";(New-Object System.Net.WebClient).DownloadFile(\"https://dl.odrive.com/odrivecli-win\", \"$comm_bin\oc.zip\");$shl=new-object -com shell.application; $shl.namespace(\"$comm_bin\").copyhere($shl.namespace(\"$comm_bin\oc.zip\").items(),0x10);del \"$comm_bin\oc.zip\";}"
This second command will query the odrive desktop status:
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status);}"
Then you can get detailed status on specific areas with these commands:
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status --uploads);}"
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status --downloads);}"
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status --trash);}"
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status --waiting);}"
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" status --not_allowed);}"
The “status --trash” command will show you all of the items that are being held in the odrive trash. You can look through these, just to make sure nothing stands out.
The “emptytrash” command will tell odrive to empty the items in the trash:
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";$(& \"$o_cli_bin\" emptytrash);}"
This final command will push through any errors and continue trying to empty the trash until it is done:
powershell -command "& {$comm_bin=\"$HOME\.odrive\common\bin\";$o_cli_bin=\"$comm_bin\odrive.exe\";while ($(& \"$o_cli_bin\" status --trash) -ne 'No trash.') { echo \"There are items in the trash. Emptying...\";& \"$o_cli_bin\" emptytrash;sleep 2};echo \"Trash fully emptied!\";}"
That command to force delete everything in the trash ran for a while. After about 5-10 minutes, I left for dinner and came back about two hours later and it was all done: Image 2017-08-21 at 9.09.48 PM
As far as too many requests to ACD, I think it’s because I clicked on all of ACD in my Finder (the root of ACD) and tried to sync everything of 7MB or less… so it went through all of the folders before it went to the files within the root. So is there a script (or GUI) to tell it to only download items in the folder I’m in now? For example, if I select 10 files and right-click and click Sync, it actually only syncs that one file that I specifically right-clicked on… I’ve wanted it to sync all files selected in the Finder window for quite some time, but now it’d come in really handy.
Hi @cliffordmpaulick,
The too many requests may have actually been triggered by the deletes happening in rapid succession. It looks like it all finally went through, though.
For multi-select + sync, this is a current limitation of the odrive software, so you cannot select multiple files or folders and sync them all with the right-click->sync option.
If you want to download all files in a folder, only, and not sync files in any subfolders, you can right-click->sync on your desired folder, slide the slider to the size you want, and leave both checkboxes unchecked.
As far as feature requests go, please do eventually implement the right-click for multiple files/folders, as I mentioned before.
Also, please add additional timeframes to Set Auto Empty Trash menu option to allow something like 1 week, 2 weeks, and 1 month.
As far as the root files/folders goes, I synced the files all at once like you said but then synced each folder one-by-one… I found 1 folder that wasn’t empty and I wouldn’t want to lose its data and could actually tell where it came from… and it was a duplicate of a really old file, not something I think I touched recently. To be clear, it was in the location it should have been in AND it was in the root. I deleted the root one and emptied the trash and the “real”/expected location folder was NOT deleted, which is good.
I also noticed quite a few folders like “Taxes” and “Taxes (2017-08-01T00_34_13.197)” (not a folder name I would have assigned manually)… similarly: “searchform.php” and “searchform (2017-08-01T00_34_33.739).php” (one of 58 named searchform …php)
I also had a root-level folder named “RECORDS – PAPERWORK” and I found this file in the root: “RECORDS – PAPERWORK (2017-08-01T00_34_14.262)” – no file extension. I deleted it, emptied the trash, and my main folder isn’t gone (phew!)
However, I ran into the ACD rate limiting during deletes again (eventually completed via your script again). Is this a new/lower limit recently? And is it size/data quantity or is it quantity of files/folders/items?
There’s still more cleanup for me to do, but there are definitely some that aren’t duplicates so that’s scary, but I think I’ll make it through.
Hi @cliffordmpaulick,
Thanks for the update.
Amazon Drive seemed to have issues yesterday, with high rates of API request failures (including stricter rate limiting, which may still be in effect), unexpected exceptions, and incomplete returns. This caused a number of problems for users yesterday.
In any case, I’m glad things are looking better today. It sounds like you’ve got a handle on things now, despite the odd/erroneous Amazon Drive behavior.
It happened again. No!!!
Because I’ve now signed up for Amazon Prime (unlimited photo storage), I think I will keep my photos there (376 GB per Amazon Cloud Drive) and move everything else (408 GB video + 142 GB other) to either Dropbox or Google.
In addition to this Amazon Drive mess of late (what this thread was originally about), I also recently had an issue where I went to restore from Time Machine and realized the placeholder file is what got saved in TM and that made me realize that I probably want to move away from Amazon Cloud Drive sooner than later.
I’m leaning Dropbox because of the option to keep file versions longer than 30 days (additional $39/yr) – although I probably won’t sign up for that right away – and because I feel it’s a slightly better experience than Google Drive for the same price.
Then again, although I’m a little fuzzy on what it actually is, I think I like the direction of Backup and Sync for Google Photos and Google Drive , but I haven’t tried it yet an I’m unsure if it’d replace my Crashplan or my Amazon Cloud Drive or both and for how much?
I also see Dropbox has remote wipe for lost devices (unsure if Google does) and will eventually have Smart Sync (which is like odrive’s main feature) – although that won’t help me unless I can buy more than 1 TB from them to move my photos over too.
If you had to pick Dropbox or Google, both of which are $99/yr for 1 TB, which would you choose and why?
And how would you recommend making the transfer when my internal drive (where odrive is now) is smaller than 1 TB? With Dropbox’s native sync agent uninstalled, I’m thinking to connect odrive to my external hard drive, download EVERYTHING, then copy (not move) from Amazon to Dropbox, using odrive to do the uploading… because if I used Dropbox’s sync, it’d be on my internal drive, and it’d fill up my drive whether odrive or Dropbox was doing the syncing.
Hi @cliffordmpaulick,
Dropbox and Google Drive are both good services, offering good performance, versioning, robust trash options, and activity history.
I tend to like Google’s interface better than Dropbox and prefer that they do not auto empty their trash. I also like Google Photos, which will allow you to upload unlimited photos and videos (although possibly not in the original quality if they are too large), and can be linked to Google Drive for management and viewing.
If you are considering migrating to another service, you can take a look at this thread for some info on that:
If your local disk is not large enough to pull down everything, you can approach it in a piecemeal fashion, where you download a section of your data in the old storage, copy it over to the new storage, then unsync it on both the source and destination, to clear that space once you have synced the data up to the new storage.
Some folks have used cloud compute (AWS, Azure, or Google Cloud, for example) to facilitate an easier migration, but this is a bit more advanced. There is a post at the end of the thread above that mentions this method.
Thanks, Tony.
I looked closer into Google Drive and it looks good. I’ll try it out.
Is setting up odrive on an external drive instead of internal drive not advisable?
I setup Google Cloud Compute > Getting Started > Deploy a prebuilt solution > Windows Server 2016 Datacenter Edition (changed the Boot Disk from 50 GB to 2,000 GB – 1TB of Amazon + 1TB of Google)
To sign in, I set the Windows User password, downloaded the RDP file, enabled File Downloads and Javascript in IE 11 so that I could download Google Chrome so that I could download odrive’s exe
I couldn’t get odrive’s Google sign-on to work in this IE 11 (even after enabling JavaScript, it gave error about cookies not being enabled, but they were). Everything worked in Chrome, but I wasn’t able to download Chrome without JavaScript enabled in IE 11.
I’ll get all of Amazon and Google downloaded and synced and let you know how it finishes up and how much it cost (of the $300 free trial credit).
@Tony, one thing I’m curious about is seeing Google Drive documents (docs, spreadsheets, forms, etc) in my Finder.
Any suggestions how to keep those separate from actual files, non-g-cloud files other than just being diligent with putting things in separate folders?
For example, if I transition away from Google Drive to any other storage provider, will the .gsheetx, .gdocx, and the .gformx documents move over too and continue working (e.g. click to open in Google Docs or maybe even Microsoft Word)?
I hope this question makes sense. Thanks for sharing your experienced insights.
Well I just saw that iCloud Drive now (as of June) allows 2TB instead of 1TB for the same $9.99/mo as Google’s and Dropbox’s 1TB price.
I don’t need 2TB right now, but iCloud Drive’s 200 GB plan has been working well for me to keep my Desktop and Documents in sync between my iMac and MacBook… neither has enough internal hard drive space to store everything (almost 1 TB), but iCloud Drive does have an Optimize Storage on this Mac option (kind of like odrive’s functionality I’d guess).
One limitation I see is that each file must be 50 GB or smaller, but it also has a 30 day Empty Trash, like Google Drive and Dropbox do. I don’t think I have any file over 50 GB, but Google Drive’s max file size is 5 TB, which is obviously better.
The one hope I have for iCloud Drive vs Google Drive is that Optimize Storage + Time Machine will allow Time Machine to backup the actual file instead of the .cloud/.cloudf file (useless as a backup).
Additionally, although not ideal for odrive, for $9.99/mo I’m basically getting twice as much storage space plus odrive’s functionality via the Optimize Storage Space option.
One downside is that the iCloud Drive for Windows cannot be installed on Windows Server so that server instance I’ve got running wouldn’t be able to upload to iCloud Drive… ugh, this isn’t as easy as it should be… I really like the idea of odrive providing a cloud conversion (or call it a cloud backup/duplicator) service.
Please share your thoughts.
Sorry for yet another message but I realized iCloud Drive is missing the critical functionality of file versioning so I’m going to decide against it.
Hi @cliffordmpaulick,
no worries about the messages. I ended up getting pretty swamped today, so I couldn’t get back to this thread until now.
It sounds like you’ve been doing some good research.
Windows Server always locks down their IE. It can be really frustrating to get a normal web browsing experience when you first start out.
Google Docs are tied to Google. If you wanted to move them out you would need to export them to standard documents types. Google has a service called “Takeout” that can help with this bulk processsing, should the time come: https://takeout.google.com/settings/takeout/custom
You can always convert docs to standard Microsoft formats by downloading them from the Google Drive web client, too. Or, you can just avoid using Google Docs and use standard file-based formats for your work, although Google Docs do have their advantages.
odrive -> Amazon Cloud Drive on Windows is throwing the error, “Illegal file name” – https://cl.ly/421A0Y0h0124
I’m primarily a Mac user, but I can’t think of any file names that would be illegal that would be sent from Mac to ACD to Windows.
Is there a way to discover what file name(s) is causing this?
Hi @cliffordmpaulick,
It may show up in the odrive menu under “not allowed”. If not, you can send a diagnostic after triggering this and I can take a look.
There are a few characters that are not allowed on Windows that are allowed on Mac.
Some specific characters to look out for are:
':'
'/'
'\'
' ' (leading or trailing whitespace)
'?'
'<'
'>'
'|'
'"'
I didn’t see anything “not allowed” so I sent the diagnostics. Thanks.
Hi @cliffordmpaulick,
I think it is this folder that is causing the issue:
odrive/Amazon Cloud Drive/cmp/Tech/Media/Audio/Movie Voice KING/Movie Voice mp3/Zach/One...
Can you take a look?
Via my Mac, I renamed /Users/cmp/odrive/Amazon Cloud Drive/cmp/Tech/Media/Audio/Movie Voice KING/Movie Voice mp3/Zach/One.../ to remove the ... from the folder name
I then restarted the server and odrive has a pink icon as if it’s still working but the system tray icon doesn’t display a percentage. And the “One” or “One…” folder doesn’t exist on Windows: https://cl.ly/3n101B1v2W16
Hi @cliffordmpaulick,
Do you see the One folder in the odrive web client? If so, then it could be a refresh issue of some sort so you can try right-click->odrive refresh on the Zach folder, or a right-click->unsync on the Zach folder, and then resync it.
With the difficulties of dealing with the server, odrive’s messages about illegal file names giving me concern, and me not wanting to babysit things… I instead used CloudHQ
Simplicity-wise, it was just what I wanted. Unfortunately, it took 1 month to transfer 1TB from Amazon Cloud Drive to Google Drive. After doing some quality control, I think it worked 100% correctly (phew!)
