Hi, I see that file mode (permissions) and apparently ownership of files is not preserved in Google Drive over odrive, and after a little reading I see Google Drive inherently does not appear to preserve this metadata inherently, but is this something that odrive can/will be supporting in later versions?
I specifically mean file permissions and ownership of the file. It would appear that all synced files get 644 permission, even if they were executables. This is obviously a problem when syncing scripts and binaries, among other things that are permission sensitive. File user and group ownership also appears to be lost and replaced with the user and group of the user downloading the file.
I think you are talking about the POSIX information of your files or the ACL’s if its winders data. If you’re not, then please ignore the rest of this. 
Honestly with most cloud providers you are working with some soft of an Object based storage not a file based storage. To boil it down, you don’t really have a file and a directory anymore, you just have “objects” that have enough metadata to reference themselves in a traditional tree. If you really need to preserve your file permissions, I recommend using something like Tar or Zip that will encapsulate all the Attributes of your files given filesystem and move that tarball or zip file into the cloud. There are windows tools like xcacls that may let you export this to some sort of a file in of itself, but you’d probably have to write your own script logic to reapply it if you had to bring it back.
1 Like
While you are right about the fact that files are stored as objects, (Actually, the are called BLOBs) most “proper” file sync providers like BOX and DropBox preserve the metadata when you use their native client. With DropBox API v2 which was released to developers in November of 2016, you have access to these features as well. Alas, poor odrive doesn’t support API v2 yet even though it’s over 4 months old now. (However, I am told by @Tony that will be rectified in your next, and arguably first, proper release, which is the only reason why I am still giving you guys money!) DropBox preserves all file POSIX metadata natively, such as file user and group ownership, creation and modification date, permission bits, etc… natively however, in cases where the cloud provider’s object store does NOT store metadata like this (making them inferior to DropBox), like Amazon or Google Drive, if you use an odrive Encryptor Volume, the file’s metadata appears to be preserved (most of it, anyway). I presume this data is kept separately as its own data object separately from the file data itself. It appears that some of odrive’s support of cloud volumes similarly preserves this data by virtue of its implementation of file sync over the cloud and honestly should be the case for all cloud providers, regardless of whether its on an encryptor volume or not.
We will always strive to retain the basic attributes, like modtime. The other stuff is less straightforward for a number of reasons:
Storage source metadata capabilities
Operating system differences (Windows vs MacOS vs Linux)
uid/guid differences across systems
Capabilities of the running odrive process (the ability to change file ownership on the local system, for example)
Most storage sources are starting to support arbitrary custom metadata attributes, which means you can store anything you like, so this stuff is possible (barring local system restrictions). It is something we will keep in mind as we continue to develop our integrations.
Absolutely… trying to come up with a notation for representing the metadata for numerous OSs is no small task since you have to group some stuff under things you can calll “common elements” like ownership and permissions, but even these vary subtlely, so you still end up having to “fiddle” with things where you think you got the data translated right but forgot about that one weird case when you have the sticky bit set in linux which has a different meaning in MacOS… and… and… well you get what I’m saying- making it coalesce is tricky businesses and prone to compatibility problems, especially around major OS release times.
I was going to suggest something you might pass along to the engineers:
When syncing metadata, work with two versions in your data store: one that is a carbon-copy of the native OS that goes untouched, another that is the “unified” version that works to bridge functions between OSs wherever possible and making “best guesses” for places it isn’t. That way, when you are syncing between the same OS, you can always rely on the native metadata and the “unitied” version of the metadata never comes into play. This would of course mean two versions of the metadata for each file within the oDrive datastore, but it’s only metadata, right? Sure, the data type for the metadata has to be overridable so that it will take the form of whatever OS the file originates from, but assuming that there is only going to be one version of these that is in the form of whatever OS the file was LAST MODIFIED on, you will always just have those two. In this model, the use of the native metadata would not come into pay much if there is a lot of sharing between, say, windows and mac, so the fact that the native metadata goes back and forth with its type casting is probably not that big a deal- though it might be extra cycles that slow things down- so maybe in those cases… skip the native metadata and just rely on “unified”. However, wherever there are files which are “pristine” and have never been modified on another OS, why not just keep the metadata in the same form it was to start with, using the “unified” version of it only when you sync to another OS? Just a paper napkin of an idea… but might be worth something.
Thanks for the feedback @DarfNader.
I see the logic in what you are proposing.
I think we could come up with a viable translation matrix for the “common” metadata to “OS-specific” metadata. It would allow for consistency in the approach and formats. If not, the two-pronged approach could possibly work, too. These things can scope-creep pretty quickly as you dive into the possible lifecycle of a file and the implications for single-user vs group use, but I am all for retaining more metadata.
I made a reference to this thread to the product team for consideration.
1 Like
Cool!
Like I said, it’s just an idea, and there are lots of ways one can preserve file metadata for a specific OS. I know that Windows has a lot of metadata that is Windows specific, and as a Mac user I am not particularly concerned about Windows metadata, I am sure Windows users would really appreciate this function. Sure, there is the small matter of the fact that you’re often limited to the metadata support that the cloud providers offers with their datastore, but I suppose you could always keep your own version. That’s where things really get into scope-creep-land because really, where does it end?
Ultimately, the goal is the provide enough metadata so that basic things that we have all come to rely upon like permissions, last modified date, creation date, etc… and if we could get that stuff down so that you could reliably expect the support of those key fields in any OS in any cloud datastore, that would be an enormous improvement from the way things are which is pretty willy-nilly. This goes back to the “Features Support Matrix” which I proposed in another ticket, but to be able to just stick with a set metadata fields and say with confidence “these are preserved” anything else is “your mileage may vary” would be big progress. The only thing worse that finding that an essential feature isn’t supported is expecting that feature support but it isn’t there.
Anyway, I am happy to prattle on and on about the virtues of metadata to anyone brave enough to listen, so feel free to ask me for feedback or beta test something. I have enough computers where this is actually not a big deal for me.
1 Like
Thanks @DarfNader,
I appreciate the feedback. We will certainly keep you in mind for any beta programs we roll out, for this area as well as others.
Hi @Tony, it’s been ages since we’ve talked about this (I haven’t been using Odrive for over a year before realizing that I was paying for premium still- whoops!) and since then I have learned a few things about the native Dropbox does not actually preserve files’ POSIX information (file mode, ownership)) and instead “fudges” it. @toupeiro earlier pointed out that was probably because Windows is “special”. However, since I have discovered tools like google-drive-ocamlfuse which allowa you to make a mount point of your Google Drive folder as well as preserve POSIX information.
So my question remains does ODrive support any POSIX information preservation (most notably, file permissions) for any cloud vendor? I realize that the complexities of Windows ACLs remains, but it this would be huge if you use cloud storage for shared “workstation environments” like Ruby gemsets within rvm or rbenv and you need those “rubies” to be identical if you develop code using multiple computers which you want to be in-sync. Honestly, any traction in this department would be most welcome!
Hi @DarfNader,
There was a post here about this, but I think you may have missed it when Amazon corrected their login issues:
The short answer is that we do not preserve any POSIX info for any integrations, but I have raised the idea again and pointed the product team to the above conversation and this one now so that they can keep this in mind. So far there just hasn’t been enough demand for this feature to put it on the roadmap.
1 Like
Greetings.
I’m doing an odrive evaluation, and I was curious if the “retaining ownership and permissions” went anywhere. It’s not a showstopper for me, but it would be incredibly useful if permissions (ownership is not so much an issue for me ) were preserved across synced folders.
cc: @Tony
Hey @raklahn ,
Thanks for reaching out.
There hasn’t been any changes in this area, sorry to say. There just hasn’t been much demand, and allowing for this type of metadata retention adds complexity to the core sync engine, which we generally want to avoid (if we can help it).
Are you able to share some details about your use case for this feature?
Hi @Tony
Thanks for your quick reply. Let me try and describe my use case. I regularly have to switch back and forth between two Mac laptops. I was kinda forced into this situation, it a long story, but that part of this saga I can’t do anything about. So, I’m looking for a solution that can keep the two of them in sync, at least for the working directory I use.
This working directory contains executables. (In particular, Terraform providers Provider Documentation - Terraform by HashiCorp, but there are others) that update on occasion. When they update, they sync to Google Drive and the other Mac, and lose their executable bit on the other side.
I have a workaround, a short shell script that uses “find” and “file” to identify executables, and then uses “chmod +x” to turn the bit back on, which takes all of 30 seconds, but it’s still a hassle.
This is not an urgent feature for me, and probably will not keep me from a purchase, but would make my life easier.
1 Like
Hi appreciate the response @raklahn .
For your use case, it sounds like you only need the execute access mode to be retained. It is a reasonable request and something I’ve hit a few times in my own use.
There is likely to be some work done in this general area in the not too distant future and we can take a closer look at adding some minimal file mode retention, although I can’t make any promises.
Again, @Tony thanks for the quick response. Hope you are enjoying the weekend.
Yes, the deal for me is the execute access. I have not run into any other part of the permissions, and ownership is also clearly not part of my use case.
1 Like