One-way sync? (Newbie question)

I’m just starting with odrive, so please excuse if this is a silly question. I’m hoping to set up a 1-way sync such that a cloud directory just always mirrors a given local directory. The “backup” functionality almost seems like what I want, but I don’t want to keep file revisions or deleted files around…I just want it to be a mirror of the local dir. To further complicate things I only want it to mirror files with certain extensions to the cloud drive, and ignore all others. I think I can probably accomplish this with the sync functionality, but I don’t really want odrive to ever be making modifications to the source/master local directory…ideally I’d like to just give it read-only access to that dir. Can anyone give me any tips/tricks on how best to accomplish this? Thanks!

Hey @chris,
This is an interesting use case.

As you said, backup is close to what you want, but it is “traditional” backup with versioning instead of a mirror.

Sync is going to always want to sync bi-directionally. Giving odrive read-only access may work, although I haven’t tested that. I think you’ll need the mount folder to be writable during setup, at least, but after that it may function the way you want. If a remote change happens in the cloud odrive will keep trying to sync it, and be unable to, but that may be okay for your purposes…

How large will this folder be and how frequently do you need it to sync?

Just throwing things against the wall… You could potentially exploit the mount unmount commands if you only need periodic mirroring, although it feels a bit “dirty”…

  • When you create a mount odrive will treat it like a “merge” operation. Anything local that is already on the remote is left as is and considered synced. Anything new on the local side is uploaded, anything on the remote side that is not on the local side is shown as a placeholder.
  • When you unmount, odrive forgets all of the information it knew about the mount, so mounting again just repeats the merge operation.
  • If you basically just always kept odrive in “merge” mode (mount, wait for sync to finish, unmount, repeat), odrive would never have enough information to know to delete or overwrite any local files.
  • One caveat to this is it that it is expensive because odrive will need to look at all of the local and remote folders each time you mount (although it is not really much different, work-wise, than the periodic remote scans odrive does every 14 hours). You wouldn’t want to do it more than a couple times a day, especially if its a large structure.
  • Another caveat is that local deletes won’t be reflected to the remote (the files that were deleted locally but exist on the remote would just show up as placeholders), so you would probably need to add an additional step after the mount and sync to delete any placeholders and then empty the odrive trash before unmounting…

We have an advanced Premium feature that allows you to define a list of items to exclude. It sounds like you are looking more for a way to be inclusive rather than exclusive, but maybe it would work well enough for your needs: https://docs.odrive.com/docs/advanced-client-options#blacklistcontains-blacklistextensions-blacklistnames-blacklistprefixes

It’s all a bit “hacky”, I’m afraid, but definitely interesting.

Hmmm that is very interesting. It’s sooo very close to what I was hoping for. Just curious, has there ever been any consideration given to having simple remote file manipulation utilities? Like just the ability to list/copy/delete individual files from a remote store without having a local sync location set up? I can imagine the need for this is probably fairly limited, but it sure would be handy :slight_smile:

To answer your questions, the files I want to mirror are ~500GB consisting mostly of 3MB-5MB files but it doesn’t need frequent syncing - once a day is probably sufficient. The entire directory structure is north of 3TB, which is why I want to exclude the unwanted files…

1 Like

Hey @chris,
We actually were working on a “toolkit” that had features like you mentioned (ability to perform basic, one-off operations against cloud storage). Unfortunately, as you surmised, there wasn’t enough interest to continue, so it was shelved. Maybe it will be resurrected some day…

Awwww shucks…well if there’s backlog tracking on that feature please add my vote to the list :grin:

1 Like

Hey @chris,
This is an interesting use case.

As you said, backup is close to what you want, but it is “traditional” backup with versioning instead of a mirror.

Sync is going to always want to sync bi-directionally. Giving odrive read-only access may work, although I haven’t tested that. I think you’ll need the mount folder to be writable during setup, at least, but after that it may function the way you want. If a remote change happens in the cloud odrive will keep trying to sync it, and be unable to, but that may be okay for your purposes…

How large will this folder be and how frequently do you need it to sync?

Just throwing things against the wall… You could potentially exploit the mount unmount commands if you only need periodic mirroring, although it feels a bit “dirty”…

  • When you create a mount odrive will treat it like a “merge” operation. Anything local that is already on the remote is left as is and considered synced. Anything new on the local side is uploaded, anything on the remote side that is not on the local side is shown as a placeholder.
  • When you unmount, odrive forgets all of the information it knew about the mount, so mounting again just repeats the merge operation.
  • If you basically just always kept odrive in “merge” mode (mount, wait for sync to finish, unmount, repeat), odrive would never have enough information to know to delete or overwrite any local files.
  • One caveat to this is it that it is expensive because odrive will need to look at all of the local and remote folders each time you mount (although it is not really much different, work-wise, than the periodic remote scans odrive does every 14 hours). You wouldn’t want to do it more than a couple times a day, especially if its a large structure.
  • Another caveat is that local deletes won’t be reflected to the remote (the files that were deleted locally but exist on the remote would just show up as placeholders), so you would probably need to add an additional step after the mount and sync to delete any placeholders and then empty the odrive trash before unmounting…

We have an advanced Premium feature that allows you to define a list of items to exclude. It sounds like you are looking more for a way to be inclusive rather than exclusive, but maybe it would work well enough for your needs: https://docs.odrive.com/docs/advanced-client-options#blacklistcontains-blacklistextensions-blacklistnames-blacklistprefixes

It’s all a bit “hacky”, I’m afraid, but definitely interesting.

a big noisy information regarding sync and backup, although i am always confused about differences between sync and backup

Hi @perthwebmedia ,
Did you have any specific questions that I can answer?