Family (Home) network - best setup for odrive using Amazon Drive‏

I tried this by email first but haven’t heard back so maybe forum is a better venue.

One line summary: Hoping YOU can tell me the optimal way of setting up odrive folders to work with Amazon Drive with multiple family members and devices.

As you will see below, I have spent a very long time on the forums already trying to figure this out, but with limited success. If there already is a thread for this, with it clearly summarized, then please just let me know.

I’m trying to set up a family/home network and want to get it right the first time. I think I see where Tony suggests “a separate odrive folder for each user and using the default locations that are created during the initial odrive run for that user”. I was hoping someone could help me expand that into simpler terms and steps. How do I set up a “separate odrive folder for each user” and what is meant by the “default locations that are created during the initial odrive run for that user?”

Let’s try to simplify my situation. Let’s say I have just three family members using 3 different laptops and 3 different external hard drives in various combinations, but want to keep one odrive (O1) and one Amazon Cloud Drive (ACD1) account link.

U1
U2
U3
L1
L2
L3
H1
H2
H3
O1
ACD1

U= user
L= Laptop (hard drive)
H= External hard drive
O= odrive account
ACD= Amazon Cloud Drive

For the most part, U1 uses L1 as his primary laptop, and usually uses H1 there. Sometimes, however, U1 also attaches H2 to L1. At other times, he may use H2 on L2. And so on. Until U2 comes along and jumps back on his machine, L2, which still has H2 attached.

U1, L1, H1
U1, L1, H1, H2
U1, L2, H2
U2, L2, H2

I currently only have one UID for odrive (O1). Is the recommendation that each user should have their own UID for odrive with concomitant folder, and need to log in separately each time they access a machine? That will be too much to ask I think you’ll agree, or certainly too much to expect to be able to rely on happening.

Assuming the entire family can use the same odrive UID (O1) with persistent login on all machines, then are you suggesting the single odrive folder with sub-folders for each user?

L2/odrive/U1/Documents …/Movies …/Pictures …/Music
L2/odrive/U2/Documents …/Movies …/Pictures …/Music

Then ACD1 would also have separate folders based on User:
ACD1/U1/Documents …/Movies …/Pictures …/Music
ACD1/U2/Documents …/Movies …/Pictures …/Music

But I thought Tony was concerned about syncing across all devices, so there would be concern with the placeholders when viewed by U3 back on L1:
L1/odrive/U3/Documents …/Movies …/Pictures …/Music

Or if they go into U1’s folders to collaborate:
L1/odrive/U1/Documents …/Movies …/Pictures …/Music

So perhaps it would be better to keep separate folders by machine as well, which would then be synced across all machines, with every machine seeing all folders?
ACD1/L1/U1/Documents …/Movies …/Pictures …/Music
ACD1/L1/U2/Documents …/Movies …/Pictures …/Music
ACD1/L2/U1/Documents …/Movies …/Pictures …/Music
ACD1/L2/U2/Documents …/Movies …/Pictures …/Music

Finally, does U1 need to keep H2 only connected to L1 for sync, or will odrive recognize and sync the folders on H2 regardless of what laptop U1 attaches H2 to?

So… I think I’ve given enough of a start here to all the possible combinations so you know where I’m going with this. This doesn’t even count the phones, which will have to be synced separately with ACD1 outside of O1 using Amazon’s phone sync app.

Before I get too far down the rabbit’s hole on my own, perhaps someone with intimate knowledge of odrive could summarize for me (and for other users’ benefit here on the forum) what YOUR best recommendation is for setting up such a family/home network, if you were doing it.

Thank you!!

Appendix: Blog References to show you where I’m coming from based on research on this forum, but which I couldn’t make sense of:


One thing I did want to address, though, was the notion of creating a pro sync folder to the same target on multiple machines. This particular scenario is one that I don’t recommend because you are now sharing what I will call “odrive local space” across systems. This may cause some unpredictable behavior. It is actually a similar question, in principle, to this thread here:


I strongly recommend having a separate odrive folder for each user and using the default locations that are created during the initial odrive run for that user.

Chances are that each family member is working on different files, which odrive facilitates better than anyone else with “progressive sync”. You’ve probably already seen this in action, but odrive allows you to sync exactly what you need and nothing more. So, each user can navigate directly to the file they need and work on that specifically, without downloading anything else. This is different than Dropbox, where everything is downloaded by default, regardless of if you need it or not. The result is that you only have duplicate data if users are working on the same exact files.

I realize there may be instances where users will want to access the same files (shared media, for example), but I can’t recommend pointing separate user instances of odrive all to the same odrive directory. Honestly I’ve never tested it, so I can’t say for certain what the result would be. Just off the top of my head, I can imagine some trouble when dealing with placeholder files.

For example, user A expands folder1, and placeholder files are laid down. User B’s odrive instance now sees a new folder appear and foreign placeholder files (it did not create these). Since it did not create the placeholder files it remove s them and lays down its own. User A’s odrive instance may see the removal of the placeholders before the new placeholders are put down and see that as a delete of the file. The item will end up in the odrive trash for user A. Then it will pick up the new placeholder file that User B’s instance put down and see that as foreign and delete it… etc… etc… Since this would be timing-based, it may eventually work out, but I can’t say for certain.

So, bottom line is: don’t smash all the odrive instance folders together. The good news is that odrive has enough safeguards that you shouldn’t lose any data, even with the different instances trying to reconcile all of the foreign local changes that are happening. You could end up seeing stuff randomly appear in odrive trash, though, and other strange behavior…



Pro Sync folders are a “local” construct. They are relationships created between local folders and remote folders. This means they are not proliferated across client instances. It would be unwise and potentially disastrous for us to assume that the relationships created on one local system are going to be the same on the other systems you use for odrive. In most cases it makes no sense to do so. I could argue that in all cases it doesn’t make sense to do this because of the potential for user confusion, odrive confusion, and general unexpected behavior. I covered this a bit already in this post: General Sync Questions and Observations

Since you chose to replicate the local pro folder relationships on a new device, that client now has to not only scan the entirety of the local structure, but it needs to scan the entirety of the remote structure to determine what exists in both areas, what needs to be synced, and what doesn’t. There is no general equation to determine the length of this process. It will depend on the number of folders, the number of files, the speed of the local source (network mounts will be much slower than physical mounts) and the speed and rate limiting of the storage source that is being interrogated for every file and folder.

My recommendation would’ve been to leave the system that had already done the 2TB of imports as the single importer. If you plan to switch again you can expect there to be overhead, as the client will have to figure out what the current state of everything is locally and remotely. Again, I also want to strongly caution against running both Pro Sync folder setups concurrently when they are pointing to the same local storage.



Pointing your local Documents folder on multiple systems to the same folder in the cloud is actually safer because you do not have to worry about multiple odrive instances acting independently, and more importantly, reacting to what the other instances are doing in the same local location. With the containerized local folders (like Documents or Pictures on your desktop or laptop) sync will do its thing. You will run the risk of overwriting documents that have the same names, but everything else should work out okay. It would just create a confusing resulting dataset if you aren’t prepared for it. Even though that is a cool and potentially viable use case, we do not want to push out those Pro Sync folders to every system without the user’s explicit say-so.

Wow! That’s quite a write-up :wink:

There are a lot of puzzle pieces here, but it is not entirely clear to me where they all fit, so I’m going to start by clarifying a few things with some questions.

It sounds like you use common machine logins for everyone (a single machine user). Is this the case and does it apply to all devices?

What is it you are actually using Amazon Drive for? Is it for backup, syncing, remote access… all of the above? How does it factor into your user’s daily work flow?

External hard drives are mentioned, but it is not clear how they are used or how they relate to Amazon Drive and odrive. Can you explain that further? I’m guessing that these hard drives are currently the way that your users are keeping their data “containerized” within the common systems, although it seems like users can share the external hard drives too?

Are you ultimately trying to get all of the user data (non-application, non-OS data) into Amazon Drive and then work out of there? Or, are you trying to target only specific user data to be in Amazon Drive, while keeping other user data local-only? Why?

My responses in BOLD:
Thanks for replying so quickly!

It sounds like you use common machine logins for everyone (a single
machine user). Is this the case and does it apply to all devices?
our laptops all have set machine logins, for example, U1 on L1, but everybody who works on that laptop uses that same machine login or guest. So on L1, if I use it, I log in as U1. If odrive sync (O1) to Amazon Drive (ACD1) would work better with multiple machine logins, then I’m open to setting that up on each machine. Maybe that’s the piece I’m missing. But some laptops, like L3, do not even require a password and so everyone is using U3 machine login on that one. Again, I’m open to making everyone use their own login on machines.

What is it you are actually using Amazon Drive for? Is it for backup,
syncing, remote access… all of the above? How does it factor into
your user’s daily work flow?
I am primarily using ACD1 for syncing and backup, and occasionally would access ACD1 for remote access when stuck on someone else’s computer and needing a file. I want everything synced in case of catastrophic hardware failure and for remote access and while simple switching machines within the house. For backup, I am working out a system of regularly saving TM backups to ACD1 using disk images of the TM partitions, maybe once every quarter or so. The TM backups are on external drives again to protect against catastrophic hardware failure. If L1 hard drive dies, having TM there doesn’t help me, so TM is on the external. Also, the local hard drive for L1 is too small for TM of all my son’s movies, another reason why TM is on the external drive.

External hard drives are mentioned, but it is not clear how they are
used or how they relate to Amazon Drive and odrive. Can you explain that
further? I’m guessing that these hard drives are currently the way that
your users are keeping their data “containerized” within the common
systems, although it seems like users can share the external hard drives
too?
See above, TM is on the external drives as well as a “Large Files” folder for items like 9 GB movies that quickly cause storage problems on the laptop itself (MBP with only 20 GB storage), especially if you try to TM locally. One external drive H1 had 2 TM instances on it for two separate laptops, L1 and L2

Are you ultimately trying to get all of the user data
(non-application, non-OS data) into Amazon Drive and then work out of
there? Or, are you trying to target only specific user data to be in
Amazon Drive, while keeping other user data local-only? Why?
The former. I plan to trust ACD1 as a central location with all user data, some of it encrypted. ACD1 would be the central source, and most local data would be placeholders. Video in progress would have to stay local of course, but in this case “local” meaning on the external hard drive H1, because you can’t work with it truly “locally” on L1 internal drive. Make sense?

I am completely open to the idea that I am going about the ENTIRE thing wrong, so hopefully these answers to your questions can help isolate that if true, and point me in the right direction overall. I feel quite lost at times trying to keep it all straight.

I’m sure these answers will spawn new questions, so I look forward to your reply.

Busy day today, but I plan to revisit this thread “tomorrow” (later today)

I know I’ve been keeping you busy on another thread, and also that I’m not your only customer. :wink: But as there is a 4-day lag on the other thread, was hoping you might have time to consider this one. Once we’re done, I would think this would be a great blog entry and selling point for odrive, with a straightforward suggestion for any other users with multiple devices, even if they choose to modify slightly for their own purposes. thanks

Don’t worry, I haven’t forgotten about this. :slight_smile:

This became a looong post, so look for the TL;DR summary at the bottom if you get bored

We think odrive is the way you should be interacting with your storage. This means all storage. Given that, I will always generally say that each user who uses odrive should have their own account, so they can link everything they have (ACD, Facebook, Gmail, Instagram, Google Drive, etc… etc…) and maintain control, security, and privacy over that storage. In a shared-user setup this can’t be done. With today’s hardware and current OS versions, switching between machine users on the same system is easy (i.e. fast user switching), so this shouldn’t add too much complexity and it has a lot of benefits.

Putting that aside for the moment, and concentrating on the current use cases, it sounds like you are in need of centrally located and accessible local storage that can be used from all of your disparate devices. I see a lot of complexity in the current shuffling of external drives across the 3 different systems. Right now it sounds like you have large videos, Time Machine backups, and other things distributed across the 3 drives, and those 3 drives can be mounted on any of the 3 systems, used by any of the 3 users, at any given time. I’m actually wondering how you are keeping track of the data on these drives, as they are moved around and data is added to them.

Typically, in an environment where you are wanting data that locally accessible to multiple systems, you would setup a dedicated system on the LAN to cater to all of the local devices. A NAS is what comes to mind, but I suppose you could also do something like construct a cheap, network accessible external drive hub with a powered usb hub and a Raspberry Pi (as an example).

Let’s set that aside, too, for the moment.

I’m struggling a bit to try to boil down your use cases and environment into something simple. Here is my attempt, based on what I understand:

  • You have cloud storage that you want to use as a primary backup destination for your user data for 3 users and your Time machine backups from 3 systems.
  • You want to also be able to use your cloud storage to “work out of”, once you have it all in the cloud.
  • You are currently using a variety of external drives to store your Time Machine backups (which you want duplicated in the cloud) and some user data.
  • You are using the same variety of external drives to house a “shared” library of movies (it is not clear if you want these in the cloud too).

Given the above, here are some of my thoughts. I’m sure some will be met with resistance/clarification/additional insight, but it should at least help move the conversation forward:

As you get closer to achieving the goal of getting all of your user data in the cloud, the need for traditional full backups becomes less and less. Conceivably, once all of the user data you care about is safely tucked away in the cloud and you are working against it in that form, directly, you no longer need to continue with full backups, like Time Machine offers. The reason is because it becomes redundant. You are storing the same data in the cloud, just in a less accessible, less precise form.

Admittedly you may want to backup the “system state”, with applications and application data, to make it easier to spin up a “working” system after a catastrophic failure, but it is not essential. In fact, you could make an argument that it would take less time to just install the apps again instead of trying to recover them from a backup. At the very least the rate at which you are backing up your system should be significantly reduced. Keep in mind that the assumption here is that all of your user data is in the cloud already. This means it is accessible from any system at any time. It is no longer anchored to a particular system or hard drive. This makes computers “disposable”, in the sense that they are just dumb tools used to access your data.

This is actually how I currently have my own systems setup using odrive. I can lose any of my laptops/desktops and it doesn’t matter, from a data perspective. Because of this, I don’t perform any sort of full-system backups: I just don’t need to.

If you were to follow this type of setup, you could eliminate the Time Machine shuffle you are doing right now, or at least reduce it.

Taking that leap, I would say the next step would ideally be to get all of your users on their own system login, with their own odrive account. As they jump on a system, they fast-switch to their own user and have a storage experience that is specific to them and their current needs. Using odrive’s concepts of dynamic sync and placeholders allows users to target specifically what they are working on/looking for. They can have the same exact view of their data without any dependencies on what current external drive is connected or what system they are currently using

After that, the final piece of the puzzle is the local access to large-sized data (the aforementioned movie library). Since this is data that consists of very large, single files and is actively (and somewhat randomly) accessed, it is not practical to use it directly out of cloud storage - at least not yet. This type of data with this type of access needs a local repository. I mentioned previously that something central, like a NAS, would be ideal here. Even the series of external hard drive would be okay, provided the users understand their contents and can mount them on-demand. In any case, this particular data and data access lies outside of the odrive sync scope, in my mind, and can be addressed independently of the odrive use cases above.

TL;DR Summary:
To summarize my (rose-colored?) thoughts:
Get all of your local user data into the cloud, link all your cloud storage, and then work on that data from the new cloud-based/backed, united, sync view (via odrive). In doing so you can reduce the complexity of the environment substantially. User data becomes independent. No need to worry about specific laptops, no need to worry about external hard drives, no need to worry about traditional, full-system backup.

Your digital life becomes simple :slight_smile:

2 Likes

Wow! First of all, thank you immensely for this thoughtful and thorough response. This definitely “helps move the conversation forward.” Thank you!

I think first off I need to go with the different user logins, to keep the user experiences distinct. I never saw much of a compelling need for that before, but given how fundamental this storage experience is, and ubiquitous, I think I now have a reason why to set it up and enforce it as a minor inconvenience.

With that said here are the steps I see I must follow, as well as any one else considering a similar setup:

  1. Set all users up with separate logins on each system.
  2. Set all users up with separate odrive accounts.*
  3. Set all users odrive accounts to sync to the same Amazon Cloud Drive account.**
  4. Get all local user data into the cloud via odrive sync from all devices, per normal odrive usage.
  5. Unsync data not needed locally, leaving placeholders per normal odrive usage.
  6. (Optional) Set up router (Airport Express in my case) with large attached external hard dive for any central storage accessibility needs. Or purchase actual NAS.
  7. (Optional) Continue using local external hard drives for large files and physical portability of local access files.
  8. Enjoy a simplified, digital life! :slight_smile: ***

Let me know if I’ve missed anything in this summary of your post, and I have a few followup questions as you predicted:

_ * I use premium features like throttling and splitting. I see no family pricing for odrive yet. At present, as much as I love odrive so far and am willing to pay for one premium account, this approach would cost me $500/yr, an amount I can’t afford and which is almost unthinkable in today’s low-cost app world, where people think twice about paying $1.99 for an app. odrive is so much more than an app, which is why you have my $99, and so far, gladly, especially given the attention my posts have received. But multiplied times 5 family members will make this approach infeasible for me. In short, is there a way to use odrive with a single account and take care of my challenges? Or maybe use just one premium account for the large file throttling and splits, and all the rest free accounts? (That seriously diminishes the functionality, here, and without Unsync, I think it hobbles the entire proposal.) I hope you take this conversation as potentially helping your long term business model, and not as unfair criticism. I want odrive to thrive so I can enjoy it’s benefits into the future.

**Can multiple odrive accounts be pointed to a single Amazon Cloud Drive (ACD) account successfully? Would each users experience be differentiated by the folders they set up for themselves, and could they still access other shared folders as well? Could some users be shut out from some private folders, just like access on a company network and NAS? If not, again, separate ACD accounts would become exorbitantly expensive, at around $160 each for Prime plus unlimited. That’s an extra $640.

***Finally, part of TimeMachine ™ “backups” includes versioning. You address current state storage above correctly pointing out that the laptop becomes disposable with you having all your current data in the cloud already. What do you suggest however for versioning, also an important part of any “backup” plan and certainly integral to the TM paradigm (to go back in time)? This is critical for my sons when their large school reports and videos get corrupted occasionally (but always perfectly timed to have the worst impact possible). :worried:

Thank you again so much for all this consideration and for your suggestions. It has indeed helped move things forward so far, I just need help exorcising the few remaining ghosts in the machine. :grin:

Hi Tony,

Anything further on this? Now that all the time has passed proving sync unsync resync and restoring from a large encrypted dmg file successfully and using IFS successfully with Amazon, I’m ready to start setting up the family network. Look forward to hearing ay further suggestions you might have.

Hi Paul,
You bring up some good points. I agree that the cost of buying premium for everyone in the family is not really going to be viable for most people. There may be a solution that can be conceived of that doesn’t require 5 premium accounts though.

As for sharing an ACD account, technically it is possible via odrive, but you would probably have to inquire with Amazon on how they feel about that. My expectation is that they probably expect it within a household, much like they allow Prime sharing with your household. If it was used this way, there wouldn’t be a good, built-in way to separate access.

However, you could use some odrive features to do so. Using encryption can at least protect the premium account holder’s files and keep them from being accessed by anyone who doesn’t know the passcode. Spaces could also be used to share out storage to users, carving out specific pieces for their use. Creative use of these two features could open up some new possibilities.

The versioning is a good point. Versioning can become important in a backup scenario. Our upcoming backup feature will support versioning and may be of use to you as an added tool for your overall solution. In the meantime, it sounds like you will want to keep some amount of “frozen” backups, in case “bad things happen” and unintended changes occur that you need to revert.

What is the current state of your solution? I am curious to know if the features mentioned above give you any more ideas on how you can set things up to your liking.

Paul, I just wanted to add my family’s support to your situation. We have a very similar situation in our household and, in fact, I’m surprised that there aren’t many more families in some type of similar solution. Families have important and personal files (pictures, tax returns, etc.) that they don’t want to lose or to be openly available. As children get older, they don’t want siblings or (sad to say, but always true) parents meddling in their stuff either.

Soon you end up with a small company scenario with multiple users having the need for individual and shared file storage. So, thank you again for sharing your story, which may be more common than we know.

Glad this situation resonated with you. I too am surprised that more people haven’t mentioned it here, and I do think this could be a clear differentiator for odrive if they were to pull it off. I think it could be easily achieved with a family membership option. I’ll let you know how I make out as I try to set this up.

Let’s just hope that Tony or someone from odrive recommends a solution.

I think there is a solution in there somewhere with a combination of Encryption and Spaces where you can pull this off with a single paid account, but it may require a little bit of thinking.

Did you guys see my previous comment with regards to this?

Any thoughts in this area?

This situation Paul and Chris describe is similar to that of our hopes for odrive as a business solution - with the added complication of using free accounts instead of all paid. So far, we have tried to deploy odrive on to several employees all of whom have multiple computers. Odrive is also running on a few servers containing master copies of shared files. The majority of our cloud data are encrypted by odrive. That was one of the main attractions of odrive; not only does it eliminate dependence on any one particular cloud provider, but encryption and sharing support come along with the package.

You can share encrypted folders using Spaces. I can’t speak to what restrictions, if any, the free version of odrive imposes. I’d worry about the ability to unsync. A problem we have run into with all odrive encrypted folders is that odrive does not sync remote changes to data reliably. It doesn’t matter whether the folder is shared with multiple users via Spaces or not. Odrive reliably propagates changes made to a file on computer #1. Any other computers that use odrive to monitor the encrypted share rarely if ever automatically pick up the change. Making odrive download the updated file requires manually browsing to the folder (Win 10, OSX El Capitan sometimes) or performing a right-click/odrive refresh (Win 7 always, OSX rarely).

If you alter the shared file on a second computer before odrive picks up the remote change, you generate an odrive (conflict) file, Unfortunately at that point, the (conflict) files are not consistent across computers. If you are dealing with something like a Word document you can spend enough time swearing at the files to merge changes. If it’s an image or other binary file, you’re screwed.

If you are willing to manually sync files on each computer, the process works. It may be possible to automate the syncs using odrive’s CLI (Command Line Interface). However that is a band-aid over the underlying problem; whether it’s something you want to tackle is another issue entirely. The actual fix needs to wait until odrive solves monitoring remote encrypted shares for changes.

@paulschroeder1: If you are looking for a Time Machine-like cloud backup solution, investigate Arq Backup. It isn’t a true system backup; you can’t restore a full computer from the backup set. It does let you save all your documents to any one of a number of cloud providers. Zero-knowledge encryption is baked in as is data validation, deduplication (only uploads changed bytes of each file rather than the full file), etc. Unlimited versioning is also supported as is a retention policy like Time Machines (once an hour for a day, once a day for the past week, etc.).

1 Like

Thank you, I will check that out, but manual sync might be fine for my needs for now. Will see.

I’m looking into it on my end, but sorry, slow going for me with large gaps inbetween efforts. Hope to do more this coming week.