Pool Storage, Parallel IFS, and Parity. Oh my!

So here are a couple of ideas for you to consider. :thinking:

Pool Storage providers
I would like to have odrive use multiple providers but to appear as a single cloud storage on my desktop. I don’t really care where in the cloud they are going, just that they are up there and safe.

For example, say I have the basic cloud accounts from Google, Dropbox, OneDrive, and Box. These together add up to about 45GB but they could range individually from 1 to 15 GB. Normally I have to be aware of how much space is in each and can only upload a file that would fit.

It would be great if I could configure a group of linked accounts on the website that would appear on my desktop as “Pool Storage 1”. Behind the scenes, odrive would manage what files are uploaded to whichever storage account has the available space.

I would just like to ask that there is the ability to set a capacity limit, say 80%. I wouldn’t want odrive to fill up my primary Google Drive so that I’m unable to use other services like Gmail or Google Docs.

Pretty neat idea, huh? :grinning: Wait, it gets better.


Parallel IFS
So if odrive could pool the storage spaces, it would not have to upload all segments of that file to a single provider. Using the IFS feature, it could round-robin the segments among all of the storage providers in the pool. It could also run the upload/download transfers in multiple parallel streams. This would increase the transfer speeds and you would not be limited to the speeds or allowed number of connections to a single provider.

So that would be awesome, but what if one of the providers are down? I wouldn’t be able to restore a file. :worried:
Well then, let’s talk about…


Parity
With IFS segments split across providers, you could include an option to generate parity stripes which could be used to repair or replace missing or corrupted IFS segments, just like a RAID.

So for example, I have a 270MB file that is added to the pool storage folder, which is configured for 4 providers. The file would be IFS split into 3 segments of 90MB each and a parity stripe would be created. These 4 files would then be uploaded in parallel :astonished: to the 4 storage providers.

If any one of those providers is unavailable, odrive would use the 3 available segments to recombine back into the original file.


Now close your mouth and take a deep breath. :triumph: Let’s talk about how to make these suggestions happen!
:tada: :fireworks:

2 Likes

Appreciate the ideas @Smudge!

Treating a user’s linked storage as a single namespace is an idea we’ve floated, but its not something we are planning to pursue, at this time.

I agree it opens up some interesting possibilities and the underlying ideas parallel some of our own general thoughts of how storage should be viewed.

1 Like

The principle sounds similar to stablebit clouddrive product. Might be worth a look. You can definitely pool cloud storage using that but last I looked it was still in beta. Gives options for on the fly duplication etc.

Give the boy an extra Gb of data, awesome idea for a uber premium service.