Creating backups piece by piece question

Hello,

I’m going to provide a bit of background to better help understand my question… I have a freenas rack server that has about 16 terabytes of storage. I was not able to get a backup going from there directly so I am doing it from my windows machine. Sadly the backup process required that I have as much free space on my windows machine as on my server to back everything up at once. So instead, I bought a 4 TB external to move data in chunks to encrypt and upload, that data then would be removed from the 4 TB drive once it was on ACD. My question is, if I do it this way how can I re-download my data and decrypt it using odrive? Is this possible to do this way? Thanks in advance for any answers!

1 Like

Hi @dj16921,
This sounds like a pretty massive backup job, and I tend to recommend that folks wait until our backup feature be released for something like this, since sync tends to be too heavy of a process for a traditional backup use case. A little more on this here: Backup (one-way sync)

Additionally, we have a new iteration of encryption coming that will make encrypting and decrypting data more flexible and easier to use.

Currently, you can access your encrypted data from the Encryptor folder. Since you are already using encryption you are familiar with Encryptor, are you asking how to access an Encryptor folder in another location other than the odrive folder? You cannot do that, currently, but our next version of encryption will allow encryption in other locations.

1 Like

Hey Tony,

Thanks for your reply.

Its kind of hard to explain what I want to do. I take chunks of my data and encrypt, sync to odrive / ACD, then delete my local copy in my temp folder (which is encryptor folder) to add more data to encrypt, sync, and repeat the process. My question is how can I easily access the data sync’d to ACD that was locally removed in encryptor (I realize this isn’t a sync I’m trying to do more so a encrypt / upload) I look forward to reading more about the backup feature. Is there any way to sign up for a beta? Also, do you have any sort of ETA? I really like the way odrive looks and functions for syncing and would be interested to see the backup feature (which seems is more what I need :slight_smile: )

Hi @dj16921,
It sounds like you are using an Encryptor folder, but it is unclear how you have separated it from the default odrive folder. Are you using a symlink/junction?

Are you using a single Encryptor folder, or creating and then removing Encryptor folders as they finish sync?

I would say you don’t want to delete the items in Encryptor. This is a sync engine, after all, and you wouldn’t want the deletes to be synced back to the cloud, which would delete the remote data. odrive has the odrive “trash” concept to prevent inadvertent deletes, but the trash will start filling up and could create increasing overhead. Plus, when you delete the items locally, they will be removed from the local view, even though they will still exist in the cloud until you’ve emptied the odrive trash. I have a feeling this may a major component in you asking this question in the first place :wink:. Do you see many items listed in the trash from the odrive tray menu?
More on the trash here: https://docs.odrive.com/docs/sync-changes#section--delete-existing-files-

Unsync would be a better option here, especially if you can unsync entire folders of content as you pushing things up via your temp folder.
More on unsync here:

I know I’m throwing out a lot of terms and concepts that you may not have fully explored yet, so it may be best if you can give me the lay of the land, so to speak, on how this is setup. The way you describe it may not be the best way to go about this, but I also might not fully understand the setup.

Hello,

Yeah thats kinda how I was thinking of doing it. Obviously before I delete it locally on my temp drive (the 4TB drive attached to my windows system) I would unsync it as to not lose it in the cloud. My question is, if I ever wanted to retrieve that data again, how would I do that? It wouldn’t be in the encryptor folder and from what I’ve seen I can’t tell odrive to sync something in ACD that isn’t in the encryptor folder. Would this be done by leaving the parent folder in the external drive that encryptor is on?

To try to explain again, I have a 16TB FreeNas server that I’m copying data from to a 4 TB external attached to a windows system that has odrive encryptor folder on it. I am then syncing that content to ACD. Once it is sync’d I am then unsyncing and removing the content from that 4 TB drive to bring more data over to upload to ACD. My question is, when I go to get this data back, how would that be done via odrive? I realize I’m not using the program the way it is supposed to be since it is a “sync” program and not a “backup” one… yet :stuck_out_tongue: but if this is possible in its current state that is how I would like to go about it.

Hi @dj16921,
Encryptor is essentially a virtual view overlaid on top of some part of your remote storage. If you are encrypting content with Encryptor you need to access it via an Encryptor folder too. So here is how I imagine it working:

Encryptor folder -> Folder A/B/C/D/etc. containing your data to be uploaded.
Folder A/B/C/D would be the folders you are creating for each “chunk” of your upload.
Once a chunk is done encrypting and uploading, you unsync (no deleting) that A/B/C/D folder
After all is done you can always access that data again through the existing Encryptor folder, or create a new one that points to the appropriate encrypted place on your remote storage.

Keep in mind that odrive doesn’t officially support junctions/symlinks or moving the odrive folder to an external drive, so there isn’t a supported way Encryptor folder onto the external drive, at this point.

The backup feature should be available within the next 8-12 weeks. I know its still a ways out, but it is what I would recommend using for your backup job since it is built specifically the use case.

Yeah I can’t leave the data in the temp drive unless I had a 16 terabyte external to hold it all, otherwise I couldn’t back up all of the data. Looks like I’ll have to wait for the backup feature. I look forward to using it

To clarify, just from a feature education standpoint: assuming you eventually had all of that data in the cloud, regardless of how it got there, you could then navigate it all through an Encryptor folder without needing any of the data local.

Placeholders and progressive sync would allow you to browse the entire, remote-located 16TB, without taking up space on the local drive. If you wanted to pull something down, you could download that specific items without needing to pull down the whole lot.

More on that here: https://medium.odrive.com/sync-differently-b993694e1544