Mount vs backup because of high CPU usage

I Have a Question
1

I am currently using the CLI with 8 mounts to backup certain folders of my linux server to Amazon.

CPU is always at 100% with the top 5-8 entries being the odriveagent (htop shows me 10-20 entries for odriveagent in total).
I checked the status, there is nothing syncing right now.

I could not find anything here regarding CPU usage, besides the tip to use backup instead of sync to lessen the burden. This would work for me, I only need one-way backup.

So this leaves me with a couple of questions:

  1. Is there a way to fix the root problem (CPU usage)?

If the answer to 1 is “no”, then:
2. is “backup” going to be a “one-shot”, or will it monitor changes on its own?
3. how do I change my mounts to backups without reuploading everything?

Many thanks in advance!
Andreas

2

Hi @andreas1,
Sync can be a heavy process because it is constantly trying to keep both sides in sync, scanning and monitoring both the remote and local locations. If you have sprawling structures/lots of objects to track it can create some heat.

The backup feature in the agent is a really early prototype. There are folks that use it, but it isn’t something we ever released as a finished product.

Reducing the amount of work that odrive has to do can reduce CPU. This can include partitioning data out so odrive only has to scan a subset, or unsyncing folders to collapse the structure.

You could also try something like nice/renice on the agent process to reduce its priority. This may not prevent 100% CPU utilization, but it will ensure other applications get priority if you have other apps battling for CPU.

There are other utilities that can try to keep a process under a CPU ceiling, like cpulimit.

I have only user nice/renice with odrive, so I can’t say for sure how cpulimit works with agent, but you could give it a shot. Here is an srticle going over these methods: Restricting process CPU usage using nice, cpulimit, and cgroups | Scout APM Blog

Backup is continuous and will run every hour.

Unfortunately you can’t. Backup is really for a “fresh start” configuration, where you are uploading local content to an new remote folder.

3

Hi Tony
Thanks for the quick answer. The necessary fresh start for backup is quite unfortunate, so I’d like to understand backup a bit better before I commit to reuploading everything.

  1. So backup will only generate load once an hour on scanning the local disk for changes, then uploading these changes? That should drastically reduce load, correct?
  2. Can I use encryption with this?
  3. As it is an undocumented feature, any danger that it will disappear in a future release?

Thanks!
Andreas

4

Hi @andreas1,

Backup is much simpler and it doesn’t perform continuous tracking, so the load should be reduced by a fair amount.

The backup feature in the agent doesn’t support in-line odrive encryption. You could use another type of encryption on the files before having the agent back them up.

I don’t expect it to be removed from the agent, but I can’t guarantee it. It is highly unlikely.