CLI Syncing Scripts

[arvin]

For CLI command is there a way to set sync to include subfolders and items similar to right click? I’m using a parsing method with cli that does it per item, but… it’s much much much slower than right click - sync - include sub folders and items. I’m guessthis has something to do with doing stuff in parallel.

Any solutions?

[Tony]

Hi @arvin,
You can use some basic scripting to perform a recursive sync: How can I force a recursive sync (download) with the CLI?

I also have a fork of the CLI that implements a recursive sync function: GitHub - amagliul/odrive-utilities: Utilities for odrive

A couple of Linux-based scripts from @sterlingbutters

CLI Syncing Scripts

There is probably something like this already existing but thought I’d share my solution for easily syncing oDrive files and directories: “Top Level” Sync: https://gist.github.com/SterlingButters/c9a12b6617977ca1334d3ec060297839 Recursive “deep” sync: https://gist.github.com/SterlingButters/11c132d757304d4a1262c3453d0d8583

Additional resources:

Odrive Sync Agent: A CLI/scriptable interface for odrive's Progressive Sync Engine for Linux, OS X, and Windows

Linux/MacOS bash-based: This uses find to recurse and sync. The output is not ideal, though, since it is all queued up. You would need to change the path from ~/odrive-agent-mount/Dropbox/ to whatever you require, of course: output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -exec python "$HOME/.odrive-agent/bin/odrive.py" sync "{}" \;); echo $output; done This second one uses an extra file descriptor trick to push the output out in realt…

Odrive Sync Agent: A CLI/scriptable interface for odrive's Progressive Sync Engine for Linux, OS X, and Windows

For your backup and unsync use case, you can script something like this in bash: #!/bin/bash agentdir="$HOME/.odrive-agent/bin" export agentdir agentmount="$HOME/odrive/Dropbox (Personal)" targetdir="$agentmount/temp" sourcedir="$HOME/Downloads/test_directory" python "$agentdir/odrive.py" sync "$targetdir.cloudf" cp -R "$sourcedir" "$targetdir/" python "$agentdir/odrive.py" refresh "$targetdir" echo Waiting for odrive to start syncing $targetdir ... while [ "$(python "$agentdir/odrive.py" syncs…

[arvin]

Hi Tony - The reason I ask is actually not because of trouble recursing. I’m getting the inside folders fine - but rather for speed. I tried to assign them as jobs, but as far as I can tell they’re still syncing one file at a time.

Currently if I do a right click sync for a folder of about 50 items takes about 1 minute. On the other hand a recursive sync takes me about 3 - 4. Do the recursive methods help in terms of speed? Thanks a ton for getting back to me on a weekend Tony

[Tony]

Hi @arvin,
With the method I have in my CLI fork it is sequential, so it will not perform concurrent downloads on its own. The sync engine can perform concurrent downloads with the CLI as the interface, but it amounts to just making multiple requests at once.

Which OS are you scripting on?

[arvinmoses]

Hi Tony - sorry Iwe had a web site launch and got a bit packed. What you’re saying makes sense. I’ve tried some of this but with no success - basically doing concurrent calls to sync. I’m currently using powershell with the executable cli - not python based. I tried running it both as a job and workflow and had no success but I may be just unfamiliar with how to properly execute it.

Thank you for the help.

[Tony]

Hi @arvinmoses,
As you stated above, something like jobs should work, where multiple instances are kicked-off, although I haven’t attempted it myself.

An easy way to see how this could work is to just manually run the command in multiple windows, simultaneously. For example, use the command here (How can I force a recursive download with the CLI?), open a couple of command windows, and execute the command. You will see that the odrive sync engine will process those commands simultaneously, and multiple files will be downloaded at the same time.

[sterlingbutters]

There is probably something like this already existing but thought I’d share my solution for easily syncing oDrive files and directories:

“Top Level” Sync: https://gist.github.com/SterlingButters/c9a12b6617977ca1334d3ec060297839
Recursive “deep” sync: https://gist.github.com/SterlingButters/11c132d757304d4a1262c3453d0d8583

[Tony]

Thanks @sterlingbutters!

I moved your post to this thread and added a reference to it to keep the information in one place.