79 posts were merged into an existing topic: Odrive Agent Troubleshooting, Feedback, and Questions
Linux/MacOS bash-based:
This uses find to recurse and sync. The output is not ideal, though, since it is all queued up. You would need to change the path from ~/odrive-agent-mount/Dropbox/ to whatever you require, of course:
output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -exec python "$HOME/.odrive-agent/bin/odrive.py" sync "{}" \;); echo $output; done
This second one uses an extra file descriptor trick to push the output out in realtime. Again, you will need to change the path:
exec 6>&1;output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -exec python "$HOME/.odrive-agent/bin/odrive.py" sync "{}" \;|tee /dev/fd/6); done
If you only wanted to sync the files immediately inside the specified folder:
exec 6>&1;output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -maxdepth 1 -name "*.cloud" -exec python "$HOME/.odrive-agent/bin/odrive.py" sync "{}" \;|tee /dev/fd/6); done
Adding parallelization to the mix with xargs, contributed by @stuckj. num_procs = max # of parallel processes (set to 4 in the example):
exec 6>&1;num_procs=4;output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -print0 | xargs -0 -n 1 -P $num_procs "$HOME/.odrive-agent/bin/odrive.py" sync | tee /dev/fd/6); done
Dynamically adjust parallel processing “pool” using SIGUSR1(10) and SIGUSR2(12), contributed by @paradox606:
Decrease parallelism by 1 (down to 1 process): kill -12 $(pidof xargs)
Increase parallelism by 1 (up to max specified in xargs -P): kill -10 $(pidof xargs)
Windows Batch:
Batch via @bojandjuric:
Place into a .bat file
@echo off cd /d %~dp0 set FOLDERPATH="Folder path to sync here" set ODRIVEBIN="odrive cli binary client path here" for /r "%FOLDERPATH%" %%i in (*.cloudf) do "%ODRIVEBIN%" sync "%%i" for /r "%FOLDERPATH%" %%i in (*.cloud) do "%ODRIVEBIN%" sync "%%i"
Windows Powershell
One-liner script (run via a standard command prompt):
powershell -command "& {$syncpath=\"Folder path to sync here\";$syncbin=\"odrive cli binary client path here\";while ((Get-ChildItem $syncpath -Filter \"*.cloud*\" -Recurse | Measure-Object).Count){Get-ChildItem -Path \"$syncpath\" -Filter \"*.cloud*\" -Recurse | % {echo \"Syncing: $($_.FullName)n";& “$syncbin” “sync” “$($_.FullName)”;}}}"`
Python:
I forked the official odrive CLI and added recursive sync here:
1 Like
Will a folder be automatically syncd? What is the interval for agent to check? What does refresh do?
Feature request:
I’d like to use folder for linux backups that will sync to cloud and then unsync from local (when operation is complete) Ideally this would be an option on the folder. So any items put in the folder will automatically sync (ASAP after file is written/closed) and then automatically unsync. I assume this could be scripted but lot’s of crazy polling of status.
Folders within the scope of agent will automatically sync. Refresh will request that odrive refresh the targeted directory, which will scan for changes and sync if any are found.
For your backup and unsync use case, you can script something like this in bash:
#!/bin/bash
agentdir="$HOME/.odrive-agent/bin"
export agentdir
agentmount="$HOME/odrive/Dropbox (Personal)"
targetdir="$agentmount/temp"
sourcedir="$HOME/Downloads/test_directory"
python "$agentdir/odrive.py" sync "$targetdir.cloudf"
cp -R "$sourcedir" "$targetdir/"
python "$agentdir/odrive.py" refresh "$targetdir"
echo Waiting for odrive to start syncing $targetdir ...
while [ "$(python "$agentdir/odrive.py" syncstate "$targetdir" | head -n 1)" == "Synced" ]; do
sleep 5
done
echo odrive is syncing changes in $targetdir ...
while [ "$(python "$agentdir/odrive.py" syncstate "$targetdir" | head -n 1)" != "Synced" ]; do
find "$targetdir" ! -name '*.cloud*' -type file -exec bash -c 'thepath="$1"
echo "$(python "$agentdir/odrive.py" syncstate "$thepath" | head -n 1)"
if [[ "$(python "$agentdir/odrive.py" syncstate "$thepath" | head -n 1)" == "Synced" ]]; then
echo "$thepath" is synced! Unsyncing...
python "$agentdir/odrive.py" unsync "$thepath"
fi
' _ {} \;
sleep 5
done
echo Sync complete! Unsyncing $targetdir ...
python "$agentdir/odrive.py" unsync "$targetdir"
This will first sync the target directory residing inside the agent mount (since it is assumed that it is starting in an unsynced state) and then copy a source directory into the target directory. It will then wait until syncing starts. Once syncing starts, it will begin looking for files that have finished syncing and unsync them if they have. At the end it unsyncs the whole target directory.
I did this quick and only tested a couple of times on OS X, so think of it more as a starting point, but it may get you to a simple solution. It does do some polling, but it it isn’t a problem.
1 Like
odrive sync agent is awesome!
this is pure gold! i always wanted something like odrive. is exactly as described the way it sould be 
here i’ve made few osx bash scripts for some heavy duty amazon cloud drive sync. i need to be in control of what i sync at some point. here’s my magic. use it. adapt it. make it better. share it 
scripts
~/odrive-sync/odrive-sync.sh finds .cloudf folders
#!/bin/sh
find ~/odrive/amzdrive-sync/cloud-sync -type f -name *.cloudf -exec echo odrive sync \"{}\" ";"
~/odrive-sync/odrive-sync-log.sh logs output of sync script and compress the log
#!/bin/sh
log=`date "+/full/path/log/dir/odrive-sync/logs/%Y%m%d%H%M%S.odrive.log"`
echo sync started. check $log
time ~/odrive-sync/odrive-sync.sh | sh >> $log 2>&1
echo sync log
echo -----
cat $log
echo -----
echo compressing log
gzip $log
echo sync completed
usage example
mbp2k9:~ nevrax$ ~/odrive-sync/odrive-sync-log.sh
sync started. check /full/path/log/dir/odrive-sync/logs/20160528143123.odrive.log
real 0m55.131s
user 0m4.013s
sys 0m2.393s
sync log
-----
/secret/path/2014-10 auckland new zealand
/secret/path/2014-10 boat trip riding bicycles gili trawangan gopro
/secret/path/2014-10 gili trawangan sergiu gopro
/secret/path/2014-10 gili trawangan
/secret/path/2014-10 glowing worms new zealand
/secret/path/2014-10 hobiton new zealand
/secret/path/2014-10 hot water beach new zealand
/secret/path/2014-10 kuta bali
/secret/path/2014-10 pool timelapse sankara ubud
/secret/path/2014-10 reverse bungee hot water beach hobiton gopro
/secret/path/2014-10 romantic timelapse sankara ubud
/secret/path/2014-10 sam nada bali
/secret/path/2014-10 sanur bali
/secret/path/2014-10 seawalk bali indonesia extra
/secret/path/2014-10 seawalk bali indonesia gopro
/secret/path/2014-10 snorkeling gili trawangan gopro
/secret/path/2014-10 sydney australia
/secret/path/2014-10 tamaki maori village new zealand
/secret/path/2014-10 taupo new zealand
/secret/path/2014-10 timelapse sankara ubud
/secret/path/2014-10 ubud bali
/secret/path/2014-10 wai o tapu geothermal wonderland new zealand
/secret/path/2014-10 wai o tapu waikite thermal pool gopro
/secret/path/2014-10 wellington nelson new zealand
-----
compressing log
sync completed
check the logs
mbp2k9:~ nevrax$ ls ~/odrive-sync/logs
20160528140745.odrive.log.gz 20160528142143.odrive.log.gz
20160528140758.odrive.log.gz 20160528142211.odrive.log.gz
20160528141344.odrive.log.gz 20160528142217.odrive.log.gz
20160528141407.odrive.log.gz 20160528142454.odrive.log.gz
20160528141639.odrive.log.gz 20160528142508.odrive.log.gz
20160528141640.odrive.log.gz 20160528142515.odrive.log.gz
20160528141652.odrive.log.gz 20160528142547.odrive.log.gz
20160528141655.odrive.log.gz 20160528142549.odrive.log.gz
20160528141732.odrive.log.gz 20160528142553.odrive.log.gz
20160528141757.odrive.log.gz 20160528142557.odrive.log.gz
20160528141904.odrive.log.gz 20160528142612.odrive.log.gz
20160528141924.odrive.log.gz 20160528142829.odrive.log.gz
20160528141936.odrive.log.gz 20160528142907.odrive.log.gz
20160528141943.odrive.log.gz 20160528143118.odrive.log.gz
20160528142002.odrive.log.gz 20160528143123.odrive.log.gz
20160528142040.odrive.log.gz 20160528143223.odrive.log.gz
20160528142112.odrive.log.gz 20160528143245.odrive.log.gz
20160528142118.odrive.log.gz 20160528143332.odrive.log.gz
20160528142125.odrive.log.gz 20160528143407.odrive.log.gz
20160528142129.odrive.log.gz
i’m so happy i can finally do my photo stuff and spend less time on api bla bla stuff or stupid amazon “not for mortals” cloud drive api access key. seems like you need a separate api key and wait for their approval to access your own data. wtf amazon? 
thanks odrive 
1 Like
Thanks for the feedback and for contributing with your own use case and script!
Hi all,
In case it helps anybody, here is a simple recursive refresh script that will recursive run through all your directories and subdirectories and upload any files that might have been changed:
find ~/odrive-agent-mount/ -type d -exec python "$HOME/.odrive-agent/bin/odrive.py" refresh "{}" \;
(change ~/odrive-agent-mount/ to wherever your odrive is mounted)
schedule the task in cron on linux to run it every 15 minutes or so.
I did run into a strange issue where I renamed a folder and the text file in it, and the textfile didn’t upload anymore. Any new text files I added to that folder didn’t upload either. I tried to reproduce the problem with another folder but wasn’t able to. If anybody knows what the issue might have been, let me know!
1 Like
Just want to let folks know that we’ve released the CLI to the desktop client, which many have requested. I just added a blog post that dives into it a bit. The content in there also applies to Agent, so double whammy there. 
We also added a CLI section in our usage guide:
FYI, if you want parallelized recursive downloads, here’s a modification to the recursive oneliner mentioned above. If you have a LOT of files you’re trying to sync this will be much faster.
exec 6>&1;output="go"; while [ "$output" ]; do output=$(find "$HOME/odrive-agent-mount/Dropbox/" -name "*.cloud*" -print0 | xargs -0 -n 1 -P 10 "$HOME/.odrive-agent/bin/odrive.py" sync | tee /dev/fd/6); done
1 Like
stuckj,
I too am using xargs to parallelize the syncs. One useful tidbit, if you are saturating your internet connection, and want to ease off during work hours, you can fire a SIGUSR2 at the xargs process to tell it to reduce its parallelism by one, so hit it 9 times with kill -12 $(pidof xargs) (on linux) and any xargs processes will be reduced by 9 processcount.
When you are ready to ratchet it back up, use kill -10 $(pidof xargs) 9 times to get back to 10.
I’ve had a sync running for a few days and this way it doesn’t stop in the middle of a file, rather xargs just stops running the parallel job count when current jobs finish.
How do I uninstall the Linux sync agent? I cannot find a command to uninstall this.
If you followed the instructions then the only 2 folders you would need to remove are:
~/.odrive-agent (which contains the application information)
~/odrive-agent-mount (which contains the odrive files/folders)
In an attempt to declutter this thread a bit, I have moved a lot of the Questions and Troubleshooting to this thread:
https://forum.odrive.com/t/odrive-agent-troubleshooting-feedback-and-questions/1297
If you have issues or questions, please post there.
Thanks!
New release of Agent is available to bring it into parity with the fixes that have been released for the Desktop. We also now have a Raspberry Pi (ARMv6) compatible build!
1 Like
5 posts were merged into an existing topic: Odrive Agent/CLI Troubleshooting, Feedback, and Questions