Using Sync Agent to sync a folder outside odrive folder?

aaah can’t test anymore, I’m out of trial period… could you add 2-3 working days to finish tests? thanks

Hi @D_Sec,
I sent you a PM. Please take a look.

Hi @Tony,

With odrive client installed (and odriveagent not running), I just sent those commands in CLI:
authenticate
status --mounts (only default $HOME\odrive active)
mount localpath "/Google Cloud Storage"
status --mounts (ok localpath is added)
refresh localpath (was already busy syncing… but it was the observed behaviour on mount or re-mount)

When sync was completed, I changed, some files on the Computer 1, and without having to re-issue any command the changes were reflected on Computer 2 (files were correctly deleted and new ones appeared as .cloud)

Hi @D_Sec,
Just to clarify, with that configuration is sync working as expected?

Hi @Tony,

Yes.

But the problem is that odrive full client need a user logged in (and sync automatically periodically).

The agent (in theory) was perfect for intended use. When computer starts agent starts (even without users). Then some scheduled tasks instruct him what to sync and when.

Hi @D_Sec,
When you switch back to the agent, are you still seeing the placeholder reflection issue?

If so:
Is this running as a normal user under a user login, or is it running as a service?
If you run the agent in the foreground, instead of as a background task, do you see anything output to standard out? I’m wondering if there is an error being hit on the attempt to write out the placeholder files.

Hi @Tony,

Yup, if I switch back to agent, cloud storage to local isn’t working anymore even on refresh.

During test period it’s running under a domain admin user which have of course (direct) write access on any location used with odrive (so not a SYSTEM account).

How am I supposed to run agent in the foreground? even if I start it manually from a command prompt it’s forking in the background.

NB. When I killed process tree odriveagent, to restart it from command line, he picked up the changes. Anyway after the fresh start, he continues to behave just like the one I start during boot (if I add new files, they are not transferred on local even with refresh).

So, so far, only way to pick up new files from storage are:

  • kill odriveagent and restart it
  • use unmount / mount

Hi @D_Sec,
Apologies for the late response on this.

I’ve been trying to reproduce what you are seeing without any success. I’ve tested the Windows agent against Google Cloud Storage using both simple delimited and enhanced and a refresh always picks up new changes. One thing I want to mention is that I noticed that the output of the refresh command doesn’t actually list the new items, but they will show up when you issue an ls or dir after the refresh.

I’m testing as my logged-in, local user on a Windows 10 system under powershell.

As for running in the foreground, I forgot that the agent is set to run in the background by default. You can redirect stdout and stderr to a log file, in case something is spit out with a command like this:
& "$HOME\.odrive-agent\bin\odriveagent.exe" 2>&1 > D:\odriveagent-log.txt

There are also logs in the $HOME.odrive-agent\log folders, although they are not very verbose. Do you see anything in those logs?

Is the behavior the same if you run as a local machine user?

Hi @Tony,

No problem…

I’m giving up tests with Google Cloud Storage, I absolutely need to move on now.

On my side, nothing was showing up after a dir…

Will run odriveagent with the redirect to a log file from now on (just in case).

As you said nothing interesting in the default log files agent.log and backup.log, and nothing at all in error.log.

Will keep you posted, have upgraded my subscription, took 1Tb on Google Drive and will run it on production server with dedicated user. Now initial sync just needs to replicate a few hundreds Gb first…

Thanks

Thanks for the follow-up @D_Sec.

I’m not sure what could be happening here. I tried this on Windows 7 and Windows 2012R2 and still wasn’t able to reproduce it. Let me know if you see any issues when using Google Drive.

Hi @Tony,

Funnily enough now it seems that it’s the opposite behaviour :slight_smile:
When I start the agent and authenticate:

  • mount is directly there (no need to mount /localpath/ /cloudstorage/)
  • on refresh it says that it’s active (yet to be confirmed that it’s syncing without issuing any command)

Anyway looks that it works fine, so that’s the most important.

Thanks for your help and debugging all along!

[EDIT] Just maybe one last question, as now odriveagent will be running all the time in background.
Is there any other way than unmount to pause replication during the day?

In reference to your statement:

A refresh on the folder containing any local changes that you want to send to the cloud should trigger an immediate scan for odrive in that location (unless odrive is busy doing something else in another location), which should then send any “dirty” files to the remote storage.

Does this mean refresh can be used to delete files from a remote? or are “changed” files only files that exist and are modified? but I can’t sync a directory then sync all the files in it, then move those files away leaving the directory empty and then refresh that folder manually to delete it from the cloud? (I’d actually want this to happen, but it doubt it works this way? I assume refresh works about the same as killing the agent and restarting odrive? or rebooting and restarting odrive right?) not this alternate bizarre function I just made up? (I’ve never run a refresh command manually, and given what I just said, I’m slightly afraid to).

Hi @D_Sec,

I didn’t see your edit here. You can use the shutdown command to stop the agent, and then launch the agent again when you want to resume. Is that what you are looking for?

Hi @bouya.daman,
A refresh is really just a manual way to tell odrive to look at the current folder and make sure the sync engine is up-to-date. For some integrations and configs odrive cannot see change events, so a change will not be picked up until odrive gets around to doing its periodic walk-and-check. A refresh will shortcut this on the target folder.

Deletes will be picked up in this case, and any local files that have been deleted in that folder will be picked-up and those delete actions will be held in the odrive trash until the user empties the odrive trash, which will sync those deletes to the remote storage.

This probably renders dozens of other questions I also just wrote irrelevant.
So I can simply delete a regular file (that odrive previously sync’d) then refresh the folder and then “empty trash” and then the file will be deleted from ACD as well?

That’s exactly how I wanted things to work. If true you can ignore all the other silly questions I wrote over at

Hi @bouya.daman,

That is correct. :slight_smile:
After it sees the delete, the sync engine will hold on listing the delete in the trash queue for about 30 seconds, so just keep that in mind.

1 Like

Didn’t even have to refresh, odrive was very quick about picking up on deleted files.
One comment though the emptytrash command could use feedback, my trash had/has 40,000 items in it, and I run emptytrash command and I see nothing, if I did something similar with sync I’d be getting spammed by lots of line by line information (which would assure me it was working).

Yes I can see the trash number shrinking by using the status command, but these 40,000 deletes might take all day so it would be nice if each delete triggered feedback in the command line. The deletes are going slow enough that next time I delete a migrated folder I might just delete files larger than 1MB, and let ACD remain polluted with a million tiny tiny files. heh.

edit: I wonder, it might just be faster to issue the delete command via ACD’s website, I think the reason odrive is deleting 40,000 files so slowly is throttling by ACD in regardless to api access

Hi @bouya.daman,
I agree, some feedback would be nice.

The odrive trash is a safety mechanism which prevent unintended deletes from hitting the remote storage, so Amazon doesn’t even know about the deletes until odrive sends them. That means that it has to be odrive that does it and there isn’t really a way to shortcut it… Once you clear the current queue it will at least be faster as you in-line the empty command.

What do you mean by inline the empty command? (also btw yes, the emptytrash command finished correctly eventually),

Hi @bouya.daman,
I just meant adding the emptytrash command the trash as part of the overall script flow, so it is happening frequently as things are deleted.