I’m trying to sync a specific folder (outside odrive folder) on two different computers. Using the agent (on Windows) so I don’t need to have a specific user logged in.
So far it’s working pretty well using the odrive client (even if the two locations are different, it does sync without problem) but now I have some trouble with the agent.
But when I run this part:
echo “syncing all files in $DIRPATH”
Get-ChildItem -Path “$DIRPATH*” @PARAM | % {& “$SYNC_BIN” sync “$($_.FullName)”}
I get the following error message:
Unable to sync DIR1. This file is not inside an odrive folder.
Unable to sync DIR2. This file is not inside an odrive folder.
etc…
How can I automate such replication (without any users logged in)?
both command below complete without error:
& “$HOME.odrive-agent\bin\odrive.exe” mount “C:\Data\Pictures\Xyz” /
or
& “$HOME.odrive-agent\bin\odrive.exe” mount “C:\Data\Pictures\Xyz” “/Google Cloud Storage/Xyz”
but when trying to synchronize, both give the following message:
Unable to sync DIR1. Please use a placeholder path.
Unable to sync DIR2. Please use a placeholder path.
etc…
NB. No “Google Cloud Storage” directory is created when using existing path, so that’s probably the reason… but well it still leaves me without sync
An other option would be using srvany (it was working with owncloud client), has anyone already tried that?
Hi @D_Sec,
The mount option should work for you. Can you provide a listing of the mounts using the status --mounts CLI command?
Can you then provide an example of the sync command you are using against this that is failing?
Once the mount is setup, any local files that do not exist on the cloud already should actually start syncing. You can see the general odrive status with the status CLI command.
Thanks for your answer, I just figured out, the problem thanks to a colleague.
In fact when you issue, the mount command:
& “$HOME.odrive-agent\bin\odrive.exe” mount “C:\Data\Pictures\Xyz” “/Google Cloud Storage/Xyz”
It’s syncing at that moment the changes the changes to file with .cloud* extension.
So sync command is only used to “convert” those to the real files.
My problem was that I was assuming that I had to sync all files, so I just had to change this code:
Get-ChildItem -Path “$DIRPATH*” @PARAM | % {& “$SYNC_BIN” sync “$($.FullName)”}
to
Get-ChildItem -Path “$DIRPATH*” -Include .cloud@PARAM | % {& “$SYNC_BIN” sync "$($.FullName)"}
Now I have another related question, is there any way to get/send new files (or changes) from/to “Google Cloud Storage” without an unmount then (re)mount?
Could you also point me to some resource that would explain what are those command doing?
backup, stream, refresh*
I tried refresh hoping that it would send change from local to storage and from storage to local, but actually it seems that it does nothing (maybe related to something that I read in an other post saying that there was no API in “Google Cloud Storage” to notify the changes, even if such API exists in “Google Drive”).
Hi @D_Sec,
A refresh on the folder containing any local changes that you want to send to the cloud should trigger an immediate scan for odrive in that location (unless odrive is busy doing something else in another location), which should then send any “dirty” files to the remote storage.
So you are adding or changing files in a folder that is mounted against Google Cloud Storage and you are not seeing the files upload, even after issuing a refresh command? Can you tell me what the path of the file is?
Backup and stream are alpha-state features that are more there for experimentation. Backup will create a one-way sync (local to remote) relationship with the specified local and remote folders. Stream was an attempt to allow streaming to an application as the file is being downloaded.
Refresh will tell odrive to scan the specified location (both locally and remotely) for changes.
After some tests here’s the behaviour:
If I add files to comp1:
C:\Users\Me\My Pictures\Xyz\DIR2
then issue a refresh in powershell, it says: Synced and list all the folders
Now, on comp2:
I issue a refresh, no matter how many times and how long I wait, no .cloud files are created BUT if I unmount, then mount again, files will appear (as .cloud) in C:\Data\Pictures\Xyz\DIR2
So on “Google Cloud Storage” refresh seems to work from local to storage but not from storage to local.
Hi @D_Sec,
Yes this is odd. What if you add a file on Computer 1, refresh to sync, then add a file in the corresponding location on Computer 2 and refresh. Does that file on Computer 2 upload and then is the file from Computer 1 shown?
What if you add a file from the odrive web client and then refresh on Computer 2?
Is the behavior the same in the opposite direction (Computer 2 -> Computer 1)?
I’m just trying to figure out where the root cause lies here.
Can you also clarify the configuration on both systems for me? You are using the odrive agent on both Windows machines, or are you using the desktop client on one Windows machine and the agent on another?
Your first question sounds kinda strange to me, how can I verify that something actually happened?
(only thing that could be verified is that file from Computer 1 is actually well synced to cloud).
Refresh after adding a file from odrive web client generate the same problem. No .cloud file is downloaded.
And yes, just tested the other way around (not working either).
Both system are Windows7 with only odrive agent installed (so without the normal client).
NB. I don’t know if it matter, but while testing I’ve created a multi-regional bucket.
Hi @D_Sec,
Sorry that the first question was confusing. I was wondering if an upload on Computer 2 would trigger reflection of the file from Computer 1. When odrive syncs it needs to “come current” before it uploads, so I would expect it to see the new file from Computer 1 when it is trying to upload the file on Computer 2.
I can’t reproduce the problem with the agent, so I’m not sure what is happening.
Are you able to install the desktop client as opposed to the agent? The CLI can be used for both, so you can still use the CLI to control the desktop client, but we can at least get some better diagnostics from the desktop client if it exhibits the same behavior.
Are you also testing on “Google Cloud Storage” bucket?
Because I suspect more a behavior like you explained in this post (i.e. refresh asking for change instead of full listing):
I will unlink my Google Drive from account and see if it changes something… If not working, I’ll try full client with CLI.
With odrive client installed (and odriveagent not running), I just sent those commands in CLI:
authenticate
status --mounts (only default $HOME\odrive active)
mount localpath "/Google Cloud Storage"
status --mounts (ok localpath is added)
refresh localpath (was already busy syncing… but it was the observed behaviour on mount or re-mount)
When sync was completed, I changed, some files on the Computer 1, and without having to re-issue any command the changes were reflected on Computer 2 (files were correctly deleted and new ones appeared as .cloud)
But the problem is that odrive full client need a user logged in (and sync automatically periodically).
The agent (in theory) was perfect for intended use. When computer starts agent starts (even without users). Then some scheduled tasks instruct him what to sync and when.
Hi @D_Sec,
When you switch back to the agent, are you still seeing the placeholder reflection issue?
If so:
Is this running as a normal user under a user login, or is it running as a service?
If you run the agent in the foreground, instead of as a background task, do you see anything output to standard out? I’m wondering if there is an error being hit on the attempt to write out the placeholder files.
Yup, if I switch back to agent, cloud storage to local isn’t working anymore even on refresh.
During test period it’s running under a domain admin user which have of course (direct) write access on any location used with odrive (so not a SYSTEM account).
How am I supposed to run agent in the foreground? even if I start it manually from a command prompt it’s forking in the background.
NB. When I killed process tree odriveagent, to restart it from command line, he picked up the changes. Anyway after the fresh start, he continues to behave just like the one I start during boot (if I add new files, they are not transferred on local even with refresh).
So, so far, only way to pick up new files from storage are:
Hi @D_Sec,
Apologies for the late response on this.
I’ve been trying to reproduce what you are seeing without any success. I’ve tested the Windows agent against Google Cloud Storage using both simple delimited and enhanced and a refresh always picks up new changes. One thing I want to mention is that I noticed that the output of the refresh command doesn’t actually list the new items, but they will show up when you issue an ls or dir after the refresh.
I’m testing as my logged-in, local user on a Windows 10 system under powershell.
As for running in the foreground, I forgot that the agent is set to run in the background by default. You can redirect stdout and stderr to a log file, in case something is spit out with a command like this: & "$HOME\.odrive-agent\bin\odriveagent.exe" 2>&1 > D:\odriveagent-log.txt
There are also logs in the $HOME.odrive-agent\log folders, although they are not very verbose. Do you see anything in those logs?
Is the behavior the same if you run as a local machine user?
I’m giving up tests with Google Cloud Storage, I absolutely need to move on now.
On my side, nothing was showing up after a dir…
Will run odriveagent with the redirect to a log file from now on (just in case).
As you said nothing interesting in the default log files agent.log and backup.log, and nothing at all in error.log.
Will keep you posted, have upgraded my subscription, took 1Tb on Google Drive and will run it on production server with dedicated user. Now initial sync just needs to replicate a few hundreds Gb first…
I’m not sure what could be happening here. I tried this on Windows 7 and Windows 2012R2 and still wasn’t able to reproduce it. Let me know if you see any issues when using Google Drive.
Funnily enough now it seems that it’s the opposite behaviour
When I start the agent and authenticate:
mount is directly there (no need to mount /localpath/ /cloudstorage/)
on refresh it says that it’s active (yet to be confirmed that it’s syncing without issuing any command)
Anyway looks that it works fine, so that’s the most important.
Thanks for your help and debugging all along!
[EDIT] Just maybe one last question, as now odriveagent will be running all the time in background.
Is there any other way than unmount to pause replication during the day?
