Linux CLI / Assorted Issues

[mdecorte.decorte_ama]

Hi,
I recently started to use odrive with AWS for backups and finding it very nice. I have having a few issues though.

I’m using the CLI on Linux with AWS with the backup options. I understand that “backup” isn’t formally released yet and hoping that reporting these issues can help with th development and backup.

• Ubuntu 14.04.5 LTS
• Python 2.7.6
• Odrive, current 64 bit version, prod 924 per “status”. I’m using the free version. If the paid version will address any of these issues I’m happy to do so.

1. Is there a way to turn on verbose logging to better troubleshoot issues?

2. Is there a way to set a schedule? odrive backs up every hour and I’d like to change that to once per day in the middle of the night.

3. I’m seeing a small number of repeatable but unexplained errors. I’m hoping verbose logging will help isolate the problem. Example below. Removing the file on ACD, or renaming the file locally doesn’t resolve the issues. The errors continue indefinitely. The file names are unique and not related to case.

21 Feb 09:02:13AM INFO Unable to back up file /home/downloads/www.ubuntu.net/netatalk/netatalk-2.2.2/man/man1/Makefile: Amazon Cloud Drive error.

21 Feb 08:01:52AM INFO Unable to back up file /home/mdecorte/linux-config/docker/squeezeserver/chef/chef-repo/cookbooks/AirSqueeze/files/default/AirSqueeze/strings.txt: Amazon Cloud Drive has detected a conflict.

4. If the directory that is being backed up has any symlinks, the entire backup for all files fails. It would be nice if sym-links were supported, but lacking that they should be skipped over with the balance of the backup continuing. The work around is to remove the sym-links in the source but this is problematic.

20 Feb 09:17:17AM INFO Backing up /home/mdecorte/linux-config/chef.
20 Feb 09:17:17AM INFO An unexpected error occurred while backing up.

5. a /etc/init startup script would be helpful. I’ve written one but after a reboot, the odrive agent is running per ps but it doesn’t respond to client commands such as status. Not sure why. verbose would be helpful.

[Tony]

Hi @mdecorte.decorte_ama,

@mdecorte.decorte_ama

Is there a way to turn on verbose logging to better troubleshoot issues?

Unfortunately no. The logging you see is the only logging available.

@mdecorte.decorte_ama

Is there a way to set a schedule? odrive backs up every hour and I’d like to change that to once per day in the middle of the night.

You could do this with a couple cronjobs to shutdown the agent (shutdown command) and then start it again at the appropriate times.

@mdecorte.decorte_ama

I’m seeing a small number of repeatable but unexplained errors. I’m hoping verbose logging will help isolate the problem. Example below. Removing the file on ACD, or renaming the file locally doesn’t resolve the issues. The errors continue indefinitely. The file names are unique and not related to case.

I can’t say what the issue is here. I would expect removing t he offending file on the ACD side would resolve it. If you remove the file on ACD and restart the backup, it still hits that issue? When it does, has the file returned to ACD?

@mdecorte.decorte_ama

If the directory that is being backed up has any symlinks, the entire backup for all files fails. It would be nice if sym-links were supported, but lacking that they should be skipped over with the balance of the backup continuing. The work around is to remove the sym-links in the source but this is problematic.

Do you see an error in the logs when this happens? Curious what the message is, if so.

@mdecorte.decorte_ama

a /etc/init startup script would be helpful. I’ve written one but after a reboot, the odrive agent is running per ps but it doesn’t respond to client commands such as status. Not sure why. verbose would be helpful.

Are you running the agent and the CLI as the same user? That is the only thing I can think of there.
When you run a sudo netstat -nple what do you see listed for the agent?
Does the CLI return an error, or just nothing?

[mdecorte.decorte_ama]

Tony, thanks for replying and your help.

@Tony

Unfortunately no. The logging you see is the only logging available.

Maybe put verbose logging on the backlog?

You could do this with a couple cronjobs to shutdown the agent (shutdown command) and then start it again at the appropriate times.

That’s what I am doing. I have cron to start and later stop odrive. However if the backup is in taking longer than expected and the cron stop job runs then I’m stopping the backup job midway. It works, but not optimal. If the agent didn’t ever start a backup on its own and only via the cli that would work. Or if the agent started a backup on launch and once completed just exited. Lots of other options here. The current issue is that the agent seems to be hardcoded to backup every hour.

I can’t say what the issue is here. I would expect removing t he offending file on the ACD side would resolve it. If you remove the file on ACD and restart the backup, it still hits that issue? When it does, has the file returned to ACD?

For the below error, I have removed the entire directory on ACD. From then on the deleted directory is no longer backed up with no error or message. Just disappeared. Not sure how to fix this. I have a few of these.

21 Feb 09:02:13AM INFO Unable to back up file /home/downloads/www.ubuntu.net/netatalk/netatalk-2.2.2/man/man1/Makefile: Amazon Cloud Drive error.

For the below error, after deleting the file on ACD, the file is uploaded and error cleared. It’s a viable work around, but a more robust design seems appropriate.

21 Feb 08:01:52AM INFO Unable to back up file /home/mdecorte/linux-config/docker/squeezeserver/chef/chef-repo/cookbooks/AirSqueeze/files/default/AirSqueeze/strings.txt: Amazon Cloud Drive has detected a conflict.

If the directory that is being backed up has any symlinks, the entire backup for all files fails.

Do you see an error in the logs when this happens? Curious what the message is, if so.

The error I receive is always along the lines of below. No other errors. The entire backup (e.g. cli backup /home/mdecorte “/Amazon Cloud Drive/home/mdecorte”) is skipped. Removing the sym-link on the client side and the backup is completed within the next hour)

0 Feb 09:17:17AM INFO Backing up /home/mdecorte/linux-config/chef.
20 Feb 09:17:17AM INFO An unexpected error occurred while backing up.

@Tony

Are you running the agent and the CLI as the same user? That is the only thing I can think of there.

When you run a sudo netstat -nple what do you see listed for the agent?

Does the CLI return an error, or just nothing?

I’ll have to play with this some. As reboots are required it’s a bit time consuming. The agent always runs as root.
The CLI errors saying the agent is not running. My work around for now is to run sudo /etc/init.d/odrive start manually after a reboot.

[Tony]

Hi @mdecorte.decorte_ama,

@mdecorte.decorte_ama

The current issue is that the agent seems to be hardcoded to backup every hour.

Currently it is hardcoded to that. I suppose you could add a check for backup jobs status prior to shutting down status --backups

@mdecorte.decorte_ama

I have removed the entire directory on ACD

The problem here may be that agent thinks the folder is still there, and is trying to upload to it. Try removing the folder locally, run backup, then returning that folder. The agent backup logic is not super sophisticated and it may be getting hung up there.

Keep in mind that the backup capabilities in the agent was just an initial release preview, which is why its only in the agent and never made it to the desktop client. We ended up re-writing the whole thing as part of the next generation product, where backup is already much more robust. I don’t have a date for release of the new stuff, but we are nearing preview/beta stage.