Wasabi some files sync some do not

I have some wasabi folders synced to odrive on a NAS via the sync to odrive function. Some of the files ( large ones) will sync perfectly and automated after I drop files in. But some files will sit there for days until i refresh the folder.

I am rather busy so I haven’t caught up on the functionality of refresh too well. Maybe I missed something. I have a folder two levels up set to sync everything at any size and all new files though ti seems to still miss
stuff.

I also seem to have some issues with a random file disappearing once in awhile. The wasabi people tried to break it down to me as a unidirectional and bidirectional sync issue. Seems weird as one file in 200 will go missing.

Please help!

Hi @conor,
Under normal circumstances odrive will receive a “filesystem event” that tells it when a file has been added. External drives like NAS may not provide filesystem events, so there can be a delay in picking up new files. However, odrive also has a periodic local scan that it performs every 2 hours. You can see this in the sync activity log. This means that local changes shouldn’t take more than 2 hours + the time it takes to complete a local scan.

If you are able to reproduce this issue, we can take a look at the sync activity logs and diagnostics to try to identify why the new files may not be picked up.

For disappearing files, odrive should also have this recorded in the sync activity logs when it picks up the “remote delete” and applied it. We should be able to trace this and see when it is happening, at the very least.

Do you have any auto-trash rules set, or are you explicitly emptying the odrive trash when you want to apply remote deletes?

You can find the sync activity logs here:
sync activity log

I am manually emptying the trash. I have a feeling there’s a conflict somewhere between multiple computers syncing on active folders with backups also running through these folders as well. I haven’t been able to find the file error in the logs though I can see when I fix it by reuploading it. I guess more careful day of examination of the logs is going to show me where the issue is.

Reproduction of the issue might be tricky because it happens from multiple nodes doing many different things. Controlling the variables to reproduce it is a bit beyond my operation at the moment.

If I find a specific error or sync pattern I don’t understand I will report back here and see if it illuminates anything.

Thank you!

Sure thing @conor!

Also make sure you’ve got the latest version installed which has a few tweaks and fixes that will be good to have.

We’ve released a couple of updates recently, although we haven’t pushed them to the auto-updater just yet (we are always a bit paranoid about doing a mass-update without a good amount of time for feedback from manual updaters and new users).

I think I documented the error in a log. There was a single file that multiple nodes were opening saving and closing. After awhile it thinks it is deleted and syncs the delete and a move. im tracking down if these action were done manually, but in any case there’s the log - the stretch of 300 lines is all file updates for the same file.

we also have one mac node reporting it still syncing to their node, but on the nas it is gone.

I am also able to recover the file from trash, plus I manually updated to 6984

Hey @conor ,
Thanks for this info.

We recently added the additional delete logging and a couple of double-checks for local deletes. The 300 entries you are seeing are the recorded filesystem events that Windows is giving to odrive. Once odrive puts something into the odrive trash, it is well and truly gone from local, or at least odrive isn’t able/allowed to see it (the file is checked and then re-checked to make sure it is actually no longer listed in the local folder).

The filesystem event that shows a delete at 10:54:45 is when the file was reported as being deleted by Windows. It is interesting that it is right after a successful upload of the file.

Can you explain this a bit more? I don’t understand the environment or setup, but it sounds complex :wink:

There are two systems syncing the same structure wasabi account, files are added on both ends we avoid deleting files in most situations and it all should sync up perfectly in an ideal scenario. I will manage some cleanup and restructuring on one node and let that propagate to the other.

The really weird thing is no user manually deleted a file - it was dragged and dropped to an external drive on the mac machine. I want to figure out if wasabi reads moves as deletes and copies - which I have seen in their own file management software.

Hey @conor,
In image it shows that the file was seen as deleted by Windows on the Z: drive (mounted NAS drive).

Is this NAS also shared with a Mac machine and was that file moved from the mounted NAS drive to a separate external drive on the Mac?

I’m just trying to understand the relationship between all of the pieces and the flow for this drag and drop action.

it is not shared to the mac machine - only has one SMB share

I do have good sync copying files unidirectionally to other folders on the same NAS and different account

I have turned off all Goodsync backups as I can see the timestamp in it of the file in question as the same as the one in the logs. This might be related to that software and me not propagating deletions. For now its off and I will see if the same thing happens again in the next few days.

Hey @conor ,
Thanks for the follow-up!

I don’t want to keep bothering you with questions because I know you are very busy, and it sounds like we can just continue to investigate if this occurs again. If you do see this again, I can help take a look at the logs from both systems to see what the flow and sequencing is on both sides. That should give us a good starting point.

Sounds good, thank you for the help so far breaking apart the logs.

I am addding more to this thread as I have seen a LOT more behavior in this same vein.

I am seeing it mostly on a google workspace account now.

I will work on some files and watch them disappear right in front of me, sometimes after I save a file and sometimes awhile after I work on them. I am looking at tifs in bridge while also working on them in photoshop.

I am having to actively monitor folders for deletions and recover them as I go through the day. It seems like this behavior is coming from odrive or my NAS as I have seen it on folders all on the NAS and all synced with odrive over multiple accounts of dropbox/wasabi/google workspace.

Hi @conor,
Any files that are removed via odrive should be logged as a “Remote to Local” delete in the “sync activity log”. When this happens it indicates that odrive requested a remote listing of the folder and the file was no longer listed.

All local deletes will actually have a few logs about when the file was noticed to be missing and if/when it is deleted by odrive from the odrive trash (“Local to Remote” delete).

If you send me the sync activity log and an example path I can trace the behavior and analyze what is happening. You can send me the log via direct message by clicking on my name and selecting “Message”.

Also, to clarify odrive behavior, odrive will never delete a local file unless it is in response to a remote delete. Whenever odrive discovers a local delete, it was done by something/someone other than odrive. When odrive next looks at that local location it sees that a file that was previously there is now gone, records the change, and puts the item into the odrive trash queue.