Odrive keeps crashing

I thought I had found a good solution to sync my large work folder to Amazon Cloud Drive using odrive (with Pro), but unfortunately odrive keep crashing silently after a few minutes. Nothing shows on Amazon (not even the new folder I set it up to sync to). The folder I am trying to synch has about 1TB. Would love for this to work; what do you need to assist troubleshot this?

I tried synching a small test folder and that work, so I assume the issue is with the large folder…

Thanks!

Hi,
Can you give me the characteristics of the folder you are trying to sync in terms of # of files and # of folders?
Can you also provide a number of files for the largest folder you have?
Additionally, can you tell me where the data is residing? Is it on an external drive or a network drive, for example?

I just want to understand the scope of data that needs to be processed.

Tony,

Thanks… here are some details:

Folder size: 742 GB
2,958,419 files, 240,733 folders

This is on an internal drive.

Brendan

Thanks. That is quite a large structure. Can you try to send along a diagnostic? You can find the option in the odrive tray menu.

Tony,

Indeed quite a large structure…

I sent the diagnostic report.

Brendan

Hi.
I’m having trouble locating the diagnostic. Can you tell me what the local username on that machine is?
Can you also tell me what OS and version you are running?

Thanks!

Tony,

Local username would be Brendan. It is a Windows 10 PC.

Brendan

Hi Brendan,
Can you try launching odrive from the command prompt? I want to see if anything is spit out when the crash occurs:

  • Open a command prompt (cmd.exe)
  • cd to %userprofile%.odrive\bin
  • cd to the latest version directory listed in here
  • odriveapp.exe

Once the crash occurs, can you copy and paste the output here?

Thanks!

Tony,

Here is some of the errors, but there are lots of that same error.

error: can’t start new thread


Exception happened during processing of request from (‘127.0.0.1’, 55743)
Traceback (most recent call last):
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 295, in _handle_request_noblock
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 610, in process_request
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\threading”, line 736, in start
error: can’t start new thread


Exception happened during processing of request from (‘127.0.0.1’, 55744)
Traceback (most recent call last):
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 295, in _handle_request_noblock
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 610, in process_request
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\threading”, line 736, in start
error: can’t start new thread


Exception happened during processing of request from (‘127.0.0.1’, 55745)
Traceback (most recent call last):
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 295, in _handle_request_noblock
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 610, in process_request
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\threading”, line 736, in start
error: can’t start new thread


Exception happened during processing of request from (‘127.0.0.1’, 55746)
Traceback (most recent call last):
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 295, in _handle_request_noblock
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\SocketServer”, line 610, in process_request
File “C:\hudson\workspace\Odrive_Win_Master_Build\stage\artifacts\build\odriveapp\WinOdriveApp\out00-PYZ.pyz\threading”, line 736, in start
error: can’t start new thread

So, this is tricky. The size of the structure is presenting a problem. One thing that can be attempted is to close all of your explorer windows to see if it stabilizes a bit. The badging on that enormous structure is going to wreak havoc on your resources, in addition to just the scan and processing.

So these steps could help and will at least give us more insight into the amount of resources needed:

  1. Close all explorer Windows
  2. Open up the Task Manager and select the details tab. Right-click on the top bar and select more columns to display (threads and handles)
  3. Start odrive
  4. Monitor the odriveapp process and see how the resource utilization is affected as it grinds through the structure.

Can you also tell me what the specs of your system are (memory and cpu)?

Thanks!

Tony,

It ran for about 15 minutes using at peak time about 1.8GB of memory and about 20% CPU, then CPU went to 0% for a while until it crashed.

I am running a Windows 10 64 PC with 16GB RAM, Intel Core i7 CPU 2.67GHz 2.70 GHz.

Looks like the software can’t really handle the number of folders and files I have, right?

Brendan

Thanks. Did you see what the handles and threads were at?

It appears that it is just too much for odrive to take all at once, at this time. The next thing to try is to break up the set with several pro sync folders instead of just one huge one. odrive supports having many pro sync folders, and I think by breaking it up you will have a better experience.

Have you considered this approach yet?

Tony,

Where would the handles/threads be?

The main goal was to find a solution that would require minimal setup and provide for selective synching… looks like I may have to wait till it can handle much larger file/folder set.

For the handles and threads take a look at step #2 above. Here is a screenshot of what it would look like:


I think you will have to look at this realistically as being an unusually large number of objects for a single user/system. I can’t say we will ever be able to reliably track and sync 3.25 million objects without significant overhead.

There may be some adjustments we can make to try to better handle it, but I believe you can get a working solution now by breaking up the set.

Can you tell me what your use case is? Is it just to backup the files to the cloud, or to migrate so you can work out of the cloud against that data?

Tony,

Yeah, it is too much for how things work now.

The idea was to have it all in the cloud and accessible as needed (selective syncing) from a notebook while traveling etc. (beyond VPN and in case the office computer is unavailable).

Thanks again for your help.

Hi,
To be clear, I think you can still so this, it will just take more steps. Given your use case, here is how I think it can be done:

Use Pro Sync to map your data in, progressively. So let’s say you have a main folder you want to get into the cloud with subdirectories underneath. For each of those subdirectories, you can create a pro sync mount. If you do this, one at a time, you can map the folder in, get it uploaded, and then remove the pro sync mount. This will effectively get the data into the cloud for access on other devices. You can then work out of the cloud folders directly. This is really how we encourage cloud to be used anyway, instead of maintaining a constant attachment to local storage.

Tony,

There are just too many folders within folders, so it would take hours to do them one by one etc. and would not be effective.

Thanks for your help though!