SFTP error when creating remote folder

[jk1]

When synchronizing a folder with about 10 subfolders in SFTP (Infomaniak Swissbackup), I see a lot of errors such as:

14 Jun 11:16:53PM ERROR Failed Create Folder (Local to Remote) for C:/path/to/syncdir/mySubFolder Error: code SFTP_UNKNOWN_IO_ERROR caused by SFTPRequestException(code SFTP_UNKNOWN_IO_ERROR - Low level IO/permission sftp error caused by IOError([Errno 2] No such file))

Also occasionally I see errors on file such as:

12 Jun 12:29:17AM ERROR Failed Upload File for File: C:/path/to/syncdir/mySubFolder/myFile - Size: 194400 Bytes - Date: 2016-09-24 22:39:18 - Error: code SFTP_TOO_MANY_SIMULTANEOUS_CONNECTIONS

Apparently, the number of connections on the server is limited, but I don’t know the limit. Are both issues related ? Is there a way to limit the number of connections in odrive ?

[Tony]

Hi @jk1,
The first error is an indication of a server error, of come sort. The “no permission” could be a little misleading, since it is somewhat generic. For these items that are erroring, do you see the folder being created on the remote side?

Check the permissions on the parent folder, as this could be indicative of a “no permissions” error.

For the second error, you can change the concurrent connections for the link on the web client by going to the listed link and selecting “Edit”. This will allow you to change the “Max Concurrent Connections” parameter. You may need to restart the desktop client to pick up the link change.

[jk1]

For the error “no permission” I don’t see the folder created on the remote side. I don’t think this is a true permission issue because when I’m connected to the remote side using FileZilla FTP Client (same username/password than with odrive), I can create the folder manually.

I set the “Max Concurrent Connections” to 1 and it looks like more files can be transfered without errors.

I noticed from the main.log that the folder is correctly created only right after connection:

15 Jun 01:33:05AM INFO End scan and refresh mount C:/Projets/Mobilière
15 Jun 01:33:06AM INFO Connected (version 2.0, client Infomaniak)
15 Jun 01:33:06AM INFO Connected (version 2.0, client Infomaniak)
15 Jun 01:33:06AM INFO Authentication (password) successful!
15 Jun 01:33:06AM INFO [chan 0] Opened sftp connection (server version 3)
15 Jun 01:33:06AM INFO Authentication (password) successful!
15 Jun 01:33:06AM INFO [chan 0] Opened sftp connection (server version 3)
15 Jun 01:33:09AM INFO Successful Create Folder (Local to Remote) for C:/Projets/Mobilière/Articles/GLOSSAIRE Assurance Vie - Arca Patrimoine Blog_fichiers
15 Jun 01:33:09AM INFO Successful Create Folder (Local to Remote) for C:/Projets/Mobilière/Acceptation électronique
15 Jun 01:33:10AM ERROR Failed Upload File for File: C:/Projets/Mobilière/Acceptation électronique/2020-07-30 00_29_38-Conclusion électronique d'une assurance I la Mobilière.png - Size: 101715 Bytes - Date: 2020-07-30 00:30:32 - Error: code SFTP_UNKNOWN_IO_ERROR caused by SFTPRequestException(code SFTP_UNKNOWN_IO_ERROR - Low level IO/permission sftp error caused by IOError([Errno 2] No such file))
...
15 Jun 01:33:34AM ERROR Failed Upload File for File: C:/Projets/Mobilière/Articles/GLOSSAIRE Assurance Vie - Arca Patrimoine Blog_fichiers/style.css - Size: 103438 Bytes - Date: 2016-09-24 22:23:54 - Error: code SFTP_UNKNOWN_IO_ERROR caused by SFTPRequestException(code SFTP_UNKNOWN_IO_ERROR - Low level IO/permission sftp error caused by IOError([Errno 2] No such file))
15 Jun 01:33:34AM INFO Successful Upload File for File: C:/Projets/Mobilière/PIQ4-2020/OU EST CHABLAIS.png - Size: 5906854 Bytes - Date: 2021-02-06 14:11:53
15 Jun 01:33:35AM ERROR Failed Create Folder (Local to Remote) for C:/Projets/Mobilière/Blockchain Error: code SFTP_UNKNOWN_IO_ERROR caused by SFTPRequestException(code SFTP_UNKNOWN_IO_ERROR - Low level IO/permission sftp error caused by IOError([Errno 2] No such file))
15 Jun 01:33:35AM INFO Successful Upload File for File: C:/Projets/Mobilière/Articles/GLOSSAIRE Assurance Vie - Arca Patrimoine Blog_fichiers/addthis_widget.js - Size: 372095 Bytes - Date: 2016-09-24 22:23:54
15 Jun 01:33:35AM INFO Successful Upload File for File: C:/Projets/Mobilière/Articles/GLOSSAIRE Assurance Vie - Arca Patrimoine Blog_fichiers/layers.js - Size: 268386 Bytes - Date: 2016-09-24 22:23:54
15 Jun 01:33:35AM ERROR Failed Create Folder (Local to Remote) for C:/Projets/Mobilière/LBA Error: code SFTP_UNKNOWN_IO_ERROR caused by SFTPRequestException(code SFTP_UNKNOWN_IO_ERROR - Low level IO/permission sftp error caused by IOError([Errno 2] No such file))

So the “LBA” folder could not be created at the first connection. But after a second connection it can be created:

15 Jun 07:01:18AM INFO Connected (version 2.0, client Infomaniak)
15 Jun 07:01:18AM INFO Authentication (password) successful!
15 Jun 07:01:18AM INFO [chan 0] Opened sftp connection (server version 3)
15 Jun 07:01:28AM INFO Start scan and refresh mount C:/Users/Julien/odrive
15 Jun 07:01:29AM INFO Scanned (mount) 14 local folders
15 Jun 07:01:29AM INFO End scan and refresh mount C:/Users/Julien/odrive
15 Jun 07:01:31AM INFO Start scan and refresh mount C:/Projets/Mobilière
15 Jun 07:01:31AM INFO Scanned (mount) 7 local folders
15 Jun 07:01:31AM INFO End scan and refresh mount C:/Projets/Mobilière
15 Jun 07:01:33AM INFO Successful Create Folder (Local to Remote) for C:/Projets/Mobilière/LBA
15 Jun 07:01:35AM INFO Successful Upload File for File: C:/Projets/Mobilière/LBA/GUI/Coder en HTML et Javascript.txt - Size: 611 Bytes - Date: 2020-06-20 14:20:28
15 Jun 07:01:35AM ERROR Failed Upload File for File: C:/Projets/Mobilière/LBA/GUI/partner_PP_F.png - Size: 6769 Bytes - Date: 2020-06-19 16:21:43 - Error: code SFTP_TOO_MANY_SIMULTANEOUS_CONNECTIONS
15 Jun 07:01:36AM INFO Successful Upload File for File: C:/Projets/Mobilière/LBA/GUI/partner_ENT.png - Size: 4305 Bytes - Date: 2020-06-19 16:21:43
15 Jun 07:01:37AM INFO Successful Upload File for File: C:/Projets/Mobilière/LBA/GUI/partner_PP_F.png - Size: 6769 Bytes - Date: 2020-06-19 16:21:43

It’s a bit strange that SFTP_TOO_MANY_SIMULTANEOUS_CONNECTIONS are still found in the main.log even if the “Max Concurrent Connections” has been set to 1 (odrive Windows client have been exited then restarted).

[Tony]

Hi @jk1,
We will probably need to test against this source to figure out what might be happening, since all of our typical SFTP testing and use has not revealed this type of issue.

Is there a way for you to create a test account for us to use to debug?

[jk1]

Thanks for your support. I sent you test user connection data in a private message.

[ergolds]

I’ve been experiencing the exact same issue, the SFTP_TOO_MANY_SIMULTANEOUS_CONNECTIONS even though I have limited downloads and uploads:

  "maxConcurrentDownloads": 1, 
    "maxConcurrentUploads": 1,
    "maxConcurrentJobs": 1

This is making odrive unworkable for me! I’m on OS X 12.6.

[Tony]

Hi @ergolds,
The issue with the user above was actually resolved with a new version of software, although you would already have that version.

In that case, the server was behaving badly with session handling, so we made a compensating adjustment to the integration. Your issue is likely different.

As a start, see if increasing the connection limit in the integration setup. To do this:

1. Go to odrive | Sign in
2. Click on the “V” icon next to your integration and select “EDIT”
3. Increase “CONCURRENT CONNECTIONS” to “10” or more.
4. Save the change
5. Restart the odrive desktop client.

If the issue persists, please send a diagnostic from the odrive tray menu and let me know.

How do I send a diagnostic?

When you send a diagnostic, the odrive desktop client will provide a report to odrive that contains lower-level information that may be necessary to troubleshoot a particular issue. A local report is also generated that you can look at. You can find this report, named “current_odrive_status.txt” in the root of the default odrive folder. To send a diagnostic: On Windows: Open up the odrive tray menu, located in the Windows taskbar Click on “Send diagnostics”, located at the bottom of the me…

[ergolds]

Thanks Tony, I still have the same issue after that update so I sent a dignostic.

[Tony]

Hi @ergolds,
I took a look through the diagnostic. You have the following settings set in the configuration:

maxConcurrentDownloads: 5
maxConcurrentJobs: 20
maxConcurrentUploads: 5

• This will create a lot of concurrency, so you’ll want to set all of these values to 1, for the time being, while testing.
• What did you end up setting the SFTP link’s concurrent connections to (at odrive | Sign in)? This should be set to 10 or higher for our test.
• Let’s also get you on the latest desktop version. To do this:
  • Exit odrive
  • Download the latest version from here: Set Up Your odrive
  • Install

[ergolds]

that seemed to work, I don’t see errors anymore. Thanks!

[ergolds]

hi again, I’m getting this again on a Linux machine (Ubuntu 22.04). Settings are same as other machines. Any thoughts?

[Tony]

Hi @ergolds,
Can you send a diagnostic, so that we can take a look? (use the diagnostics CLI command)

What do you have the SFTP link settings set to?

[ergolds]

Hi Tony, see below – maybe someone from the team can help me troubleshoot? SSH and SFTP works fine for everything else. Maybe has something to do with my ssh-agent and stored keys?

odrive Status
buildNumber: 1016
product: prod
odriveEmail: XXXXXXX [redacted]
odriveName:
authorizedAccountSourceType: Google
downloadThreshold: Never download
unsyncThreshold: never
autoTrashThreshold: never
xlFileThreshold: never
downloadThrottlingThreshold: unlimited
uploadThrottlingThreshold: normal
odriveFolder: None
LocalEventsEnabled: True
RemoteEventsEnabled: True
LocalScanEnabled: True
RemoteScanEnabled: True

generalConf:
disableSparse: False
allowFlagged: False
disableRemoteFileUpdates: False
disableAutoupdateRestart: False
ignoreSizeMismatch: False
disableLocalFileUpdates: False
allowOldDownload: False
suppressTrashNotifications: False
disableConflictDetectionAll: False
localScanIntervalSecs: 1200
processMemoryLimitMBytes: 3584
disableFSEvents: False
maxDownloadRetries: 3
disableConflictDetectionStrict: False
allowZeroByteUpdate: False

premiumConf:
backupDisableMerge: False
disableEncryptedNames: False
maxConcurrentDownloads: 5
blackListContains: []
initialUploadBatchSize: 1
remoteScanIntervalMins: 840
blackListPrefixes: [u’.~‘, u’._‘]
blackListNames: [u’thumbs.db’, u’desktop.ini’]
blackListPrefixesRemove: []
maxTransferMBytes: 256
maxConcurrentJobs: 20
backupIntervalMinutes: 1440
forgetEncPassphrase: False
blackListNamesRemove: []
maxConcurrentUploads: 5
blackListExtensions: [u’.tmp’, u’.download’]
blackListExtensionsRemove: []

Trash (0):

Not allowed (0):

Waiting (0):

Active (0):

Uploads (0):

Downloads (0):

Sync folders (1):
ProSyncFolder - localO2Path:/[XXXXXX]odrive remoteUri:/ remotePath:/ status:Active

Folder sync rules (0):

Blacklist:
Names: com9, com8, lpt8, lpt9, com1, lpt5, com3, com2, com5, com4, icon/r, com6, nul, current_odrive_status.txt, desktop.ini, $badclus, $upcase, $objld, prn, lpt1, .odrive, lpt4, $attrdef, .localized, thumbs.db, thumbs.db:encryptable, com7, $extend, .fcpcache, odrive_user_general_conf.txt, $logfile, $volume, $boot, $reparse, $mftmirr, lpt2, .ds_store, icon, lpt3, .oxygenmeta, icon, $mft, odrive_user_premium_conf.txt, .oxygenreserved, .dropbox, $quota, lpt6, $secure, System Volume Information, .lock, $bitmap, lpt7, con
Prefixes: .lock, .~, ._, thumbs.db, ~
Extensions: .dwl2, .m~, .crdownload, .vpsh, .tmp, .download, .lrprev, .dwl
Contains:

Recently added to remote (1):
XXXXXX

Recent Errors:
2023-09-07_06:39:50PM 139623439246912 139623430854208 RAISED remote_list_folder([u’uri=/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FOLDER:/private/peek’, u’pageToken=None’]) → NetworkException(code SFTP_NO_NETWORK caused by SFTPNetworkException(code SFTP_NO_NETWORK - SFTP socket error caused by SSHException(Server connection dropped: )))

2023-09-07_06:40:24PM 139623439246912 139623430854208 RAISED remote_list_folder([u’uri=/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FOLDER:/private/peek’, u’pageToken=None’]) → NetworkException(code SFTP_NO_NETWORK caused by SFTPNetworkException(code SFTP_NO_NETWORK - SFTP socket error caused by SSHException(Server connection dropped: )))

2023-09-07_06:41:04PM 139623439246912 139623430854208 RAISED remote_list_folder([u’uri=/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FOLDER:/private/peek’, u’pageToken=None’]) → NetworkException(code SFTP_NO_NETWORK caused by SFTPNetworkException(code SFTP_NO_NETWORK - SFTP socket error caused by SSHException(Server connection dropped: )))

2023-09-07_06:45:25PM 139623439246912 139623481210432 RAISED remote_list_folder([u’uri=/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FOLDER:/private/peek’, u’pageToken=None’]) → NetworkException(code SFTP_NO_NETWORK caused by SFTPNetworkException(code SFTP_NO_NETWORK - SFTP socket error caused by SSHException(Server connection dropped: )))

2023-09-07_06:45:42PM 139623439246912 139623481210432 RAISED apply_local_expand_file([u’cloudO2Path=/home/evan/Documents/odrive/1and1 FLKR/private/efinance.cloud’, u"remoteAttr=FileAttributes(id=u’/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FILE:/private/efinance’, name=u’efinance’, isFolder=False, size=629145600, modTime=1513196038, uri=u’/cf39efba-1cc7-4171-86a9-66bfc892d652-5995f429/FILE:/private/efinance’, contentTag=None)"]) → NetworkException(code SFTP_NO_NETWORK caused by SFTPNetworkException(code SFTP_NO_NETWORK - SFTP socket error caused by SSHException(Server connection dropped: )))

[Tony]

Hi @ergolds,
From the diagnostic it doesn’t look like you are hitting too many connections, but are encountering network errors?

In any case, the maxConcurrentJobs setting you have is much too high at 20. This should be 1-2, since each job will have multiple threaded actions. It is essentially a multipler for the other settings, so I recommend not raising this above 2.

[ergolds]

aha, don’t know how that got set so high but it seems fine now! Thanks.

[ergolds]

ok I had this happen again after not using my computer for awhile – restarting the odrive agent helped clear the error. I have a feeling it has something to do with my network reconnecting (I have a VPN client). Any thoughts?

[Tony]

Hi @ergolds,
When I was looking at your last diagnostic, it looked like you were hitting general network exceptions. If you are using VPN, then that is a likely suspect, especially if you are seeing that the network needs to be reconnected frequently. Those reconnects would generate a network break, which would be consistent with the error I was seeing.