Facebook file parser has problem which rejects some file names

Troubleshooting
1

(Bear with me there is a solution and Odrive does work) If you have had a problem in fb downloads to your local drive where you get “illegal file name” errors etc, here what I found by (a lot) of testing:- if you put an album name with a few full stops (these things …) at the END of the album name the Odrive parser for file names in sync modes throws up filename errors - here is my specific line testing of this syndrome - I can’t get a support response from Odrive, however , here is is as I find it. I am a worse-case tester and I have 344 folders on FB with over 18000 pictures in it. I am fully satisfied Odrive is good for backing up facebook, and deals perfectly with FBs clunky API, apart from what I just found… (See…I love putting dots a lot haha…) Ok Initial conditions PC removed all Odrive old versions and files cleaned off:-

revo uninstaller / some manual / all storage directories and all traces of odrive

rebooted, ADMINISTRATOR level

lots of local storage space fast 150mbits/sec internet

64bit CPU windows 8

dloaded sync V odrivesync6442.exe

installed without modification

/facebook login, access facebook, empty 344 directories

problems :-

INDIVIDUAL SYNC, FILENM ------------------ RESULT

[A Summer’s Day 5 years ago…cloudf] (include dots result cannot sync ERROR UPDATING THE FOLDER ILLEGAL FILE NAME)
[A trip to the Cow Wash…cloudf] (include dots result cannot sync ERROR UPDATING THE FOLDER ILLEGAL FILE NAME )

reverted to facebook to edit dots outs

resync all parsed ok - seems 2-5 or more dots (full points) stop parser.

checking other illegals

[Anil, …cloudf] (include dots result cannot sync ERROR UPDATING THE FOLDER ILLEGAL FILE NAME )

result many filenames (i use) which are initially collected then fail due to too many full points in the text filename string,
then I tried to /facebook resync from top level after editing a few facebook strings, I get cant sync facebook, illegal name and no other indications

So it writes the full (344) directories for the album folders, in its new first pass,
afterwards when is starts to build the internal files from the cloudfs then it fails as it parses the first part of the file name,

TEST SEQUENCE

“a_test” 1 image added no full point (ignore “”)

“a_test.cloudf” found in folder on local drive I didnt open it or try to sync it…

“a_test.” edited to add one full point

“a_test.cloudf” found in folder on local drive then it synced with image in it

“a_test…” edited to adding several full points Error updating illegal filename occurs,

and the top level facebook /rightclick resync also fails, which means its trying to probably look ahead inside the local folder and hitting a bad filename update.

I use “this is my album …” as a literal string a lot, indicating time and sequences. Moral of the story - Simply remove full stops (full points - periods ) from your facebook album names especially at the end and keep them simple, they may fix it I hope they do…!

2

Hi @facebook1,
Thanks for detailed writeup!

This is actually done as a preventative measure for Windows, since Windows does not support having periods at the end of a folder. You can technically create one, but then you won’t be able to browse into it because Windows Explorer will chop off the trailing periods.

The initial test you performed where you added the . to the end of a_test may not have reflected to odrive in time, which is why it was able to open (the name was still seen as not having a trailing period). It should’ve been seen as a_test..cloudf if it had fully reflected.

You can test the Windows behavior by creating a folder with periods in it with the following command from a command prompt:
mkdir \\?\c:\test_folder...

This will create a folder called test_folder... in C:, but you won’t be able to browse into it from Windows Explorer (you will see an error about the folder test_folder being unavailable). This is because it ends up trying to go to C:\test_folder (it chops off the periods), which doesn’t exist.

3

so instead of users wasting time trying to work out if its your parser and or windows or anything else hidden cant you code up better dialog boxes with more helpful highlighted information??? I’m now spending hours removing trailing …s any more windows rubbish land-mines? Does it handle 100% thai language filenames and other non-roman extended character sets for example??? (I’m in thailand) or was this the only glitch?

4

Hi @facebook1,
When odrive produces an “Illegal File Name” exception, it is always predicated on the operating system’s limitation and indicates that the name is illegal for the operating system you are on. For example, you won’t see this particular issue on a Mac because MacOS doesn’t have the same problem.

I have asked the product team if we can improve the messaging.

In terms of other limitations, this will depend on Windows’ limitations. It definitely has several quirks, but unicode characters should be okay.

5

Hi, :blush: So far we have edited (pipeline dir > odrivetext.txt) the entire subdirectory into a text file and searched for illegal characters such as double or more full points. We then edited out all double full stops from my facebook albums …. This has had limited success, but it helped fix the main problem.
(Bearing in mind we have 344 folders all with a lot of files in them > 16,000 pics our full library but I don’t think I’m the only person on the planet with a DSLR )

Remaining “bugs”:-

  1. the odrive folder does not fully update automatically and pull in my pics, so should it do this on a free package? I’ve gone through it several times and right clicked / sync, on some folders laboriously but I expected odrive to behave automatically and unattended – this was my “buying” decision test for myself and my corporate users.

  2. Despite fishing and modifying for what I believe are all the bad file names from facebook the top level folder in the library /users/administrator/odrive/facebook still shows “cant sync facebook, illegal file name” when right click sync, even from reboot, and sometimes the odrive’s dialog drop down menu disappears – but the app doesn’t seem to be busy! But occasionally the red sync graphic shows up, this demonstrates a random behaviour. something is triggering “illegal filename”?

Can you explain this behaviour, is it because we have so many folders and files you app gets confused??? Please email me direct so that I can send you a copy of our /facebook list so you may see if there are any illegal characters in it or combinations as I’m not pasting it here?
I stand by my previous comments about your dialog boxes that do not provide a diagnostic line text showing THE bad file name which then could be helpful to editing it. Especially where 344 folders are concerned!!!

Thanks for your kindly support response, however, I seriously don’t think the average 2019 user will know about the DOS 8.3 and later Windoze OS extensions to it or the underlying behaviour of the OS when selecting filenames, IMHO most Facebook users who are allowed to name Facebook Album folders how they wish, will run into this problem and this should be fixed by your team so they never crash and complain. (I lead software teams in software defined radio projects and I have 40 year feel for what customers can do to break things as tired CEO LoL :blush:)
Yours Paul W Encs :- SMALL SAMPLE OF OUR FB FOLDERS BELOW

Volume in drive C has no label.
Volume Serial Number is ACF4-311D

Directory of C:\Users\Administrator\odrive\Facebook

31/01/2019 12:08 .
31/01/2019 12:08 …
30/01/2019 18:21 2011 Land Goats Beaches and Bikes
30/01/2019 18:21 2012 - 2013!!! We made it!!
30/01/2019 18:21 2014 Reflections
31/01/2019 01:16 A Dangerous things to do in Asia
30/01/2019 18:43 A day in the Lands of the Sapelots (Pineapples)
30/01/2019 18:43 A day out to see the monk in the cave at Kailom (secret place)
30/01/2019 18:36 A Summer’s Day 5 years ago
30/01/2019 18:36 A trip to the Cow Wash
31/01/2019 01:16 Abbey Road Studios opposite view
30/01/2019 18:43 Aberystwyth, gardens, May and June
30/01/2019 18:43 Afghanistan from the air
30/01/2019 18:38 Album - Wai Kru - Part 2 Wizard Productions
30/01/2019 18:43 Album - Wai Kru Part 1 Rehearsals
30/01/2019 18:43 Album - Wai Kru Part 3 - (not yet fully uploaded)

6

Hi @facebook1,
Since it sounds like you are working within an IT context, you could try to use the CLI for a more advanced approach to this.

  1. odrive will stop downloading in this type of situation if it hits enough errors during download. With Facebook it is likely that you are hitting rate limits. We can use the CLI to continue through any errors you might encounter

  2. With the CLI it should be easier to pinpoint where the error is

To use the CLI, please refer to this post here:

Give this a shot and see if it helps address some of the issues you are encountering.

7

No Im not “working in a IT environment” that’s quite a rude comment, my staff are and Im not paying top coder to fix my personal PC problems on YOUR app, however, Ive now done my own final analysis and its here as it works but is “clunky”:
BUGS
CONDITIONS
All files properly backed up now except two:-
FAILS ILLEGAL FILE NAME

  1. “Pentre Springs and old photo discoveries 2017, 2016, 2015.cloudf” failed illegal filename.
    Changed to another simple name and it worked and backed up ok.

  2. “Pentre + Dyrham.cloudf” changed to “From Dyrham to Pentre.cloudf” and still fails, it’s as if the updated file name is not accepted or there is a hidden character.
    Then I changed it to a simple letter “A” and it backed up properly.
    Then I changed it back to another sensible name which then auto-deleted the “A” record and contents then
    “Pentre and Dyrham Historical Stuff.cloudf” appeared as expected. So in order to get it downloaded right click and hey presto, failed illegal file name, (also failed at /facebook level,) Interesting, this means there is only one failed file left in directory.
    Then changed back to “A2” a simple file name again to test to see if it backs up or fails……
    As expected changing to “A2” backs up perfectly – so once aghain I am changing it back to a proper name – “The quick brown fox” to see what happens:-
    Initially I get at the /facebook level a “can’t sync facebook facebook error” message, so I am going to leave it for while. And come back in the morning.
    After leaving it all alone overnight the file folder was backed up ok to the local drive.
    Conclusion… The app works well for a full backup of facebook pictures by reading all your filenames and creating name.cloudf containers, but it has a proviso that you must adhere to not using sequences of “illegal” characters/combinations in your FB names, that will fail when passing the albums file list from the FB API to the Odrive app. There are no helpful message in the error dialog boxes as to which file has failed! I had 27,827 files in 361 folders and occupying 8.02 GB. Eventually after a LOT of editing and fishing including using a DOS pipeline command (/facebook . > fred.txt) into a text file, to allow searching of a text version of the /facebook Odrive directory, which has the xxxxx.cloudf files written BEFORE it fails – when, at stage 2, it gets converted as a proper folder by the app which is itself attempting to fill the “container” with your pictures. Also, I gave up waiting for the app to update the cloudfs into proper folder, so I spent a lot of time right clicking / sync to sync up the files. Tedious, but it works. The “illegal” characters in my facebook literal titles were:- a lot of ………s, especially at the end of a name, and +++ inside the names, and 2017,2018,2019, in a sequence that seemed to upset the parser. The support guy was helpful but not interactive. I can see where people get angry with this app and leave it, but, after perseverance it seems to work although not exactly “elegant”. Facebook as you know send you a really down res ZIP file if asked, so this app is the ONLY method I know which tasks the FB API to send you the higher res versions. Happy shooting.

8

WHAT ??? I don’t agree and the app should inform users which file is failing the test and come on dudes, how many folk even know what a “CLI” or “illegal character” is??? All I want is a helpful app that automatically eats my FB folders, your app is clunky and not written with real users in mind… You should remember the old adage that “the customer is always right” even if they may be wrong, or misguided, and then (if you want to become rich) find out exactly what’s going wrong in the way THEY are trying to use your damn app. It does NOT download FB data if there are SIMPLE problems it cant handle, your whole company’s product is targeted on automatic “bulk” backups of data. Ive spent 3 days working on your app maybe I should send you a bill but I doubt you could afford me!!

9

My apologies, I meant no offense. You stated “my corporate users”, so I assumed that this was being done in a corporate IT setting for bulk sync/backup of company data. Since it sounded like you were in an environment where there were folks with technical knowhow, I wanted to offer a more advanced way to go about this that could help.

I’m glad you were able to resolve the file/folder naming issues. I have asked the product team to look at a better way of surfacing the errors to assist in identifying where those issue are.