FileCore long filename fail
Jeffrey Lee (213) 6048 posts |
File this under “things I regret offering to look at”. It’s been brought to my attention that using *cdir it’s possible to create directories with names longer than 255 chars, and that once created those directories may not be deletable via *wipe. Indeed, DiscKnight will complain that the directory name is bad and will attempt to truncate it for you. As I understand it 1K is meant to be the limit for pathnames and 256 chars for leafnames (including terminators?). However it’s rather hard to track down any specifications which actually state these as being the design limits. The best I can find is the RISC OS 4 features list, which only says “the maximum filename limit is 1024 characters”, making no indication of whether that’s the pathname limit, leafname limit, or both. Presumably there’s a specification somewhere for how the E+/F+ big directory structure is meant to work, but wherever it is it doesn’t look like it ever found its way into our CVS. As far as FileSwitch is concerned these limits are correct – see StaticName_length and util_objectname in FileSwBody (so is that 256 chars excluding terminator?). But for FileCore? It’s hard to say. BigDirMaxNameLen proudly proclaims that the maximum name length is 255 chars. But that value is only ever used for runtime stuff – it doesn’t seem to factor into the on-disc structure at all. The on-disc structure is entirely dynamically allocated, with no fixed length for the names. It even uses a word to store the length of directory or object names, so it’s fully capable of supporting names >= 256 chars. If you look at WriteName, which is used for old format discs, then you’ll see that it truncates the length of the name to 10 characters. But if you look at WriteBigName then you’ll see that no limits are applied whatsoever. Well, except for the remnants of some old limit-checking code, which was deleted a long time ago when long filename support was still in development (see manipulation of R2). Commence bogglement. So the naive approach to fixing the problem would be to add back a limit check to WriteBigName (and its friend, WriteBigDirName). But that won’t fully solve the problem, because there’s still lots of code which uses the untruncated length to allocate space for the directory (e.g. MakeBigDirSpace). So it looks like I/we will have to track down and fix all of those as well. And that’s not to mention all the other bits of the OS which are currently falling over when given long filenames. |
Rick Murray (539) 13872 posts |
(-: Hands up who knows that feeling. :-)
Might I suggest you start by writing this into some documentation somewhere, so that it is clear that “this is the spec, dammit!”. As you commented above – is that with a terminator or without? ;-) Most filesystems appear to support a filename of 255 bytes (so I suppose 256 with terminator) – http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits Note to everybody – bytes does not necessarily equal characters. How many bytes would be needed to represent “天体のメソッドPV1.mp4”? ;-) |
WPB (1391) 353 posts |
If you or somebody is having to go through and address this anyway, it could be an excellent opportunity to decouple the characters=bytes assumption that I’m guessing will have been made. At least that would be one less thing to worry about when the Filer and all its cousins get UTF-8 support. Although you may find that’s too much work. It often is nowhere near as trivial as it sounds, in my experience. Maybe I’m just slow, though! ;) |
Colin (478) 2433 posts |
Then of course there is the complication of the Wimp Message_DataLoad which limits the full pathname to 203 bytes. |
Jeffrey Lee (213) 6048 posts |
I think that’s the kind of thing that’s best left to be done properly, because there’ll be a lot of work involved to make the code behave sensibly when faced with a mix of discs using UTF-8 and discs not using UTF-8.
At the moment I’m hoping to just concentrate on fixing bugs in 15 year old features, not design flaws in 20+ year old ones :-) |
Sprow (202) 1159 posts |
You want the Ursula FileCore phase 1 and phase 2 specification which are document number 1309,207/FS and 1309,208/FS. There are mangled copies on the web, but it’s possible the catacombs under ROOL HQ contain the original master copies. I believe the limit for FileCore is 255 characters in the name plus 1 for the terminator. My evidence is mostly the cunning <= assertion for FcbName in the File Control Block definition, plus the handful of uses of BigDirMaxNameLen which have loop exits that are BLO that value, so array indices 0-254 for the name. Directories have most of the same properties as files, so there’s no distinction between leafnames and directory names. It looks like the RISC OS 4 press clipping was getting confused about 1k for the entire path, mostly imposed by the 1k command line limit. Of course in practice it’s a bit less than that since the shortest abbreviated command would be 2 letters then the name then a terminator. ie. 1024-1-1021-2=0.
And that seems reasonable in my head. The guts are generic and you only need to put sentrys on the gates – so the limit only needs checking when names are passed into FileCore everything else doesn’t care.
The other option for the short term is just to run fsbash from the root directory (which is what I’m guessing caused the problem to surface). Clearly after 15 years it’s not a use pattern that trips normal humans up, I lose track of directories after they’re nested more than 6 or 7 deep! |
Jeffrey Lee (213) 6048 posts |
Thanks!
Yeah, pretty much. Find one loose end, start pulling on it, and end up with a whole ball of yarn. I’ll probably end up expanding fsbash to allow it to generate leafnames longer than 256 chars (and a suitable warning to discourage people from running it on their main drive) |
Steve Revill (20) 1361 posts |
I ran this past Ben this morning:
and later:
|
Jeffrey Lee (213) 6048 posts |
It looks like marutan.net has copies of 1309,207/FS and 1309,208/FS as well. Thanks, Peter! |
fosjoas2015 (2631) 2 posts |
The HardDisc4 is intended for use on all platforms and as such does not contain platform specific components such as the CPU tool, which is by Andrew Conroy. All we need is a link. |
Andrew Conroy (370) 740 posts |
ROOL have made significant changes to the CPU tool since I originally wrote it, so I don’t feel it’s really mine to distribute any longer, I guess it’s become theirs. The original version, for the Pi 1 only, which I wrote (with considerable help from Jeffrey Lee and Theo Markettos) is at http://www.owlart.co.uk/Software but this is not suitable for use with the Pi 2. |
Chris Hall (132) 3567 posts |
If this is the ARM5/ARM7 (Alignment exceptions off)/ARM 7 (alignmnent exceptions on) then it is provided on RC14 in a form suitable for Pi1 and Pi2 (the first option being greyed out on Pi2). Separate similar tools are provided for Panda and Beagle and ARMX6 by R-Comp. |