How to cope with a faulty Territory module
|
We have been contacted by a user of our software who is using the Netherlands territory on an ARMX6 running RISC OS 5.23 (18-Feb-18). Our software crashes as soon as it is executed, with an undefined instruction error. We have replicated the error on our ARMX6. The problem arises when we call the SWI Territory_DaylightSaving with R0 = -1 (for the current territory) and R1 = 2. This call should indicate if the Territory supports this part of the API, which was introduced sometime in the last two or three years. The Dutch Territory module which we tested is 0.03 of 8 October 2004 so obviously predates this API. From PRM 3-794 (Territory_Register) it is clear that in the entry table for the national territory modules you are supposed to provide code for 43 SWI entry points, and those which were beyond the defined range at the time the module was created were meant to generate an error. It looks to me, therefore, as though the Dutch territory module is not doing this correctly, so the Territory_DaylightSaving call is dropping into the Dutch module at an entry point which the author never expected to be used, hence the undefined instruction. A quick look at the module in Zap confirms this: in fact the module uses a surprising method of building the entry table in its workspace, and this means that when the unexpected SWI is called the Territory module is jumping into uninitialised RMA, so it could do anything! Ideally, the Dutch territory module would be updated, but in the meantime, we would like to get our software working without the user having to switch to the UK territory. One method might be to trap the undefined instruction abort via a signal handler (the applications are written in C), but I do not have much experience of this. Another possibility would be to look out for the territory number being 34 (Dutch) and if so avoid the call. That would be a bad idea because then we would not get the enhanced functionality if the Dutch module gets updated. Another method would be to check for the version number of the Territory module, and only call Territory_DaylightSaving if the territory is not 34 or the module verison is greater than 0.03. But if we tried this method, how can we look up the version of the Dutch territory module (named “Netherlands”)? I’ve checked OS_Module but there seems to be no way to read a module version number. Do you really have to parse the module’s help string? Does anyone have any other suggestions for how to cope with this? |
|
I’ve managed to trap the undefined instruction signal, so the problem is solved. I still don’t like the way you can innocently call Territory_DaylightSaving and have such a horrible mess because of a poor implementation underneath, but there’s nothing can be done about that. The whole jump table specification seems a bit bizarre. Why did they limit it to 42 entries rather than the full 64? And it would have been better to have an API verison number passed in Territory_Register. It’s also rather unsatisfactory if you can’t read module version numbers without parsing the help string. Still, I guess people do not need to do it much. RMEnsure must be the main user. |
|
I’m sure you’re not the first person to ask about finding a module version number. I guess something like OS_Module 20 that works on softloaded modules as well as ROM modules might be handy? Edited to add: Last discussed here |
|
It is especially galling that Since module help string rules are slightly strange, here’s how to read the BCD version number of a module: ; > r0 module name ; r0 module version or 0 if no help string ; V set if module not present GetModuleVerBCD STMFD R13!, {R1-R5,R14} MOV R1, R0 MOV R0, #18 SWI XOS_Module LDMVSFD R13!, {R1-R5,PC} ; return error LDR R0, [R3,#20] TEQ R0, #0 ; no help? LDMEQFD R13!, {R1-R5,PC} ; return 0 ADD R3, R3, R0 ; 1st 16 chrs can contain ctrl codes, strangely MOV R0, #0 MOV R1, #0 skipname LDRB R14, [R3],#1 TEQ R14, #0 ; no version? LDMEQFD R13!, {R1-R5,PC} ; return 0 TEQ R14, #9 ; expand tabs ADDEQ R1, R1, #8 ADDNE R1, R1, #1 BICEQ R1, R1, #7 CMP R1, #16 BLT skipname ; thereafter any ctrl (except tab) terminates the string skiptodigit LDRB R14, [R3],#1 CMP R14, #9 CMPGT R14, #32 LDMCCFD R13!, {R1-R5,PC} ; return 0 SUB R14, R14, #"0" CMP R14, #10 BCS skiptodigit major ORR R0, R14, R0, LSL#4 LDRB R14, [R3],#1 SUB R14, R14, #"0" CMP R14, #10 BCC major MOV R0, R0, LSL#16 CMP R14, #"."-"0" ; decimal point? LDMNEFD R13!, {R1-R5,PC} ; return MMMM.0000 MOV R1, #12 minor LDRB R14, [R3],#1 SUB R14, R14, #"0" CMP R14, #10 LDMCSFD R13!, {R1-R5,PC} ; return MMMM.mm00 ORR R0, R0, R14, LSL R1 SUBS R1, R1, #4 BPL minor LDMFD R13!, {R1-R5,PC} ; return MMMM.mmmm And yes, it ought to be an OS_Module call. |
|
Very much so. Code ought to already exist – RMEnsure must read the version in some form… |
|
The previous thread I linked to, you posted a link to the sources which showed exactly how RMEnsure did it! Just the same as Nemo has posted a snippet of the RMEnsure code above :-) Yes, it has to parse the help string to get the version number. I wonder if OS_Module 20 parses the help string too? |
|
If you look more closely you’ll find it is not a snippet of RMEnsure but is compatible with it except for the obvious bug I fixed. |
|
For differing values of “obvious”! Has the bugfix been fed back into the sources? |
|
Depends whether you think there’s a serious problem with module help strings containing a VDU31 after the first 16 characters. On balance I’m inclined to not worry… but also to not repeat it. However, the permissiveness of the handling of the first 16 characters is surprising and not defensible really, though I have duplicated it. I thought carefully about the VDU31 bit, but since the algorithm is regarding all other ctrls as a terminator by that stage, it’s got to be an off-by-one. 1 Oh, no it doesn’t. Didn’t notice that change but it did, Quite Some Time Ago. Must be malformed strings. |
|
A mix-up between “greater than 31” vs “greater than or equal to 32” giving us “>=31”?
Looking at the code, it looks like it “just assumes” that the version number starts at a specific column, so it just counts characters to that place (and only cares about TAB when working out cursor placement). It doesn’t care what, only where. Of course, if it was originally envisaged that module names would only be 15 characters (plus a space/tab), they messed that up with “InternationalKeyboard”. ;-) |
|
and the answer to the question is? |
|
Nope. No point. Sadly, this is a ‘valid’ Module help string: <31><1><2>1.2<31><4><5>4.5<31><8><9>8.9<31><10><13>10.13 And you’re going to have to stare at that a bit to work out what its version number is. The bug Rick has spotted would disallow one of those bytes… actually no, it wouldn’t. So, yeah, no point. Module help strings are the Wild West. |
|
Right, it turns out I still have a problem with the faulty territory module: our signal trapping has not caught all the possible errors. The application is written in C. I am trapping the undefined instruction signal, and could easily trap address exception also. When the application calls Territory_DaylightSaving the Territory Manager looks up the appropriate address in the territory module’s jump table and jumps to that, expecting code which will provide the SWI functionality. With the Dutch territory module this address has not been set up, so it’s all a matter of what value there is in the module’s workspace where the jump table gets built. Unfortunately, on a freshly reset machine, it’s likely to be zero. So we then get this error: Internal error, no stack for trap handler: Internal error: branch through zero, pc=00000000: registers at 00138764. Is there anything I can change about my C application that I could trap this? Or is it so fatal there is no hope? If it cannot be trapped then I need to go back to the idea of reading the version or date of the territory module and avoiding ones which are too old for the jump points to have been implemented. Or I could try to update the territory module! |
|
Stating the obvious: Having tried one of the benign functions and got an error back, surely you then know not to try any of the others? |
|
No, that’s not what is happening. The function I want to use is Territory_DaylightSaving which is a Territory Manager SWI. The Territory Manager implements this by calling an entry point called DaylightRules which is one of the 43 entries in the jump table which a territory module declares when it calls Territory_Register. There are 43 entries in the jump table. The table contains the absolute memory address which the Territory Manager should jump to for executing the SWIs which are the responsibility of the territory modules. Because it’s an absolute address, rather than being relative to the module base (not sure why they did it like this, perhaps speed) most territory modules implement this by building the jump table in workspace claimed from the RMA when they initialise. In RISC OS 3 only 24 of the entries in the jump table were defined. The PRM tells you that the remaining 19 are for future expansion, and you should point the entries to a routine which will return an error (the number and text is specified). Sadly, several territory modules have been built where the remaining 19 are not even set up, so what you get in the jump table is unpredictable. Two new entry points have been added in the last few years, to provide Territory_IME and Territory_DaylightRules. I had initially thought that the only abort encountered when I call Territory_DaylightSaving would be the undefined instruction signal. But further testing on a variety of machines, with territory initialisation happening at different stages, shows that we can get other situations. So I was posting about a further case I would like to be able to catch. But I am not sure whether the C runtime environment can do this, hence the question above. So I’m afraid the benign functions (the 24 that have been implemented since the early days of RISC OS 3) do not give errors. The function I am interested in gives a variety of errors depending on the state of the RMA when the workspace is claimed. |
|
You could call Territory_Number to get the current territory, then issue Service_TerritoryManagerLoaded and inspect the table passed back to determine if it contains valid entries for the SWI’s you want to call. |
|
But Service_TerritoryManagerLoaded does not return the address of the table. It is the cue for territory modules to get them to call Territory_Register and that call has the details of the jump table location. I suppose I could intercept Territory_Register somehow and get the information, but that’s overkill for a Wimp application. If I cannot trap the possible errors (advice on this still welcome), then it would be best to watch out for the faulty module and/or try to get the module fixed and distributed. Your idea would be excellent for adding to the Territory Manager, however, so that it would be responsible for inspecting the tables it is given. But if we’re going to alter the Territory Manager I would favour altering Territory_Register so that territory modules have to declare the most recent Territory Manager they are aware of, so that the manager can avoid doing anything they cannot be expected to have implemented. |
|
Would patching the module – to zero the rma at startup – help? ie when swi Module 6 is called. |
|
Well, quite. You’re trying to work around what appears to be a fatal flaw with Territories and relying on a jump table to fail in a predictable way isn’t really a viable option. Personally, I’d create a seperate utility (or Module if the Territory can change) that does the following:
If there are various revisions of the Teritory jump table, but no way to determine which revision a Module supports, then the OS should be fixing the tables when they’re registered. The only real solution however is to fix the source of the issue and modify every faulty Territory Module. |
|
OIC.
There’s no guarantee that the jump table is within the module. Indeed, it comprises absolute addresses, so it probably isn’t.
[Edit: Nope, it’s the API that’s broken. RO5 will need to change. See later] |
|
The territory modules are available from this download: http://www.a4com.de/riscos/down/IntlTerrV120all.zip The Netherlands one is territory number 34. The others have the same issue, I think. I’ve only had a Dutch user with this problem. But please don’t put yourself to any trouble. Someone has just sent me a new Dutch territory module which solves the problem. I cannot distribute it to the end-user yet, however, because part of the module is derived from sources under a different licence. The developer who sent it to me is in the process of contacting the copyright holders. |
|
Thank you for that. OMG. Right, this is a total cock up. The NL territory module in that download is basically identical to the UK territory in RO4. So this isn’t a misreading of the documentation by a third party, this is yet another case of the documentation being dead wrong. Matthew wrote:
There’s no point saying that the remaining jump table entries are reserved and must point to an error, if you then build code that does not do that. RO3 does not do that. RO4 does not do that. RO6 does not do that. In each case, the jump table is constructed at the end of the module’s workspace. The supported entries have absolute addresses in them, the others are simply not initialised to anything, so are random on all versions of the OS except RO5, and all Territory modules in that A4COM download.
THIS is what happens when people write stuff against dodgy documentation without checking the actual implementation. RO5 will have to change. Because in all existing Territory modules there is no way to tell whether the ‘reserved’ entrypoints have been initialised or just accidentally contain a plausible address, the This is unavoidable. 1 I’m thinking a bit in R0 – b31? |
|
PS. Whoever added the initialisation to the RO5 version should have realised what its absence meant for the alleged API. Where’s the facepalm emoji? |
|
Or trusting that people followed the sort of spec and initialised the full entry table correctly instead of taking shortcuts because, oh no, stuff like that has never ever happened before, right? Before you dig out the facepalm emoji (it’s new, my phone doesn’t have it, so here’s Mr Poo instead 💩) spare a moment to think that if all of the existing territoires on the older systems are “broken” … doesn’t this indicate that the RO5 modification was thus made without bothering to test against territories already “in the wild”? Maybe we should upgrade to the mushroom cloud emoji? Okay… There’s no mushroom cloud. I can have headache girl 🤦, Friday night fever 🕺, naughty tentacles 🐙, or a frickin’ coffin ⚰️, but no mushroom cloud… Is there an emoji font for NetSurf? Hmm, we’ll then run into the problem that most emoji are in colour…but I don’t think FontManager can cope with that? |
|
OS devs don’t always follow their own rules though, as they have insider knowledge on when API’s etc change. In other words, if RO has only ever supported 24 jump points internally and the Territories were bundled with the OS, there would have been no need to implement 43 jump points. If this is the case, the Territories have simply not been updated when the OS started supporting all 43, which I’m going to guess was done in RO5. Were these Territory Modules lifted from a previous OS and then 32bitted? If so, they just need modifying.
If API changes are being considered, consider also making b31 specify the jump table is relative to the Module base, so it’s a static table. |