Resolver 0.73 is broken
Dave Higton (1515) 3412 posts |
Resolver 0.73 is broken. There are two symptoms of failure: one is that entries in the Hosts file are correctly looked up, but must be matched case sensitively; the other is that lookups in the Hosts file can fail completely. Note that, if something is being looked up that is in the Hosts file and also in an external resolver (e.g. one’s router), it will be looked up case insensitively in the external resolver, and may mislead one into thinking that Resolver 0.73 is working. See the thread in Community Support, “Is Resolver 0.73 broken?”. |
djp (9726) 54 posts |
Resolver 0.73 change log |
Dave Higton (1515) 3412 posts |
I may have misunderstood, but it looks like the real code is a binary blob, for which we can’t see the sources. © Ant Ltd. They’ve been updated, hence the header going from 0.72 to 0.73. |
Steve Pampling (1551) 7989 posts |
Well, that code from Ant has been broken in a variety of ways for over 20 years, one more peculiarity is just a small bit of decoration on the icing on the cake. |
Dave Higton (1515) 3412 posts |
Yes, but it has just gone from working to not working. Pretty clearly. |
Dave Higton (1515) 3412 posts |
The important questions are about who will fix it, and when. All the open source parts of RISC OS are available for diagnosis, fixing and review by anyone. Not the binary blobs, though. So who kicks off a fix? Do we need to notify ROOL so that they can? We can’t rely on them reading here AFAIAA. |
Andrew Conroy (370) 725 posts |
Hopefully the same person who broke it in the first place, as they should know what they did and so can undo it! |
Rick Murray (539) 13476 posts |
Another important question being why was this apparent change not mentioned in the change log? Because neither “Drop COMPAT_INET4 support” nor “built with cc 5.89” should have changed the behaviour (unless something is very wrong). |
Steve Pampling (1551) 7989 posts |
Unless the previous behaviour is INET4 compatibility and anything now sending queries to Resolver is expected to deal with mixed case response to a mixed case query. |
Dave Higton (1515) 3412 posts |
Which was the buggy version of the compiler? |
Rick Murray (539) 13476 posts |
Case shouldn’t matter. |
Doug Webb (190) 1151 posts |
Looking at the bug tracker I can’t see any entry for this issue so I am assuming that none has been made but it is expected that magically discussing it here should get it fixed? Out of interest has a test been made with a totally clean install of both the ROM/Firmware elements as well as the latest hard disc images just to further help narrow anything down that may or may not be interacting with Resolver 0.73? Normally I used to be in the habit of testing the new ROM’s and Hard Disc images each day they became live but have fallen out of that recent and on this one may not have seen it anyway as i am testing the ROD stack as well. |
Alan Adams (2486) 1131 posts |
My reading of that is slightly different – I think it says a name in all uppercase must match an uppercase or lowercase version, and similarly an all lowercase name must match either all uppercase or all lowercase versions. I don’t see anything that allows mixed case names to be matched against either all uppercase or all lowercase. |
Stuart Swales (8827) 1261 posts |
The case-insensitive comparison is done on a per-byte (octet) basis. And that’s immaterial to whether Resolver appears to be using Hosts at all, even when the same case name is given as that in the file. |
Rick Murray (539) 13476 posts |
Section 3: That is to say, a lookup string octet with a value in the inclusive range from 0×41 to 0×5A, the uppercase ASCII letters, MUST match the identical value and also match the corresponding value in the inclusive range from 0×61 to 0×7A, the lowercase ASCII letters. A lookup string octet with a lowercase ASCII letter value MUST similarly match the identical value and also match the corresponding value in the uppercase ASCII letter range.I’ve highlighted the pertinent parts. It’s not saying a string that IS ALL UPPERCASE or is all lowercase must match, it’s saying a value within the string as reading it character by character must match. In other words, case insensitive (at least for regular ASCII). But, yeah, there appear to be “other issues” too. |
Alan Adams (2486) 1131 posts |
Indeed. However A-umlaut might be expected to match a-umlaut, and maybe also A or a, which from the above would not happen. When I first encountered DNS (1980’s?) I seem to recall the character set was limited to uppercase and lowercase ASCII, numbers and one of minus or underscore – I can’t remember which. Things change… |
Steve Pampling (1551) 7989 posts |
Not as clear-cut as “it’s all case-insensitive” As Alan notes:
and then the document Rick quoted a part of, but omitted other bits. 2. Case Insensitivity of DNS Labels and for extra fun: 4. Case on Input and Output Which looks rather like the Resolver is supposed to preserve case, but what other bits do with that preserved case string in comparisons is up to the other bits – i.e. as I said earlier the update in the Resolver is highlighting a case sensitivity problem elsewhere. |
Dave Higton (1515) 3412 posts |
I’ve filed ticket 576. |
Doug Webb (190) 1151 posts |
Hopefully that will get even more attention. Also given this statement:
Would it not be prudent as I said earlier to just do a new sdcard build using the latest ROM and Hard Disc image and see if the issue is still there? |
Charles Ferguson (8243) 395 posts |
It looks just broken to me. Doing a eg: ResolverConfig Resources:$.Resources.Resolver.Messages BadSWI %.*s.%.*s /etc/hosts Inet$HostName /etc/hosts Inet$SearchDomains Inet$LocalDomain If I compare this to the Resolver module I have lying around (an old Select one, but it’ll be approximately the same… gethostname Inet$HostName makehost _gethtent _gethtbyname InetDBase:hosts _inetaddr _inetraddr _gethtbyaddr _getfwbyname _getfwbyaddr freeway_init freeway_final _getnbbyname getanswer res_query res_search res_querydomain %.*s.%.*s res_launch hostclear cacheclear cachemerge cachecheckpurge cachefind InetDBase:hosts cacheadd cacheinit cachefini However, we can be clearer with this and actually link the module and use it. Using the build script: #!/bin/bash set -e riscos-kitten Resources.UK.Messages Resources.UK.CmdHelp > resjoined riscos-resgen -apcs 3/32/nonreent/fp/swst Resources aof/resources resjoined Resources.Resolver.Messages riscos-link -verbose -rmf aof/ResolverSA aof/resources C:o.stubsG -o Resolver,ffa And then if I turn some FS debug on and load the module, issuing a ResolverConfig and printing the SHA of the repo I was testing… charles@laputa ~/external/ResolverBlob (master)> pyrodev --common --command 'rmload Resolver' --command gos --boot-debug osfile,osfind Read catalogue info with path Filename = 'Resolver' Path = None DirEntry = <DirectoryEntryNative('Resolver'/'Resolver,ffa', 1, &fffffa5a, &52c27f0f, 20300, &33)> Load file Filename = 'Resolver' Path = '@.' Read catalogue info with path Filename = '/etc/hosts' Path = None DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)> Read catalogue info with path Filename = '/etc/hosts' Path = None DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)> Supervisor *help resolver ==> Help on keyword 'Resolver' (Module) Module is: Resolver 0.73 (16 May 2022) (Acorn) © ANT Ltd, 1997 Commands provided: ResolverConfig *resolverconfig Read catalogue info with path Filename = '/etc/hosts' Path = None DirEntry = <DirectoryEntry('/etc/hosts', 0, &00000000, &00000000, 0, &0)> *git rev-parse HEAD 1c33b5a299d5dba129bbcc18e118f3ab42428dfb I don’t think I really need to go any further. The module as is presently on that branch is just stuffed. I don’t quite see how you’re getting to the point of finding case insensitiveness being an issue as it shouldn’t even be able to read the hosts file, but that’s where I’d start looking for what the problem might be. I may be a broken record, but… automated testing? |
Doug Webb (190) 1151 posts |
The sound of common sense again is heard :-) |
Rick Murray (539) 13476 posts |
I may be a broken record, but… FTFY. |
Doug Webb (190) 1151 posts |
Any testing is better than none but you have to have a plan or a set of criteria on which you are going to test and the expected outcomes you have if you want to do it correctly. Are you testing new features? Just getting something out and then thinking because you have no feedback it is Ok isn’t the best option. As I said I used to test the new daily ROM’s and hard disc components as they came out but have got out of that habit for one reason or another and perhaps that has replicated a bit across all regular users. Compounding this specific issue is that I, like I guess many users, just do DHCP and I also have the ROD stack I am testing/have tested. Anyway hopefully now it is highlighted thanks to Dave then this can be fixed. |
Charles Ferguson (8243) 395 posts |
To be clear, my own tests that check the RISC OS Pyromaniac resolver’s Hosts file implementation checked whether the addresses could be looked up in the configured file, whether the entries worked for named hosts and for aliased hosts, and that comments were ignored, plus a few of the different ways that the SWI calls can be used. I did not have any tests for the behaviour being correct when different character cases were used, nor for checking whether the |
Steve Pampling (1551) 7989 posts |
1 I have literally lost count of the occasions where suppliers did not follow the instruction to “set to DHCP” and were surprised when what they did failed to work because they missed a critical aspect or mistyped a value or two. Or their system didn’t properly register in DNS and parts of our config relied on the FQDN rather than a fixed IP. |
Dave Higton (1515) 3412 posts |
Charles, thanks for your interest and your input into this case. |
Rick Murray (539) 13476 posts |
Static IP with DHCP reservations on the router. Why? Easy. Livebox boot – measured in minutes. Pi boot – measured in seconds. When I set up everything, DHCP only worked in RISC OS for a few seconds at boot.
I’m using a slightly older ROM, but if I hadn’t been, then this is why I’d not have noticed. 1 Of course this isn’t so useful if DHCP can be invoked by the network connection coming up, as that’s the Vonets widget, and that too boots much faster than the router. |
Dave Higton (1515) 3412 posts |
One difference between DHCP and manual IP addressing, for me, is that my router’s DHCP server does not allow aliases for host names, while the Hosts file and Resolver clearly do. |
Steve Pampling (1551) 7989 posts |
Static IP with DHCP reservations on the router. One of the reasons that the router and the NAS are connected to a UPS :)
That I can go with. CNAME functionality is in proper DNS servers and standard software on home routers doesn’t do that |
Colin Ferris (399) 1757 posts |
Mmm – I wonder if the TV would work from a UPS. Wood fire – Candles :-| |
Rick Murray (539) 13476 posts |
Same here, but that’s a more recent thing borne of a great annoyance at the thing rebooting at the slightest blip in power, something they the Pi simply shrugs off.
Mine’s a 12V output. Because of this (no losses going up to 230VAC and down again), it can run for quite a while. I didn’t notice a half hour power cut in the summer as I had the light off (shutters open) and was using my phone to access the net. ;) As to the TV, if it’s a flat panel then it is basically the same idea as a monitor, and a large UPS can run those as well as the base computer. But for how long depends upon the rating of the UPS as well as the amount of loss involved in step up (in the UPS) and step down (in the TV) of the power. |
Chris Hughes (2123) 311 posts |
The Resolver 0.73 is I believe closed source and belong’s to ANT Ltd. But the note on the information page regarding the RISCOS Development webpage for the new TCP/IP says: Has this combination been tested? I also note on ROOL home pages some changes to the ROOL TCP/IP stack recently as well. |
Steve Pampling (1551) 7989 posts |
Define “all machines” |
Rick Murray (539) 13476 posts |
Probably if you have a fast PC and the habit of keeping it on… …I find just using native silicon is easier and more energy efficient. |
Steffen Huber (91) 1945 posts |
I am pretty sure that it was only noted for the main stack and the utils, but not for everything. Since it is likely that both MBufManager and Resolver are based on stack-independent code (MBufManager probably clean-room implementation, Resolver possibly based on c-ares or something?), I would guess that they might be compatible. If the provided binaries are, is of course a totally different question. And if they were tested at all in this combination…who knows. I use RPCemu only in a RO4 setup, so I will not be the one wasting time on recompile/test. |
André Timmermans (100) 637 posts |
I can confirm that Resolver works with the old stack at least on a PI4 (I havn’t tested on RPCEmu). You just have to load modules CryptRand, SLAAC and Resolver within the ROD archive ( Install.Boot.Resources.OBSD.rm(v5)). That said it is also using the ROD Resolver’s new API for IPv4/v6 independent address resolution that led me to the discovery of the incompatible layout of struct addrinfo between the ROD stack and ROOL’s new TCPIP beta library. Note: since then I have received a test Resolver module that allows to specify whcih layout of addrinfo to use. |
Steve Pampling (1551) 7989 posts |
Saving you all the effort, I tried loading it on RPCEmu, and it complains about “undefined instruction at &202874ec” you can get it to fail differently (abort on data transfer) if you load the ACE ARMChip module. So, not suitable for RPC/RPCEmu is probably the “lesson learned”1 from that one. Not that I was expecting it to support multi-decades old kit and emulations thereof. 1 Just for you Rick, because I know you love that wonderful management speak ;) |
Graeme (8815) 92 posts |
It would be interesting to see what instruction is failing. If you (or anyone) sees this again, could you post the actual instruction of the undefined instruction error (i.e. without ACE module loaded). This can be done with the star command *memoryi address +4. Replacing address with the hex value after the “undefined instruction at” error. I could then check if there is a mistake in my ACE module. |
Steffen Huber (91) 1945 posts |
You seem to have skipped the “recompile” step. |
Steve Pampling (1551) 7989 posts |
Yup. The question was whether what had been released was suitable for all – it isn’t. |
Steffen Huber (91) 1945 posts |
You even quoted parts of the posting that asked that question (although I didn’t include a question mark and left that as an exercise for the reader), including the word “recompile”. |
RISC OS Developments (9008) 32 posts |
Forgive me if I’m mis-understanding the last few posts. If the question is “does the new resolver etc work on RPCemu / old stack”, I believe the answer is “yes, but you’ll want to use a recent beta – please drop us an email and we’ll send you a copy”. Or compile your own from the sources, but easier to just get an RPCemu-safe version of the stack. The stack has been working on pretty much all hardware since May (v7.02 beta), but we have had several minor hold ups in getting 7.02 a “release” tag. Nothing that stops it being useful, but this is the downside of wanting to deliver something that is great for everyone, whilst simulatenously having to softload on top of unpredictable base !Boot sequences and network setups. There have been several public beta releases of 7.02 in the meantime. |
djp (9726) 54 posts |
Are the sources available somewhere? |
RISC OS Developments (9008) 32 posts |
They are often included with the betas, and can be requested as described on the download page on the website. It’s licensed under CDDL. |
djp (9726) 54 posts |
Blob resolved. Resolver 0.74, in today’s betas, is good here. Thanks. |
Dave Higton (1515) 3412 posts |
Thanks, djp, I’ll look it out this evening. |
Dave Higton (1515) 3412 posts |
I’ve given today’s build of RO a bit of a thrash, especially the Resolver module, whose version 0s 0.74 (12 Dec 2022). It works fine for me. RasPi 3B+. For my curiosity, what went wrong? Did it require a new binary blob? Who has the source to fix it? (No, I don’t want it; this is merely to satisfy my curiosity.) |
Stuart Painting (5389) 691 posts |
The old code was looking for /etc/hosts rather than InetDBase:Hosts which didn’t work all that well. |
Steve Pampling (1551) 7989 posts |
Thinking about it, if you look at what Charles said earlier the strings output says it has /etc/hosts there and not the normal RO style path to hosts So, Gerph being his usual educational self. His answer said not just that it was broken, but also gave the evidence of how it was broken. |
Charles Ferguson (8243) 395 posts |
Last year I created a video of how this bug was found, as I felt it was a simple example of how you might investigate the problem with RISC OS Pyromaniac. Largely this was very similar to the description I posted, but it’s often easier and nicer to see the process in a video. You can find the video on Youtube and on my presentation site: https://presentation.riscos.online/debugging/index.html |