Porting ARM32 BBC BASIC to ARM64
tymaja (278) 138 posts |
I was going to put this in Aldershot … but it does raise a point : Acorn BBC BASIC is intrinsically tied to ‘assembly language only’, but also to ‘ARM32 only’. Yet, a port is needed (to ARM64, and one day to RiscV 32/64). ARM BBC BASIC was created for the ARM Evaluation Kit, so existed before even Arthur, so is in many ways a fundamental part of RISC OS. So … is such a port possible (given the esoteric nature of the ARM32 code)? At some point, there will be a need to do this conversion. The current BBC BASIC code is tied to ‘assembly language’, in that converting it to C is pretty much impossible (at least in an efficient manner), because it relies very heavily on passing information in the status flags across many subroutines. A C conversion of the current code would need to use an emulated status flags global variable, which would need to be manually updated at each stage (with a lot of use of carry, so lots of complex code at each step to emulate ARM32 carry). One challenge to converting it to ARM64 assembler would be the extensive use of ‘TEQ’ to do comparisons without altering carry… Essentially, each TEQ needs to be fully understood in terms of impact to flags etc, and a specific replacement needs to be decided upon – that alone is a lot of work! The above is one example where it will be a challenge rewriting BBC BASIC to ARM64 Has anyone tackled this issue thus far? Any thoughts? |
Peter Howkins (211) 235 posts |
You are mistaken on both these points. BBC basic is already happily running on arm64, with two implementations written in C. |
David J. Ruck (33) 1585 posts |
It would make more sense to base any 64 bit version on an existing C implementation, particularly as if there is ever a 64 bit RISC OS, it’s OS API is likely to be almost as different to 32 bit RISC OS as the cross platform code is. |
tymaja (278) 138 posts |
This is why I said Acorn BBC BASIC is tied to assembly language only, and ARM32 only (emphasis on Acorn) – as I was referring to the BBC BASIC that comes with RISC OS, as opposed to the reimplementations of BBC BASIC. I guess I was thinking about not having to discard the original BBC BASIC source code entirely (which it seems we logically* have to at some point I of course agree that a 64 bit RISC OS API would be very different from 32 bit RISC OS, although there would be some similarities – and the Acorn RISC OS BBC BASIC is quite well abstracted (BBC BASIC for ARM evaluation system loads and runs without modification on RPi4, with error handling working, printing the intro string and >, OS_ReadLine etc, and only later throws errors due to 26-bit code). ARM BBC BASIC existed before Arthur even – It is surprisingly well abstracted (and probably influenced how early RISC OS was developed). The MOS emulation itself was probably very useful before ARM26 MOS itself existed! Edit : RPi 5 changed to RPi 4 re : BASIC 1.0 |
David J. Ruck (33) 1585 posts |
ARM BASIC was written by Sophie Wilson who both wrote the original BBC BASIC and designed the ARM instruction set, so there will not be a single clever trick you can do with it that will have been missed, including all the problematic 32 bit integer and 5 byte float equivalence stuff we’ve discussed elsewhere. None of those tricks are suitable for the rather generic and boring ARM 64 bit instruction set, which is designed for compilers to churn out rather than assembler wizards to hand craft. Taking a C implementation where the functionality has already been extracted from the clever coding, will save a lot of time, and make it hell of a lot easier to work with whatever OS API is chosen, and most importantly; to enhance further with modern programming constructs. |
Clive Semmens (2335) 3274 posts |
Very succinctly and accurately put – and the basic reason why I decided to retire when I did…the future at ARM was beginning to look boring!!! |
Sprow (202) 1150 posts |
Would it though? The survey of the current API says much would look and feel the same. If you stick to The main gotcha in API terms is when we put pointers in memory, because now you want an 8 byte quantity in a space reserved for 4. The above linked document enumerates those problem ones, and it’s only about 50 – work to do, but tractable. |
Rick Murray (539) 13747 posts |
Umm, yes? Because if you’re rewriting RISC OS in ARM64, you’re basically kicking the can a little further down the road and remaining heavily tied to specific processors. And if you’re looking to rewrite it in C, I can’t help but think that it is going to be a nightmare to try to get C interacting with a highly assembler-specific OS that has utterly no concept of a calling convention other than “whatever we decided this SWI requires”, not to mention weirdness with the flags for some of them. Plus doing it like that will still remain tied to specific processors. To be honest, I can’t help but feel that any implementation that doesn’t try to fix the many architectural issues of the past would be a fool’s errand. This is, for most software, a breaking change (only pure BASIC will escape unscathed), so it’s a good opportunity to fix all of the bad bits and jiggle the Wimp to support multiple cores (not to mention pulling the task switching out of the Wimp). This is irrespective of whether or not some LLM is used. If a ‘bot is not used, you’ll spend much time going through each line one by one to translate it. And if you use a ‘bot you’ll spend just as much time going through to ensure the output is coherent and it isn’t blithely screwing things up. Oh, and don’t forget that there’s no reason why a shiny new OS based upon RISC OS principles but written in C can’t be backported to existing 32 bit systems. |
Clive Semmens (2335) 3274 posts |
I have a nagging suspicion that this is all cloud cuckoo land, whichever option one is considering. |
Simon Willcocks (1499) 505 posts |
Meh. Having spent years on trial and error, mostly error, I’ve come to the conclusion that, like the Black Ship in the Hitchhiker’s Guide to the Galaxy, the OS is best left alone, as much as possible. Legacy code that expects to have to manage context changes, application entry and exit, callbacks, etc. has to essentially run under a single thread (actually a single legacy SVC stack). New code, though, can run on multiple cores, have transparent context changes, and communicate via simple mechanisms (pipes, basically buffers, but the tasks sleep while waiting for new data or space). They could even be 64 bit, but don’t hold your breath. (I did work on a 64-bit OS with a wierd mechanism and 32-bit VM, but much of it can be repurposed to be more RO-like while more secure.) Right now, I can’t even get RO to build. |
Grahame Parish (436) 476 posts |
Asking without any relevant knowledge, but would it make sense to create an emulator to run existing BBC Basic code rather than create BBCBasic64? I’m thinking that new applications would be written in something more modern and portable like Python, C++, Rust(?), etc., so we would only need this for supporting older code to stop it being lost. |
Simon Willcocks (1499) 505 posts |
32-bit emulators are ten a penny. |
Rick Murray (539) 13747 posts |
Run one within another. There. 64 bits. Easy. 😂 |
tymaja (278) 138 posts |
Agreed (this is one of the reasons I wouldn’t want to ‘lose’ the original source code) – it is a work of art, and tied to the creation of RISC OS, and even the ARM processor itself (quite a lot of the functionality lost during the switch to ARM64 were things that were very useful in BBC BASIC, such as RRX, shifter carryout in general, TEQ to compare without messing with carry): these are amazing features for assembly coders, but C barely or even doesn’t use these, so compilers didn’t need them, do they were dropped for ARM64 😬 Regarding emulation, I started work on an ARM32 emulator a few months ago; the platform was ‘a RISC PC with an ARMv8 processor’; I started by getting BBC BASIC to work (because that is a good way to ensure you have shifter carryout sorted, reducing the number of annoying bugs when trying to boot RISC OS itself). I managed to get it to boot to desktop, and learnt a lot doing so! Back to Acorn BBC BASIC – I want to learn how it really works, and want to learn ARM64 as well, so I have started work on an ARM64 port as of 2 weeks ago (attempt 2; attempt 1 was very experimental / helped me understand the code). I am about 50% done so far. I am writing it on Raspberry Pi OS on a Pi 5, and currently have 1.5x the speed of ARM32 BBC BASIC (the aim is 2-3x the speed on the Pi5, as the Pi5 is 2-3x faster than the 4). I am not using CBZ, CBNZ, TBNZ, TBZ, CCMP or CSEL yet, but am leaving comments in the code where optimisations will be possible. I have kept the code structured the same as the RISC OS ARM32 source. I also have an unoptimised TEQ macro, so that is another place for speeding things up. I am writing it ‘as if’ it would be on an ARM64 version of RISCOS64 in the future; it is well abstracted, so I have made macros for SWI, so I can just put RISC OS SWIs in the source code, and it works as expected. I made a simple VDU engine (enough for text, using the Zap larger characters for now). My aim with this project is to learn ARM64 (so I may be able to help with the low level code of RO64 in the future), to learn how the ARM32 BBC BASIC code really works, so, among other things, I can overcome the challenges of making a dual 32/64-bit BBC BASIC in ARM32 code … without needing to worry (yet) about the register shortages on ARM32 BBC BASIC. Once I am able to use 64-bit ints, and can also run various RISC OS ARM32 BBC BASIC programs (with 32-bit integer support), I can then add 64-bit int support back to RISC OS ARM32 BBC BASIC! Once this ARM64 BBC BASIC is fully / mostly complete, I will also write an EL2 hypervisor’ that (on the Pi 4) keeps the upper 4GB of ram free, hands control to RISC OS32 running at EL1, but then hands a core to BBC BASIC 64 so that I can experiment with 64-bit stuff while also having RISC OS as a development environment (it will involve a single semaphore being added to the RISC OS workspace 🤪) Will do a bit more work now (started Array code yesterday, mostly done too!) Edit : I should add that the ‘abstraction’ I have done uses assembly to a degree, but the I/O is actually abstracted to C functions, making it easy to compile on various OSes etc, as long as they have ARM64) |
tymaja (278) 138 posts |
I agree – although if an ARM64 port of RISCOS32 appeared, it would kick the can 30 years forward (assuming ARM64 lasts as long as ARM32 did in (ARM CPUs with MMUs etc); My thoughts on the transition of RISC OS to 64-bit include: Of course, the API, memory allocation, workspace layout etc needs to be decided upon, before any work can be done there! I have started porting the VDU drivers to ARM64 though, as I will tie those into BASIC once BASIC works. They are less well-abstracted than BASIC is, but some ‘wrapper’ code to set up the workspace VDU area has allowed me to get them to begin to work … but I have put that aside until BASIC itself is done – although it currently works well enough to make use of Acorn ARM64 VDU drivers if they existed |
Clive Semmens (2335) 3274 posts |
I think that’s a very shaky assumption. “generic and boring” was David’s very apt description of ARM64. ARM26 was, in its own way, ground breaking; and ARM32 was a largely non-deal-breaking upgrade of it. That it lasted so long is only a bit surprising; for a generic and boring instruction set to last that long without deal-breaking “upgrades” would be very surprising indeed. ARM64 is – as David wrote – designed for compilers, not for human wizards. Don’t try writing in assembly language for it (which is why I really wasn’t interested in writing assembly language manuals for it). And if you’re writing in C (or, preferably, some better higher language – has anyone designed one yet? It’s bloody needed) then you’re not tied to the particular architecture. |
tymaja (278) 138 posts |
I think that’s a very shaky assumption. “generic and boring” was David’s very apt description of ARM64” I do agree mostly; in a way it will be good if ARM64s dominance is challenged, because it does take a fair amount of the ‘enjoyment’ of ARM32/26 away. I do want to find out if there are ways to use the characteristics of the instruction set for clever optimisations, though (as an example, losing shifter carryout and TEQ is annoying – but some things, such as MOVK, CBZ / CBNZ are good additions to ARM64, and things like ANDS wiping the carry flag may be useful in certain ways (a trivial example of CBZ is reading through a zero-terminated string; .loop LDRB W0,[X1],#1 CBNZ X0,loopIs quite succinct, and the fact it doesn’t touch any flags at all can probably be used in clever ways. Other things like the bitmask immediates are interesting, as are things like TBZ/TBNZ, and the use of a zero register. It is sad that Apple essentially dominated ARM, and changed the direction of ARM during the shift to 64 bits. And also sad that ARM32 is being slowly deprecated. I have most of ARM BBC BASIC running on ARM64 now, and in reality I am doing it in my spare time, as a ‘hobby’, not as work, and it is already fast on ARM64. The worst case scenario is a fast ARM64 BBC BASIC V/VI interpreter for ARM64, which follows the design of ARM32 BBC BASIC, so may be useful at some stage for RISC OS. Plus I get to learn ARM64, which may be helpful for low level 64 bit kernel coding in some areas. Understanding the BBC BASIC code means I could also port it to RISC V 32/64 in the future (I prefer assembly over ‘higher level languages’, but have finally started doing some C as part of the project) |
Chris Mahoney (1684) 2160 posts |
I can’t find it now, but someone managed to make a very basic OS in C#. I use the language regularly but I can’t imagine e.g. rewriting RISC OS in it! Edit: Found it! |
Clive Semmens (2335) 3274 posts |
I don’t know C# – how much of an advance on C is it? I’m imagining something that makes C look very, very primitive… sadly, only imagining it, not even visualising what it might look like… |
Chris Mahoney (1684) 2160 posts |
Quick and dirty example:
Even the simple stuff like |
tymaja (278) 138 posts |
Current status (ignore the missing parts of some letter ‘g’s – I just put some 1-bit characters into my VDU code (which is in C, and is just enough to get characters on the screen for testing – it is not the RISC OS Vdu code). Got arrays mostly working today – have a few bugs in string arrays to sort. Also added colour as well (via the ‘proper’ methods – using the BASIC COLOUR command, which does VDU 17, (colour) to set the colour! (this is on a Pi5, and is an ARM64, except for the VDU code, and IO, which goes to some C code using a ‘SWI’ macro in the assembly language code – meaning I am using SWIs that conform to RO32 standards (except 64-bit registers), also meaning the code isn’t tied to any particular OS – as long as a few simple SWIs are implemented in C, then it will run fine :) |
Rick Murray (539) 13747 posts |
Isn’t that heavily tied into the .Net runtime? Doesn’t it compile to an intermediate form that is often JITed into it’s final form as necessary (akin to mobile platform apps)? Not saying it’s bad, just that maybe it’s not the best choice for RISC OS?
That’s objects and such isn’t it? (spot the plain C programmer! ;) ) |
David J. Ruck (33) 1585 posts |
I didn’t think I’d ever like anything coming out of Microsoft, but C# is actually a pretty good language, it’s what C++ should have been if they were able to ditch some of the C legacy and not bodge everything on top. It’s got a very clean syntax, and a large standard class library with lots of useful collection classes. It’s main disadvantage is is semi-compiled and requires a runtime instead of compiling to raw native code, and it is a managed language with garbage collection which can cause latency issues. I mainly used it on Linux with the mono runtime, although I did write it on Windows with Visual Studio. I think they were up to version 4 back then, they are on 12 now, and shovelled far too much unnecessary crap in to it. I would not want to go back to using plain C now, particularly manipulating strings with just stdio and without a string classes. You can get most of the functionality of C# using C++14 onwards with the Standard Template Library, but it’s oh so painful compared to the simplicity of C#. Now I mainly just use Python, which has an even richer set of support libraries than C# and a nice syntax, as long as you just get over the use of white space. The only things it doesn’t do well is multi-threading and absolute performance. |
Andrew Daniel (376) 70 posts |
Just going back to Brandy Basic, I seem to remember that Sophie Wilson was very complementary about it and it’s speed of execution on a RiscPC. |
Colin Ferris (399) 1797 posts |
!Zap running on ARM 64 :-)) |