Linux Port

I have uploaded my Port of RISC OS to Linux to github.com, which is intended to be it’s temporary home. There are few non-Linux changes as well. If you are asking what the use of a Linux port is, think of RpcEmu.

It currently has only been tested on a ARMv7 system (an ASUS Chromebook Flip running Debian Stretch) and on QEMU with the included patch. Mouse, screen and keyboard are supported, sound and network are not yet supported.

It will currently run at least:

• Standard RISC OS 5.22 disc image
• StrongHelp
• NetSurf
• DDE24
• RISC OS 5.23 IOMD32 and Linux ROM builds
• Spr2Png 0.25
• InterGif 6.19

The following modules are currently part of the ROM image:

No. Position    Module Name             Version Status                                                             
  1 System ROM  UtilityModule           5.23    Running
  2 System ROM  FileSwitch              2.84    Active
  3 System ROM  ResourceFS              0.26    Active
  4 System ROM  TerritoryManager        0.56    Active
  5 System ROM  Messages                1.12    Active
  6 System ROM  MessageTrans            0.49    Active
  7 System ROM  UK                      0.61    Active
  8 System ROM  WindowManager           5.55    Active
  9 System ROM  TaskManager             1.49    Active
 10 System ROM  Desktop                 2.76    Active
 11 System ROM  SharedCLibrary          5.92    Active
 12 System ROM  FPEmulator              4.35    Active
 13 System ROM  ScreenModes             0.59    Active
 14 System ROM  BASIC                   1.63    Active
 15 System ROM  BASIC64                 1.63    Active
 16 System ROM  BlendTable              0.02    Active
 17 System ROM  BufferManager           0.39    Active
 18 System ROM  ColourTrans             1.95    Active
 19 System ROM  DisplayManager          0.44    Active
 20 System ROM  DragASprite             0.20    Active
 21 System ROM  DragAnObject            0.09    Active
 22 System ROM  Draw                    1.22    Active
 23 System ROM  FileCore                3.71    Active
 24 System ROM  IXSupport               0.00    Active
 25 System ROM  RamFS                   2.32    Dormant
 26 System ROM  Filer                   2.43    Active
 27 System ROM  FilerSWIs               0.05    Active
 28 System ROM  FSLock                  1.24    Active
 29 System ROM  FontManager             3.77    Active
 30 System ROM  Hourglass               2.19    Active
 31 System ROM  International           1.69    Active
 32 System ROM  InternationalKeyboard   0.98    Active
 33 System ROM  Obey                    0.40    Active
 34 System ROM  Portable                0.81    Active
 35 System ROM  Pinboard                1.02    Active
 36 System ROM  PipeFS                  0.23    Active
 37 System ROM  RAMFSFiler              0.40    Active
 38 System ROM  ResourceFiler           0.20    Active
 39 System ROM  ROMFonts                0.77    Active
 40 System ROM  RTC                     0.01    Active
 41 System ROM  ScreenBlanker           2.34    Active
 42 System ROM  ScrSaver                0.14    Active
 43 System ROM  ShellCLI                0.37    Active
 44 System ROM  SpriteExtend            1.78    Active
 45 System ROM  SpriteUtils             1.13    Active
 46 System ROM  Squash                  0.30    Active
 47 System ROM  SuperSample             0.16    Active
 48 System ROM  SystemDevices           1.33    Active
 49 System ROM  TaskWindow              0.79    Active
 50 System ROM  WindowUtils             2.53    Active
 51 System ROM  FilterManager           0.28    Active
 52 System ROM  Filer_Action            0.62    Active
 53 System ROM  DOSFS                   1.09    Active
 54 System ROM  ColourPicker            0.56    Active
 55 System ROM  DrawFile                1.58    Active
 56 System ROM  BootCommands            1.49    Active
 57 System ROM  MimeMap                 0.19    Active
 58 System ROM  !Edit                   1.73    Active
 59 System ROM  !Draw                   1.30    Active
 60 System ROM  !Paint                  2.20    Active
 61 System ROM  !Alarm                  2.90    Active
 62 System ROM  !Chars                  2.02    Active
 63 System ROM  !Help                   3.23    Active
 64 System ROM  Toolbox                 1.57    Active
 65 System ROM  Window                  1.78    Active
 66 System ROM  ToolAction              0.38    Active
 67 System ROM  Menu                    0.40    Active
 68 System ROM  Iconbar                 1.23    Active
 69 System ROM  ColourDbox              0.21    Active
 70 System ROM  ColourMenu              0.22    Active
 71 System ROM  DCS                     1.14    Active
 72 System ROM  FileInfo                0.20    Active
 73 System ROM  FontDbox                0.19    Active
 74 System ROM  FontMenu                0.25    Active
 75 System ROM  PrintDbox               0.18    Active
 76 System ROM  ProgInfo                0.19    Active
 77 System ROM  SaveAs                  0.20    Active
 78 System ROM  Scale                   0.16    Active
 79 System ROM  TextGadgets             0.43    Active
 80 System ROM  ZeroPain                0.06    Active

Overall Design:

• It runs as a standard Linux process entirely in user mode.
• The RISC OS ROM image is a Linux executable.
• The memory map is changed to allow Linux to occupy addresses above 0xC0000000, and to avoid 32-bit builds of QEMU.
• A generated C header file with inline assembly is used to call Linux system calls from C. The Linux header files are used for structure and constant definitions.
• SWI calls are intercepted using SECCOMP-BPF and redirected using a signal handler to the RISC OS kernel.
• MSR and MRS are emulated using Objasm macros which call routines in the HAL via branch table at the begininng of the ROM image.
• The simulated state of banked registers and CPSR is stored at memory pointer to by TPIDRURW.
• Dynamic areas and AMBs are implemented with mmap and files where required.
• A RISC OS filing system IXFS, provides access to the entire Linux filing system and stores load and exec address (and therefore RISC OS filetypes) in POSIX extended attributes.
• I have integrated support for HAL_DebugRX in OS_ReadC to allow simultaneous uses of keyboard an HAL debug stream, the HAL debug stream is enabled by bit 2 of OS_Byte 2.
• Linux environment variables are copied to RISC OS system variables.
• A video, keyboard and mouse driver is provided, which communicates another Linux program via a Unix domain socket, which uses SDL 2 to display.

Todo:

• Complete the key map.
• Fix the realtime clock.
• Make data aborts raise errors.
• Support writing timestamps in IXFS.
• Officially allocate names and SWI numbers etc.
• Fix failure to truncate of unbuffered files when opened for output, and associated race condition.
• Fix spurious calls of FSEntry_Args 6 (flush file to disc) when files are closed.
• Perform proper error handling in IXFS.
• Split video keyboard and mouse driver into seperate module.
• Finish video keyboard and mouse driver.
• Fix overwriting of ROM image when rebuilding RISC OS, which will cause a crash if the overwritten ROM image is currently being run.
• Provide a means to run Linux programs.
• Implement PMP dynamic areas.
• Replace the changes that I have labelled “Hack”
• Fix the race condition that causes !Edit to crash when loading growing files, it reliably crashes when trying to load it’s own WimpSlot.
• Replace the branch table at the beginning of the HAL.

I have included a makefile which can be be used to build RISC OS on Linux. By default it build the Linux port using the Linux port, downloading, patching and building QEMU on non-ARM systems. It expects to find the DDE in the directory ../DDE by default.

To build and run RISC OS:

  make ACORN_CPP=../DDE/AcornC.C++ check
  ./RISC_OS

And with graphics:

  make ACORN_CPP=../DDE/AcornC.C++ check
  make sdl
  ./sdl ./RISC_OS

Other makefile targets can download RPCEmu and RISC OS.

Great project. I hope that you’ll manage to change RISC OS to make it running on a non patched Qemu.

Very interesting project, Timothy. Do you have some performance indicators?

That’s incredible work! Well done!

Fascinating! This looks like a really cool project!

A few questions:

• How are you dealing with apps/modules that use privileged modes? Because of the paravirtualisation I’m guessing that only some functionality will work correctly (e.g. OS_EnterOS + OS_LeaveOS will work, but changing processor mode via MRS won’t)
• Any plans for a hypervisor-based system?
• Any plans to feed back any bug fixes to ROOL? e.g. this looks like a genuine bug in OS_ReadC (whereas this would seem to be a bug/limitation in the paravirtualisation)
• What’s your eventual goal with this? Is it just something fun to play around with, or is it part of some grand scheme to revolutionise the OS?

I had a look through the code as I’ve similar questions to Jeffrey.

How are you dealing with apps/modules that use privileged modes?

It looks like it’s limited to well behaved apps that don’t switch CPU mode. Full paravirtualization would only be possible with a Hypervisor as MRS/MSR would need switching to a hypercall. I’m not sure that’s the final goal as you wouldn’t need to tweak RISCOS and recompile, you’d just run it under a VMM that uses a JIT to translate suspect instructions to hypercalls and either emulate or translate them. Of course, this may just be a stepping stone to such a goal.

Any plans for a hypervisor-based system?

Type 2? Isn’t that already covered by QEMU etc? Type 1 would be an interesting challenge that would need to be closely tied in with RISCOS development, emulating the bare minimum that RISCOS uses. A full Type 1 wouldn’t be possible due to the lack of documentation on BCM’s for example, but some other platforms, such as IOMD and possibly some of the newer bespoke RISCOS platforms might be achievable.

What’s your eventual goal with this? Is it just something fun to play around with, or is it part of some grand scheme to revolutionise the OS?

Dejavu…I’m sure you asked that exact same question when this project was last mentioned :)

How are you dealing with apps/modules that use privileged modes?

You don’t have to deal with this. Qemu is – as RPCEmu – a complete emulator.

Type 2? Isn’t that already covered by QEMU etc

Correct… but unmodified. Else: problems.

As for performance, 1 million OS_LeaveOS calls takes 6.1 seconds and an IOMD32 ROM build (without Internet module) with DDE24 on a BTRFS filesystem on an encrypted SDHC card takes about 12 minutes of CPU time (about 2.5 minutes user mode, 9.5 minutes Linux kernel) on my ASUS C100P Chromebook Flip clocked at 1.6Ghz. Where that 9.5 minutes is going is a good question, one guess is SWI dispatch.

Performance under QEMU is poor, I suspect this may be because RISC OS (programs) often alter pages containing code, which causes QEMU to flush translated instructions. Typical Linux programs however keep code in non-writeable pages, and data in non-executable pages.

The QEMU patch is needed to intercept SWI calls which would otherwise be interpreted as Linux system calls, and to disable the incomplete FPA emulation so that RISC OS FPEmulator can work. I, along with other developers, have made several fixes to signal handling in QEMU that were needed to run RISC OS. Perhaps this change could also be included in QEMU.

RISC OS can not run on QEMU user mode without the patch (unless one were to eliminate all the SWI calls), this port should run on Linux on QEMU system emulation, running RISC OS directly on QEMU system emulation is another subject entirely!

How are you dealing with apps/modules that use privileged modes?

• R14_svc is saved on IRQ entry and restored on exit so that code that switches to SVC mode to save R14_svc before calling SWI calls just work in a single virtual mode.
• By modifying source of the modules and recompiling.
• By a load time binary patch of the “GCC” Unixlib.
• By not using the `egrep` executable in the RISC OS build.
• Perhaps ADFFS could be useful.

OS_EnterOS, OS_LeaveOC, OS_IntOn, and OS_IntOff work. I don’t know how many RISC OS programs use privileged operations.

Any plans to feed back any bug fixes to ROOL?

Yes, starting with the security hole in Fileswitch. I would like feed at it all back.

(whereas this would seem to be a bug/limitation in the paravirtualisation)

No, R12 is clearly overwritten by reloading the saved registers from CallBackRegs, and then not so obviously used by loading r12 from CallBackRegs as CallBackRegs is r12 based. This code is not used under para-virtualisation due to conditional assembly.

Yes, starting with the security hole in Fileswitch.

Ummm… A spy may then read said memory contents by examining the disc,

You know, if we were worried about that sort of thing, we wouldn’t be using RISC OS! After all, why examine the disc in the hope of discovering memory contents when one can issue a single SWI to get “God Mode”¹? ;-)

Far more logical to say that the buffer contents should be cleared so what’s there is what the program intended, and not whatever random junk was in memory. It’s easier to debug when things are regular and known, don’tcha think?

¹ Way back when I wrote a program to extract Econet passwords by examining the memory used by the password entry Templates. It was harder to get it to look at the right time than it was to extract the information from memory!

(whereas this would seem to be a bug/limitation in the paravirtualisation)

No, R12 is clearly overwritten by reloading the saved registers from CallBackRegs, and then not so obviously used by loading r12 from CallBackRegs as CallBackRegs is r12 based.

You’re right – I was mistakenly thinking that R12 was a banked register and it was the (lack of) user-mode LDM/STM that was causing things to go wrong.

I think I was mainly in disbelief that the bug could have gone unnoticed until now – unless the situation for triggering that code path on a normal RISC OS machine is so rare that people would hardly ever see it (and most of the time it would probably load a bogus PSR, making any resulting crash hard to diagnose)

Well spotted!

The Taskwindow bug is triggered by a race condition involving callback being triggered immediately before a task switch, and the use of callbacks in Taskwindow compatible programs is rare, except for handling escape keypresses which would occur after a task switch.

I think I was mainly in disbelief that the bug could have gone unnoticed until now

As was I surprised to find the bogus call to OS_SynchroniseCodeAreas when creating a code variable.

After all, why examine the disc in the hope of discovering memory contents when one can issue a single SWI to get “God Mode”1? ;-)

Because one only has the disc they picked up at the club meeting :-)

Because one only has the disc they picked up at the club meeting :-)

I would really hope that such disc would be erased¹ and then formatted.

I think I was mainly in disbelief that the bug could have gone unnoticed until now

Isn’t R12 considered the private workspace pointer in most of those sorts of calls? Maybe at that level R12 is considered a “trashable” register

This caught my eye, it sounds like an awesome achievement.
Do I compile this on RISCOS or do I compile this on Linux (Cortex A15 laptop).
On Linux it tries to find AcornC.C++ which of course is not available. Any hints on how to get this built?

That’s right. It needs AcornC.C++ on Linux to build. Builds and runs Ok on ARM Linux and Intel Linux.

cd RISC_OS_Dev
git checkout Linux
make

./Test_Boot

Test_Boot automatically downloads and builds/intalls the required dependencies. Then runs RISC OS in an SDL2 window.
I used DDE25 to build it. The file names needed RISC OS file types on the end.

Starting phase install_rom …

HAL_Linux (mixed.Linux.HAL)…
objasm -depend !Depend -g -ihdr -i<Hdr$Dir>.Global -i<Hdr$Dir>.Interface -i<Hdr$Dir>.Interface2 -pd “APCS SETS \”APCS-32\"" -pd “Machine SETS \”Linux\"" -pd “UserIF SETS \”Sovereign\"" -APCS 3/nofp/noswst -pd “ro_size * 0×33004000 – 0×33000000” -o o.Tests s.Tests
ARM AOF Macro Assembler 4.03 (Acorn Computers Ltd) [15 Oct 2015]
Error: File “Hdr:LinuxSyscalls” could not be opened
Error: Bad GET or INCLUDE at line 33 in file s.Tests
33 00000000 GET Hdr:LinuxSyscalls
Assembly terminated:
2 Errors, 0 Warnings
AMU: * exit (1) *

this is where the build ends after some time of compiling. (which was amazing to see on the linux console :-) )

Renaming diff,ff8 to diff in mixed/RiscOS/Library/GNU fixes this.

LinuxSyscalls (mixed.Linux.Syscalls)…
%link -Output gen_header o.gen_header C:o.stubs_bootstrap
ARM Linker: (Fatal) File C:o.stubs_bootstrap not found.
AMU: * exit (1) *

apparently I needed to get the headers from my kernel in the right place.
Somehow it can’t link with
./Export/APCS-32/Lib/CLib/o/stubs_bootstrap
So perhaps the path there is not setup correctly.. I could do a manual fix for now to get it properly compiled.
Also the hdr file seems not to be installed automatically. Simple fix was copying it by hand to Export/APCS-32/Hdr/Global/
SDL gave some headaches and in all this wasn’t a painless compile but I am impressed with the result.

Would have inserted a screenshot here if that was possible.

Very well done!! This has been on my mind for a very long time.
Congratz!

RISCOSmark 1.01 (14 May 2003)
Comparison with RiscPC SA 202MHz running RISC OS 4.02 800x600,256
(HD benchmarks are in kilobytes/sec)

OS/Machine/Processor: ??
Graphics Resoloution: 800x600, 16M colours

Test                                                   Benchmark
Processor - Looped instructions (cache)                  4907609    2759%
Memory - Multiple register transfer                        41040   25333%
Rectangle Copy - Graphics acceleration test                 3875    1601%
Icon Plotting - 16 colour sprite with mask                 32491    1624%
Draw Path - Stroke narrow line                             12263     786%
Draw Fill - Plot filled shape                              12765     874%
HD Read - Block load 1MB file                            1119302   37535%
HD Write - Block save 1MB file                             28553     938%
FS Read - Byte stream file in                                126      60%
FS Write - Byte stream file out                               59      30%

are these values correct?
27.6 times faster than a RiscPC? Impressive!
Will test some other things later but need sleep now.

are these values correct?
27.6 times faster than a RiscPC? Impressive!

Sounds plausible. The fastest overall current ARM based system (RapidO Ig) reports processor as 2090%, Draw fill 1157% & FS write 1329% Figures from Chris Hall

What System/CPU were you using?

Any idea what’s going on here?

setfattr -n user.RISC_OS.LoadExec -v 0x00fdffff00000000 mixed/Linux/Support/RISC_OS.xz || true
xz --force --decompress --keep mixed/Linux/Support/RISC_OS.xz
setfattr -n user.RISC_OS.LoadExec -v 0x00e5ffff00000000 mixed/Linux/Support/RISC_OS || true
gcc -std=gnu99 -g `pwd`/mixed/Linux/Support/comma2attr.c -o comma2attr
setfattr -n user.RISC_OS.LoadExec -v 0x00e6ffff00000000 comma2attr || true
/home/user/RISC_OS_Dev/mixed/Linux/Support/comma2attr.c:37:24: fatal error: attr/xattr.h: No such file or directory
 #include <attr/xattr.h>

Edit: Solved by installing the package ‘libattr1-dev’.

The fastest overall current ARM based system (RapidO Ig) […] Figures from Chris Hall

Since it’s mathematically dubious to average percentages for things measured in different units, the table would suggest:

Of 18 categories, there’s a more prevalent metallic element in the overall top spot.

The laptop (Acer Chromebook) I use is based on Nvidia Tegra K1 (4x Cortex A15 @ 2.0 Ghz).
Looking at these results I’d have to say it’s 35% faster than anything else out there.
It’s been my dream RISC OS machine since this also supports OpenGL (yes, full OpenGL) besides GL-ES and has 4GB memory.

Which makes me want to ask the question: how do I tell the Linux port of RISC OS to use more than 16 MB memory?

Interesting.. the max wimpslot is 16MB. However starting an app with 16MB yields 16MB free.
Wish I had this on my old RiscPC ;-) Infinite Memory available! Yay! (Like in Doom with infinite ammo)

Of 18 categories, there’s a more prevalent metallic element in the overall top spot.

The 5432EVM & RapidO Ig use the same OMAP5432 SoC! Why there is any difference for many of the benchmarks is a mystery to me.

The CPU in the 5432EVM, RapidO Ig & Titanium are all the same! I’m quite willing to admit that there is not much to choose between all three. I am biased as to which I think is best:-)