Things we'd like to see happen before launch of ARMini (bounties?)

Hi everyone!

Before we “go live” on the ARMini, there are a few areas of coding
that we’d like to see happen. The ROOL guys seem to be busy with
their day jobs, so we’d like to open these things up to everyone. I
guess it is a bit like the “bounty” thing that has been proposed.
We’re also in the same boat, time-wise. Trying to get everything done
on a short timescale, and get other projects completed!

Firstly, I need to stress that all OS changes will be fed back into
the OS. We aren’t trying to hijack the open source OS and turn it
into something proprietory. Conversely, this means we can’t afford to
throw mega-bucks at anything – we’re basically budgeting money from the
“pre-order” pot for dev work. With that in mind, if anyone wants to
see any of this work carried out, but hasn’t pre-ordered an ARMini,
we’d ask you to get in touch if you’d like to help out with bounties.

Some work, which I hope won’t require much time, I have suggested as
“payment in kind”. By this, I mean that whilst we’d prefer not to
have to budget cold, hard cash, we don’t mind giving out free R-Comp
software etc.

I suggest that some of this is carried out “off list” on the usual
rcomp.co.uk email addresses.

So, to business…

Display Manager, and Display configure plugin (payment in kind?)

We’re trying to avoid the need for so many monitor-specific MDFs, and
move to a small number of generic MDFs. As such, it becomes more
important to allow users to “try before they buy” when changing mode.
Also, some screens (esp HDTVs) are fussy about exact mode timings or
refresh rates. For example, Panasonic TVs will only sync to 1080p
resolutions at 23-25hz or 50 or 60. As such, being able to configure
frequency as well as resolution becomes important.

So, 3 main changes:

Some work has been done already on this to provide a “emergency restore”
keypress to revert to previous mode, however, it’d be nice to have a proper
OS-level implementation.

USB HID support in OMAP HAL (time requirement unknown)

At the moment, there’s no HID support in the OMAP HAL, so
shift-booting to bypass !Boot (and other keypresses) don’t work. It
also means that the message “No keyboard present – Autobooting”
appears. Firstly, we’d suggest this read just “Autobooting” as an
interim fix, because there are (as we’ve seen) a variety of reasons
for this appearing, so it is potentially misleading. Secondly, some
form of basic HID support is really needed in the OMAP HAL.

mUSBdriver improvements

The ARMini makes use of the “internal” USB port on the xm, and whilst
many things are fine, some pendrives (eg. Corsair) and things cause
lockups when connected. I believe performance improvements might also
be possible. Any improvements to this would be welcomed – I believe
Jeffrey may have already started on this.

Slow filecore USB drives

We have found that drives formatted to FAT32 read/write data at much
higher speeds than those formatted to filecore specs. For example,
the xm PDF manual takes about 20 seconds to load from filecore, but
only a couple from FAT32 (and also from RAMFS, interestingly). My
suspicion is that certain filecore operations are slow over USB, but
investigation would be appreciated. We’ve run disc tests on a variety
of discs (SSD, hard drive, pen drive) and get some very odd results at
times!

Fat32FS in ROM

Jeffrey Doggett has given permission for Fat32FS latest version to be
included in the ROM, and has provided source/changes compared to original
code. He doesn’t have time to do this himself. This would allow booting
from a FAT32 device, which (especially in light of previous point) seems
quite desirable.

Additionally, a suitable !HForm type tool would be nice. Jeffrey
has written a command line formatter, which probably just needs
a friendly wrapper.

Indeed, we’d like to see a more “up to date” replacement for
HForm-type apps which allows formatting of modern devices (taking into
account max filecore sizes and so on) and allows selection of
filesystem/format. This isn’t important for shipping, but would be a
nice improvement.

Jeffrey Doggett has given permission for Fat32FS latest version to be
included, and has provided source/changes compared to original code.
He doesn’t have time to do this himself, but has written a command
line formatter.

CDFSSoftSCSIUSB

I believe Steffen has made progress on USB CD stuff, but as yet
there’s no equiv to CDFSSoftATAPI/CDFSSoftSCSI for USB CD drives.
This one is quite important to us, as users will want to be able to
get their software onto their computer, and whilst we’d use network or
SD card, not all users are comfy with that.

Steffen has been contacted about this, but apparently would appreciate
a hand in producing the CDFSsoft module since he’s current got code
patched direct into CDBurn.

EtherUSB

Current versions seem to be working nicely, but if no ethernet cable
is present, startup fails with an Abort. We’d like to find out why,
and see if a workaround/fix is possible. Again, this is quite
important to us. I have been in touch with Thomas Millius and James
Peacock, but we could use some help with “detective work” as to where
the problem lies.

Compile-time branch for ARMini

If it is possible, we’d like to be able to create a branch of the OS
that adds in an ARMini specific startup banner (etc) but still builds
automatically from the base standard OS files.

Thanks in advance,

Andrew

Andrew, probably you need a ‘release Branch’ anyway as other developers should not be kept from submitting changes. Others could benefit from this, too, being able to find sources that will give a working ROM Image when built. The stable version in that release repository branch can then be re-merged into the (existing) development.

At least up to early 2011, you could never be sure that the repository will result in a bootable ROM image. Generally it would not even build completely (At least I failed 4 times from 4 when I tried over the last months). Generally the packed sources were a good replacement for that, as they mostly worked after some smaller adjustments.

Please do not understand me wrong – I am not complaining at all:

1. this is what is to be expected when several people work at a project like this
2. I always built on the development target itself
3. I may have had bad luck

P.S. I have forgotten a lot about branching and merging under CVS as I use subversion on a daily basis since some years now – but I can remember that there were some pitfalls when merging branches with CVS.

Any improvements to this would be welcomed – I believe Jeffrey may have already started on this.

Correct. Hopefully I’ll have a releasable version ready sometime this weekend.

My suspicion is that certain filecore operations are slow over USB, but investigation would be appreciated.

The problem is that SCSIFS doesn’t support disc caching/background transfers. Currently the module’s written in assembler and is a bit convoluted in places; I think the best way forward would be to rewrite it in C, as that should be a lot easier/less risky than modifying the existing assembler.

ARMini specific startup banner

This will probably depend upon getting an answer from Castle as to whether ROOL/people are allowed to re-brand the ROM images.

Incidentally, I think a few of the things you’ve mentioned here will be among the initial bounty list once ROOL’s bounty system is properly unveiled (which will be sometime soon, I think!)

I really don’t see much need for branching here – it’s more a case of build switches for most of this stuff, or having the correct environment set up – we can already build for the RiscPC, Iyonix and Beagle Board without branching the repository.

As for bounties, please don’t confuse this suggestion with the ROOL Bounty scheme. If there are some tasks that you’d like to see happen for the ARMini then that’s all very welcome development for RISC OS. The ROOL Bounty scheme hasn’t been formally announced or launched yet so feel free to speculate about how it works! :) We actually have to be very careful about just what it is and how we run it because the more you think about it, the more complex it can become.

Expect to see our Bounty scheme go live around the time of the Wakefield Show – now less than two weeks away…

Update: a desktop FAT32 formatter has now been produced.

The display manager stuff is quite urgent for us, and shouldn’t be too complex if people have time. Please let me know if you’re up for it :)

USB HID support in OMAP HAL (time requirement unknown)

Would someone with deeper USB knowledge than I (anyone!) care to comment on how feasible this is? It seems to me like there’s quite a lot of work in getting a USB system up and running to the point where you can talk to keyboards and mice. But maybe I’m wrong…

Instead of moving USB stuff into the OMAP HAL, I would suggest to delay the output of the “No keyboard detected” output until the USB subsystem is completely initialised. The same for the detection of the various boot-relevant keys. After all, to start booting, you need an initialised USB subsystem.

Of course I speak as a complete ignorant wrt the inner workings of RISC OS…

On the other hand, I seem to remember that the IYONIX had simple HID drivers which got initialised long before the “full” USB stack. You had to put the keyboard into a specific USB port however.

Do you not need to be careful to allow users to reset the machine to a useable state very early in the system boot sequence, though? I suppose with a cut-down version of an HID driver, it might still be workable. But that’s the question, really. How cut down could a USB HID implementation be?

Keyboards are actually some of the simplest USB devices to talk to. The Iyonix uses a cut-down OHCI driver designed to talk directly to keyboards; there’s no support for hubs. But for the Beagle you’d want/need (cut-down) EHCI & MUSB support, plus hub support, which is significantly more code. It’s not impossible to do, just a bit time-consuming and annoying.

It would be nice if we could move to a system where RISC OS initialises itself enough to allow the standard keyboard/USB drivers to be used, but someone would need to do some research into how CMOS RAM is used during the OS initialisation before we can work out how feasible such a change would be.

On re-branding: set aside for a moment the issue of getting authorisation from Castle, here’s some free consultancy about how you’d achieve it technically.

Branches in CVS are a relatively expensive operation because of the way branching is recorded separately for each source file. This is different from the way other common version control systems (VCSs) like Subversion and Git operate, and is often cited as a weakness of CVS; however, this approach bestows CVS with certain abilities lacking from these other VCSs – abilities which are utilised to good effect by the Products file scheme by which RISC OS 5 builds are managed. This is one of the main reasons why we haven’t tried to migrate to an alternate VCS.

Yes, there are some uses of branches in our CVS, but they are used sparingly and only ever on a component-by-component basis, where there is ongoing parallel code development which to an extent must be protected from the other development branch. They are never used simply to alter resource files (sprites etc).

The sort of things you’ll be wanting for ARMini – release management and rebranding – are just the sort of thing that Acorn and Pace did all the time, since RISC OS products were simultaneously being developed and released by multiple teams for multiple customers, with greater or lesser amounts of overlap. Although we haven’t had much need for these things so far since the sources were published, the capabilities are still there.

One of the reasons why you might have envisaged a branch for ARMini would be to freeze certain modules to isolate them from ongoing mainline changes while you prepare and test a formal release. The way to achieve this is to create and maintain a Products file which lists the revision tags for each component in the build. The “checkout” script parses the Products file and uses CVS to check out different subsections of the repository using different tags – something you can only do with CVS.

Developers have long been encouraged to use the “srccommit” script whenever committing a change to a component. This automatically tags the component being committed each time it is run, so (nearly) every component already has a long history of revision tags in a standardised format. At Pace, nearly every Products file used revision tags – but then they had an army of engineers to keep them up-to-date. ROOL has mostly used Products files with branch tags, which are easier to maintain, but represent a moving target, so are no use for freezing a build. The only published example of a revision tag Products file you have to work from is the “Tungsten” products file, which records the exact versions of each component which went into the official RISC OS 5.14 and 5.16 builds for the Iyonix.

On rebranding, there are two different approaches that Pace used.

The first, older method uses the UserIF system variable. This is set by the Env file (which is what’s run when you select something from the “Environment” menu in !Builder). It’s possible to use exactly the same source tree – even the same Products file and Components file – to do builds with differing UserIF settings. UserIF is used in a small number of places in the source code, but is mostly used to decide which subdirectory of the Resources subdirectory of each component is used when selecting which files get installed into ResourceFS during the “resources” build phase. It just so happens that all ROOL builds so far have used UserIF=Iyonix, and the Iyonix theme, appropriately enough, has the Iyonix lozenge in all the appropriate places.

The main downside to this scheme is the proliferation of subdirectories of each component’s Resources directory that happened when theming of RISC OS was in its heyday. In particular, the Wimp sources got rather large, with dozens of variations of the Wimp sprite pool for different customers. After a while, Pace started getting around this by using dedicated components, conventionally named RiscOS/Sources/Apps/<Theme>UI to localise all the resource files for a particular user interface – though since none of these have been released, I can’t point at them as an example. However, they were really simple components, just a collection of resource files and a Makefile which implemented only the “resources” phony target, which simply copied the resource files to the Messages module. Usually, this component would be at or near the end of the Components file, meaning that it overwrites any resource files already installed by a different component. Obviously, this means that this scheme needs a separate Components file for each theme, as well as a separate Env file to select that Components file, and probably a separate Products file to ensure that your resources component gets checked out. But you probably wanted your own Products file anyway, see above.

Regarding trying to use the main USB stack for bootup keyboard scanning, there’s a bit of a chicken-and-egg problem: one of the things the keyboard scan decides is whether a CMOS reset occurs, and until you know whether a CMOS reset has been requested, you don’t know which modules have been unplugged, so you can’t initialise the module chain, including the USB driver.

>> USB HID support in OMAP HAL (time requirement unknown)

>>At the moment, there’s no HID support in the OMAP HAL, so shift-booting to bypass !Boot (and other keypresses) don’t work. It also means that the message “No keyboard present – Autobooting” appears. Firstly, we’d suggest this read just “Autobooting” as an interim fix, because there are (as we’ve seen) a variety of reasons for this appearing, so it is potentially misleading. Secondly, some form of basic HID support is really needed in the OMAP HAL.

> Keyboards are actually some of the simplest USB devices to talk to. The Iyonix uses a cut-down OHCI driver designed to talk directly to keyboards; there’s no support for hubs. But for the Beagle you’d want/need (cut-down) EHCI & MUSB support, plus hub support, which is significantly more code. It’s not impossible to do, just a bit time-consuming and annoying.

My Iyonix has always showed “No keyboard present – Autobooting”, will the Armini have direct USB ports, so the simpler driver could be used?

Could the messages simply be modified so only successful keyboard initialization is shown?

> Indeed, we’d like to see a more “up to date” replacement for HForm-type apps which allows formatting of modern devices (taking into account max filecore sizes and so on) and allows selection of filesystem/format. This isn’t important for shipping, but would be a nice improvement.

Could it create a DOS style partition table entry for the ADFS formatted part, allowing the rest of the drive to be used with NTFS or EXTFS (or FAT if FAT32FS were updated accordingly)?

https://www.riscosopen.org/forum/forums/2/topics/560

An update on some of these items:

2) Display Configure Plugin – allow frequency to be specified

Done. It fits in quite neatly next to the ‘try’ button, so there wasn’t any need to remove it.

mUSBdriver improvements

Still in progress, although I did check in some of my improvements before the show.

USB HID support in OMAP HAL

I’ve come up with a compromise solution for this. Rather than using a seperate set of custom-written USB driver sources like the Iyonix HAL, and rather than poking around in RISC OS’s initialisation sequence to allow the USB/keyboard drivers to initialise independently of the CMOS RAM, I’m going with the approach of modifying the USB modules so that they can also be built as static libraries suitable for linking with the HAL. To get this to work I do need to write a fair amount of support code, but that support code should be 100% reusable, so adding USB keyboard scanning support to future HALs should be trivial. I might even try replacing the current code in the Iyonix HAL, to see if it makes the keyboard scanning a bit more reliable for anyone.

Andrew, have you any experience with the xM rev. C? It appears that there may be some issues.