Benchmarks

There’s a 32 bit version of DataVox on my site: http://jeffd.drobe.co.uk/

Jeff, you did an awesome job with your Doom port, fancy taking on a Quake port too?

Hmm, hoped that no one would spot that!
Doom represents 13 years work and I’m getting a bit long in the tooth to take on that size of project again!
Quake uses a LOT of floating point stuff AFAIK. I think that the authors of the original Quake ports spent a lot of time trying to speed these up. All clever stuff.

I’d be happy to provide you with our sources which would then just need making ARMv7 safe…

Using version 1.1 of Richard Spencer’s benchmarking programme I have added some further benchmarks to the original table.

(original table reproduced above)

In the revised table below, the benchmarks are expressed in percentages, where 100% is equivalent to a Strong Arm 202MHz Risc PC running RISC OS 4.02

Note: I tested the newer computers at the following resolutions:
ARMini 1280×1024 C256
BeagleBoard XM 1280×1024, 256 colours
A9home: 640×480, 256 colours
Iyonix: 800×600, 256 colours
VRPC: 1920×1080, 32K colours

Note: Some of the benchmarks proved sensitive to screen resolution and so I repeated these at various resolutions:

Those images are a bit big – care to resize them?

Resized.

In the revised table below, the benchmarks are expressed in percentages, where 100% is equivalent to a Strong Arm 202MHz Risc PC running RISC OS 4.02

Note: I tested the newer computers at the following resolutions:
Raspberry Pi 1920×1080 16M
ARMini 1280×1024 C256
BeagleBoard XM 1280×1024, 256 colours
A9home: 640×480, 256 colours
Iyonix: 800×600, 256 colours
VRPC: 1920×1080, 32K colours

For the HD & FS tests was the same Drive you were using for the Raspberry Pi, ARMini’s & Beagleboard?
What type of drive USB (IDE), USB (SATA) or SSD or ?

For the beagleboard it was an ‘intenso’ pen drive, for the ARMini it was a 96Gb SSDdrive and for the Pi it was a pen drive (identical to the beagleboard pen drive). However the beagleboard test was using the original block drivers which have since been improved I understand (in the ROM?).

Isn’t an ARMini a Beagle xM at 1GHz?

If I understand it correctly, there are some rather odd timings – the Pi’s 700MHz CPU appears to run considerably slower than a Beagle xM at a restrained 600MHz (and, it would appear, to only be a shade faster than an Iyonix (although its memory access is considerably faster to make it appear overall faster). Its rectangle copy is no faster than a RiscPC, yet its icon plot beats everything else by a huge margin. HD reading is acceptable, but something looks wrong with writing. Is the figure really that pitiful?

Isn’t an ARMini a Beagle xM at 1GHz?

Yes, but the 600MHz beagleboard XM was tested on 17th Sept 2010, the 800MHz ARMini on 12th July 2011, the 1000MHz ARMini on 28th April 2012 and the Pi on 10th June 2012 and other things have changed over that time. Such as improved drivers.

I can only report what happened. I haven’t yet tried to understand it!

Dear Chris,

could you run my fractal benchmark on the RP ? I think at least the single precision VFP version should run on the Raspberry PI. I made a special version, that runs at 1920×1080. Your can find it here

Is 1920×10280 the only resolution you can run at the moment or also something like 800×600 8bit/24bit etc ?

the Pi’s 700MHz CPU appears to run considerably slower than a Beagle xM at a restrained 600MHz

Nothing particularly unusual there – the Pi is using an older CPU architecture than the Beagle. Although since the RISCOSMark CPU benchmark just involves executing a load of NOPs it’s probably not worth putting too much faith in it when it comes to comparing different CPU architectures.

and, it would appear, to only be a shade faster than an Iyonix

That’s slightly more curious – some simple tests I did with running a loop in BASIC and looking at ArcEm’s MHz counter suggested the Pi was somewhere around 15% faster than an Iyonix.

Of course you also need to take into account what I said above about the CPU benchmark :)

although its memory access is considerably faster to make it appear overall faster

The memory performance test only copies a 128KB block. Since the BB has a 256KB L2 cache, and the Pi has a 128KB L2 cache, and the other machines have no L2 cache, I think it’s pretty clear why you’re seeing the results you are.

Its rectangle copy is no faster than a RiscPC,

There’s no hardware acceleration for copies/fills yet.

yet its icon plot beats everything else by a huge margin

The Pi’s L2 cache is at the system level, so it’ll get much better performance for “uncacheable” memory than other platforms (except for versions of ROL’s OS which do have cacheable screen memory).

HD reading is acceptable, but something looks wrong with writing. Is the figure really that pitiful?

Welcome to the world of USB drivers that were hacked together in a week! There are still a few bits that I need to fix up (e.g. I’m fairly certain the driver is constantly hammering the RMA), so it’s quite possible the poor write performance is down to that, or some other issue that can be fixed.

could you run my fractal benchmark on the RP ? I think at least the single precision VFP version should run on the Raspberry PI. I made a special version, that runs at 1920×1080. Your can find it here

If you’re being a good boy and using VFPSupport then I can tell you that it won’t work yet. There’s some changes I need to test & check in in order for the module to detect and handle VFPv2. Plus at some point we’ll need to write/integrate some support code, as the coprocessor needs software support for some operations (mainly stuff involving denormalised numbers). Although the need for support code can be avoided by using the run fast mode.

Is 1920×10280 the only resolution you can run at the moment or also something like 800×600 8bit/24bit etc ?

At the moment 1920×1080×24bpp is the only mode available.

I can only report what happened. I haven’t yet tried to understand it!

Ah, you must be the BBC’s science reporter… ;-)

OK here’s some more comprehensive information – all tests repeated today:

In the table below, the benchmarks are expressed in percentages where 100%=SA202 RiscPC running RISC OS 4.02

Five of the benchmarks were very dependent on the screen mode used and so I have repeated them at different screen resolutions:

* – 1920 × 1080 C16M
+ – 1600 × 900 C16M

Nice work, Chris. Should be useful.

Why is the BB memory benchmark affected so badly by the screen mode? I wouldn’t have expected the screen refresh to be stealing that much bandwidth.

Why is the BB memory benchmark affected so badly by the screen mode?

Because it has no separate video RAM and a fixed, total memory bandwidth?

Or the graphics is not yet accelerated and some work that could be done by the GPU, is now done by the CPU and that uses cpu-memory bandwidth in staid of gpu-memory bandwidth?

@Chris and Jeffrey

…okay, I see VFP support needs some more time. Meanwhile it would be interesting to me if you could test my now tuned !FireBench (to be found here). I changed the whole thing to use a 24bit mode. You can edit the !RunImage to use any screenmode from 800×600 to 1920×1200. For the R-PI I inserted already 1920×1080. So it should run out of the box…can you give it a try ?

For the record: Surprisingly also this benchmark is about 10 to 20 % affected by the screen mode (higher is slower). It should not be, as the fire itself is always the same size. May be big screen modes are harder to address from the write cache (?). Also this benchmark is much slower on the Panda…I get about 377 Frames/s (1280×1024, Beagle) and 226 Frames/s (1280×1024, Panda).

Interesting – is your PandaBoard an ES?

What speed are you running it at?

I got 432 frames/sec on my ES running at 1.5GHz and a 1920×1200 screen resolution…

…hm, indeed interesting, I’ve got an old Pandaboard, running at 1 GHz. But your result is quite high for just 0,5 GHz more. Could you give it a run at 1 GHz ? …if there’s still a difference, it could only be the ROM…but I doubt a different ROM would have an effect…

I’ll need to dig out an old SD card to downgrade to 1GHz – i’ll let you know how I get on.

It sounds like you have the original PandaBoard – one of the things that the OMAP4460 brings to the party is much better video which might explain what we are seeing.

Right – re-run the test with the PandaBoard ES running at 920MHz and I now get 347 frames/sec.

The ROM is the one that comes with the PandaLand scheme (16/05/2012).

Screen resolution is the same 1920×1200 @ 46Hz.

Hope this helps.