Proposed GraphicsV enhancements

Goals

• Provide a well-defined method for handling multiple active displays
  • The current API supports multiple displays, but does not specify how RISC OS interacts with them or how programs should enumerate the set of available displays
• Provide an API suitable for controlling all the features of the OMAP3 display subsystem
  • So that the new OMAP3 video driver can expose all of the hardware’s functionality to RISC OS and user programs
• Provide support for displays being added and removed at runtime
  • E.g. for DisplayLink-based USB monitors/video adaptors
• Allow programs to claim sole use of displays, hardware overlays, etc.
  • You don’t want to have two media players trying to use the same YUV overlay at the same time!
• Add API-level support for the new display formats proposed in the extended framebuffer specification
  • Note that this does not necessarily mean that the rest of RISC OS will be expected to understand the formats!
• Add anything sensible that will make Geminus’s life easier?
• Attempt to re-use any of ROL’s new APIs wherever possible

Existing documentation

• Extended Framebuffer Format Specification
• GraphicsV
• RISC OS Select display changes
• RISC OS Select OS_ScreenMode changes
• RISC OS Select VDU Variables

Detail

Support for multiple displays

The current GraphicsV API specifies that the top byte of R4 is used to specify the display number, with 0 being the default. However, no method is defined (or implemented) for enumerating the list of available displays, notifying the system of display addition/removal, etc.

To rectify this, the following changes are proposed:

• Adoption of ROL’s Service_DisplayStatus service call to notify programs of display addition/removal
• Adoption of ROL’s OS_ScreenMode 11 to allow selection of the default display
• Adoption of ROL’s OS_ScreenMode 253 to obtain the current maximum display number
• Adoption of ROL’s OS_ScreenMode 254, 255 to allow drivers to register/deregister themselves as displays
• Adoption of ROL’s use of VDU variable 192 to store the current display number
• Change display 0 from being the ‘default’ to being just another display. Instead, whenever an operation is to be performed on the default display, the kernel will specify the display number explicitly. This will help reduce unnecessary driver overhead (if display 0 were the default, each driver would have to check if ((display == me) || ((display == 0) && (default == me))))
• Add a new OS_ScreenMode call to allow programs to claim sole use over displays (and another to relinquish that claim)
  • Exact API TBD!
• Add a new OS_ScreenMode call to read details/capabilities of a display
  • Adoption ROL’s OS_ScreenMode 12 and associated descriptor format may be appropriate for this. The descriptor is a tag/value-based format, so it’s easy to add additional tag IDs to describe new functionality (hardware overlays, etc.)
  • However, it may also be inappropriate, because it does not map well to reading the capabilities of hardware overlays, or if a device has multiple heads
  • Whatever the new call is, it would also have to be passed onto the driver via a new GraphicsV reason code; the OS_ScreenMode call would just be a frontend for applications to use

Support for hardware overlays

The primary aim of this change is to provide support for the features of the OMAP3 display subsystem. Thus, the new GraphicsV must support multiple hardware overlays, each with differing restrictions on available buffer formats, and the ability for programs to alter the display order of overlays (where the hardware supports it)

To add support for hardware overlays, the following changes are proposed:

• Bits 16-23 of R4 are used to specify the overlay number when calling GraphicsV
• A mechanism is provided to allow drivers to indicate how many overlays they support and what capabilities each overlay has
  • Exact API TBD!
  • Overlay capabilties include:
    • Available pixel formats
    • Size limitations
    • Positioning limitations – on OMAP3 all overlays can be positioned anywhere on screen, but on VIDC the main ‘overlay’ is fixed in place and only the pointer overlay can move
    • Whether the overlay can be used in ‘pointer mode’? (May instead be specified in the overlay configuration, see below)
    • Hardware scaling capabilities
    • Hardware rotation capabilities
    • Different transparency methods (alpha or chroma-key – may be specified by overlay configuration instead?)
    • Whether the DAG supports framebuffer formats where the line length (in bytes) is not related to the stride between lines
• A way of describing the different overlay configurations available
  • For each configuration:
    • The order the overlays are layered ontop of each other
    • The method of handling transparency
    • A flag for each overlay within the configuration, indicating whether that overlay is suitable/recommended for pointer display
      • This will simplify the task that user programs must perform to select sensible overlay configurations
    • A set of flags indicating which heads each overlay will be visible on
      • Rationale: Individual overlays in the OMAP3 can be enabled for LCD output, S-Video/composite output, or both at the same time. But under the assumption that other hardware can be more limiting (e.g. only allowing certain overlays on certain heads), flags shall be used to explicitly state the exact limitations.
  • Since a device is only likely to have a small number of overlays, a simple list of all the different configurations is likely to be all that’s required
• An OS_ScreenMode call to select the overlay configuration number, the overlay for RISC OS to use for display output, and the overlay for RISC OS to use for pointer display
  • The latter two parameters are only important if the display is to be made the default display via OS_ScreenMode 11
  • Configuration info would also be passed through to GraphicsV
• New VDU variables are added to store the current overlay configuration number, and which overlay RISC OS uses for the main display and the mouse pointer
• An OS_ScreenMode call (and associated GraphicsV call) to select/get chroma-key colour value for each overlay
• Add a new OS_ScreenMode call to allow programs to claim sole use over overlays (and another to relinquish that claim)
• New OS_ScreenMode + GraphicsV call to alter/query overlay position, rotation, scale factor
• New OS_ScreenMode + GraphicsV call to alter/query overlay size, colour format, buffer address
  • It will also be possible to specify the scanline stride, e.g. if you only want the overlay to display a small portion of a larger framebuffer
  • The parameters for hardware scrolling – DAG start, end, restart addresses – will also be specified here
  • The colour format will be one of the ones as defined in the extended framebuffer specification (The NColour value and flag bits 12-15?)
• Some form of GraphicsV interface to allow advanced configuration of DAG? E.g. display spanning with one framebuffer can be implemented on OMAP by having one overlay on each output, and programming DAGs to skip enough bytes at the end of each row to jump over the data from the other display

Support for multiple heads

GraphicsV already has some support for this via call 15. However that call only allows one head to be active at once; what’s needed instead is a way of supporing multiple active heads at once.

Proposal:

• The mechanism used to describe/select overlay configurations is used to enable/disable heads. I.e. if a configuration is selected in which a head has no overlays showing, then the head will be disabled.
• For GraphicsV operations that pertain to the current head, bits 16-23 of R4 will be used to specify the head index (as opposed to the overlay index). I.e.
  • GraphicsV 1, 2, 3, 4, 5, 7, 14 act on the head
  • GraphicsV 6, 10, 11, 12, 13 act on the overlay
  • GraphicsV 9 acts on the device as a whole? (it’s assumed most/all devices will grab pixel data for all of their overlays from the same VRAM pool)
  • GraphicsV 15 deprecated
• The device descriptor needs to specify the number of heads available, along with their individual restrictions
• Heads provide a window into ‘overlay space’ – i.e. each head has a configurable position
  • Unless the head is marked as fixed in place, e.g. OMAP3
• Heads may also provide display scaling/rotation facilities?
• Thus, new OS_ScreenMode + GraphicsV calls are needed to control the above
• I’m not sure how most graphics cards handle display spanning – can heads be moved to arbitrary locations over the desktop overlay, or can they only snap to edges of other heads? What happens when two cards are combined and you want to stick a monitor from card A inbetween two monitors from card B?
  • This isn’t an issue that needs solving immediately since the OMAP3 heads are fixed in place. Can be solved at a later date should suitable hardware arrive.

Other head related improvements

• If a head has a touch screen, it should be able to specify what touch screen device it is (i.e. provide a pointer to a touchscreen HAL device?)
• If a head has an accelerometer/rotation sensor, it should be able to provide feedback on the current rotation
• Control over hardware contrast/brightness/gamma/colour phase rotation/backlighting should be possible
• If a head is attached to a dumb LCD, it should be able to indicate this (and the resolution of that LCD), so that programs and the OS know that the resolution can’t be changed. Native colour depth of the LCD should be indicated as well – there’s not much point having a 24bpp pixels being sent to a 16bpp LCD.
• Similarly, there should be a way of indicating that an LCD is greyscale
  • And if it is greyscale, and the display hardware lacks any automatic colour conversion/palette hardware, the driver must specify the fixed palette that the display uses

HAL integration

• Should new HAL calls be added for the new features, or should HALs be restricted to a VIDC level of display behaviour?
• Should a new HAL device API be specified, allowing the kernel to automatically generate GraphicsV display drivers from HAL devices?

Other notes

• GraphicsV 14 (for reading EDID data) needs revising. The NVidia implementation doesn’t follow the spec with regards to error handling (and the current OMAP3 implementation mimics it for the sake of consistency), and there’s no way of reading anything other than EDID (e.g. DDC). Also the ‘card’ and ‘channel’ specification won’t be required if R4 is used to indicate the head.
• As part of the GraphicsV improvements, it’s not expected that RISC OS will gain a Geminus-like ability to allow the desktop to be used across multiple monitors. However the GraphicsV improvements will make the implementation of such software easier (a Geminus-like program could register itself as a display driver, claim sole use of all real hardware drivers for itself, and perform the necessary behind-the-scenes magic to configure the appropriately configure the displays to create a spanned desktop)
• For simplicity, OS_Byte 19 (wait for VSync) will only act on the display that’s currently displaying the default graphics overlay.
• It may be worth adopting ROL’s Service_DisplayChanged service call for when the default display is moved to a new display, or a new head/overlay configuration on that display
  • If the default display is being moved to a new driver, software should make sure the driver is fully configured before the change is made, in order to reduce the amount of ‘chatter’ as programs adjust to the constant reconfigurations
• We will likely need some kind of VideoGuard-style system to protect against situations where the default display is invalid
  • May have to cope with taking over control of claimed drivers/overlays – could be problematic for WIMP apps which have no easy way of receiving ‘device lost’ notifications
• With all the different options, is it really sensible to have the overlay configurations described in a simple list? Flexible display hardware may end up supporting hundreds of configurations.
• Need a way of representing the scaling limitations on the OMAP3; specifically the fact that if the mode timing values are too small, some or all overlay scaling modes will be unavailable
  • Other restrictions may also apply, e.g. rotation may only be possible in certain colour depths or overlay sizes (e.g. not supported in 1/2/4bpp on OMAP), scaling may not be available in some overlay configurations (OMAP3 doesn’t support scaling with palettised buffers or buffers containing alpha channels)
  • Therefore new vetmode-style interfaces may be required to allow programs to test out a full set of parameters before attempting to program them into the device? Either that or we add more methods to enumerate the different configurations.
  • Similarly, if a restriction exists between two settings that are programmed independently of each other (e.g. video timings and overlay scaling), how to handle the case where both need to be changed but the user has no way of knowing that changing one will temporarily break the other?
• Heads need a way to specify their restrictions with regards to accepting new display settings
  • e.g. whether DAG settings & overlay positions are updated immediately, or on HSync, or at beginning of VSync
  • RISC OS currently assumes that updates occur immediately (or at the least that they can be applied at any time during VSync), but OMAP3 uses shadow registers that are flushed to hardware at start of VSync.
    • This results in the mouse pointer being one frame behind the window you’re dragging, and (since they lack knowledge of what hardware they’re running on) it essentially means that games must use triple-buffering to avoid redraw problems.
    • I’ve also experienced similar oddities with the Iyonix NVidia driver, suggesting that the hardware isn’t “VIDC-compliant”
  • Hardware could offer a HSync capability; for OMAP3-style hardware, the OS could use this to provide better synchronisation of pointer updates, by programming the new pointer position shortly before VSync