Proposed GraphicsV enhancements

Goals

• Provide a well-defined method for handling multiple active displays
  • The current API supports multiple displays, but does not specify how RISC OS behaves in a situation whre multiple displays are present, nor does it give an API to allow programs to enumerate the set of available displays
• Provide an API suitable for controlling all the features of the OMAP3 display subsystem
  • So that the new OMAP3 video driver can expose all of the hardware’s functionality to RISC OS and user programs
• Provide support for displays being added and removed at runtime
  • E.g. for DisplayLink-based USB monitors/video adaptors
• Allow programs to claim sole use of displays, hardware overlays, etc.
  • You don’t want to have two media players trying to use the same YUV overlay at the same time!
• Add API-level support for the new display formats proposed in the extended framebuffer specification
  • Note that this does not necessarily mean that the rest of RISC OS will be expected to understand the formats!
• Add anything sensible that will make Geminus’s life easier?
• Attempt to re-use any of ROL’s new APIs wherever possible

Existing documentation

• Extended Framebuffer Format Specification
• GraphicsV
• RISC OS Select display changes
• RISC OS Select OS_ScreenMode changes
• RISC OS Select VDU Variables

Detail

Support for multiple displays

The current GraphicsV API specifies that the top byte of R4 is used to specify the display number, with 0 being the ‘default’. However, no methods are defined (or implemented) for enumerating the list of available displays, notifying the system of display addition/removal, etc.

To rectify this, the following changes are proposed:

• Adoption of ROL’s Service_DisplayStatus service call to notify programs of display addition/removal
• Adoption of ROL’s OS_ScreenMode 11 to allow selection of the default display
• Adoption of ROL’s OS_ScreenMode 253 to obtain the current maximum display number
• Adoption of ROL’s OS_ScreenMode 254, 255 to allow drivers to register/deregister themselves as displays
• Adoption of ROL’s use of VDU variable 192 to store the current display number
• The GraphicsV documentation mentions that display 0 is the ‘default’. However it’s unclear what is meant by this (that display 0 is the first in the list of displays, or something deeper, e.g. if the desktop is switched from one display to another, does the corresponding display driver now have to check for any requests being made to display 0?) Thus, for the avoidance of doubt, the new system will not treat display 0 any differently from any other display. This means the OS will always address the intended display directly, rather than relying on drivers having to perform two checks (e.g. “if ((display == me) || ((display == 0) && (default == me)))”)
• Add a new OS_ScreenMode call to allow programs to claim sole use over displays (and another to relinquish that claim)
  • See the Display Ownership section
• Add a new OS_ScreenMode call (and associated GraphicsV call) to read details/capabilities of a display
  • See the Display Capabilities & Configuration section

Support for hardware overlays

The primary aim of this change is to provide support for the features of the OMAP3 display subsystem. Thus, the new GraphicsV must support multiple hardware overlays, each with differing restrictions on available buffer formats, and provide the ability for programs to alter the display order of the overlays (where the hardware supports it)

To add support for hardware overlays, the following changes are proposed:

• Bits 16-23 of R4 are used to specify the overlay number when calling GraphicsV
  • TODO – Is this needed if we go with the settings table system?
• A mechanism is provided to allow drivers to indicate how many overlays they support and what capabilities each overlay has
  • See the Display Capabilities & Configuration section
• A way of describing the different overlay configurations available
  • See the Display Capabilities & Configuration section
• An OS_ScreenMode call to select the overlay configuration number, the overlay for RISC OS to use for display output, and the overlay for RISC OS to use for pointer display
  • The latter two parameters are only important if the display is to be made the default display via OS_ScreenMode 11
  • Configuration info would also be passed through to GraphicsV
  • TODO – How does “overlay configuration number” fit in with the settings configuration table stuff?
• New VDU variables are added to store the current overlay configuration number, and which overlay RISC OS uses for the main display and the mouse pointer
• An OS_ScreenMode call (and associated GraphicsV call) to select/get chroma-key colour value for each overlay
  • See the Display Capabilities & Configuration section
• Add a new OS_ScreenMode call to allow programs to claim sole use over overlays (and another to relinquish that claim)
  • See the Display Ownership section
• New OS_ScreenMode + GraphicsV call to alter/query overlay position, rotation, scale factor
  • See the Display Capabilities & Configuration section
• New OS_ScreenMode + GraphicsV call to alter/query overlay size, colour format, buffer address
  • See the Display Capabilities & Configuration section

Support for multiple heads

GraphicsV already has some support for this via call 15. However that call only allows one head to be active at once; what’s needed instead is a way of supporing multiple active heads at once.

Proposal:

• Add a mechanism to enable/disable heads
  • See the Display Capabilities & Configuration section
• For GraphicsV operations that pertain to the current head, bits 16-23 of R4 will be used to specify the head index (as opposed to the overlay index). I.e.
  • GraphicsV 1, 2, 3, 4, 5, 7, 14 act on the head
  • GraphicsV 6, 10, 11, 12, 13 act on the overlay
  • GraphicsV 9 acts on the device as a whole? (it’s assumed most/all devices will grab pixel data for all of their overlays from the same VRAM pool)
    • Not entirely true – for rotation support on the OMAP it may be preferable to give each overlay its own memory pool (along with better driver control over memory allocation/usage, similar to Select)
  • GraphicsV 15 deprecated
• The device descriptor needs to specify the number of heads available, along with their individual restrictions
• Heads provide a window into ‘overlay space’ – i.e. each head has a configurable position
  • The API must also support heads that are fixed in place, e.g. OMAP3
• Heads may also provide display scaling/rotation facilities?
  • i.e. in addition to/instead of the overlay scaling/rotation
• Thus, new OS_ScreenMode + GraphicsV calls are needed to control the above
• I’m not sure how most graphics cards handle display spanning – can heads be moved to arbitrary locations over the desktop overlay, or can they only snap to edges of other heads? What happens when two cards are combined and you want to stick a monitor from card A inbetween two monitors from card B?
  • This isn’t an issue that needs solving immediately since the OMAP3 heads are fixed in place. Can be solved at a later date should suitable hardware arrive.

Other head related improvements

• If a head has a touch screen, it should be able to specify what touch screen device it is (i.e. provide a pointer to a touchscreen HAL device?)
• If a head has an accelerometer/rotation sensor, it should be able to provide feedback on the current rotation
• Control over hardware contrast/brightness/gamma/colour phase rotation/backlighting should be possible
• If a head is attached to a dumb LCD, it should be able to indicate this (and the resolution of that LCD), so that programs and the OS know that the resolution can’t be changed. Native colour depth of the LCD should be indicated as well – there’s not much point using a 24bpp screen mode with a 16bpp LCD.
• Similarly, there should be a way of indicating that an LCD is greyscale
  • And if it is greyscale, and the display hardware lacks any automatic colour conversion/palette hardware, the driver must specify the fixed palette that the display uses
  • This is a low priority at the moment, since devices with low-colour/greyscale displays aren’t very attractive targets for RISC OS ports
• TV-out heads – need way of specifing & controlling capabilities (PAL/NTSC, widescreen, overscan/tuning, etc.)

Most, if not all of the above, is capable of being covered by the settings table described in the Display Capabilities & Configuration section.

HAL integration

• Should new HAL calls be added for the new features, or should HALs be restricted to a VIDC level of display behaviour?
• Should a new HAL device API be specified, allowing the kernel to automatically generate GraphicsV display drivers from HAL devices?

Dispay Ownership

Both heads and overlays have ownership states. There are three different states of ownership:

1. Free
2. System
3. Program

• Programs can claim heads/overlays only if they are in the Free or System state.
• The system may reclaim a head/overlay from a program at any time. The owning program will be notified of this, using a TBD API.
• Both programs and the system are capable of releasing the heads/overlays they own back into the ‘free’ state.
• Ownership of a head gives the owner sole control over any head-related configuration. If control over the overlays associated with that head is required, then additional requests must be made to claim those overlays.

This system allows programs to coexist with the VDU much as they do now. Programs can use the standard OS APIs to change the screen mode, or they can claim ownership of the default head and overlays and set everything manually. If the program exits then the desktop will reclaim the head & overlays for the system, resetting the right WIMP mode. Additionally, the inability of one program to reclaim an overlay that is owned by another allows for the safe implementation of media players and the like where more YUV overlays may be requested than actually exist.

Registration API

If a program wishes to do more than use the basic OS_ScreenMode interface that exists now, then it must register itself with the system via a new OS_ScreenMode reason code. Upon registration the program will provide a pointer to a callback function that will be called upon certain important events (e.g. the system reclaiming a head or overlay that is owned by the program). Upon registration the program will also provide a list of understood property IDs (see the Display Capabilities & Configuration section); this will allow drivers to make ‘best guess’ choices for configuration values that the program does not provide or understand. In return for the registration request, the system will return to the program a unique ID that allows the system to identify the program. The program uses this ID when it wants to claim or relinquish ownership over resources.

New calls:

• OS_ScreenMode X1 – Register with system
• OS_ScreenMode X2 – Deregister with system
• OS_ScreenMode X3 – Claim entity (head or overlay)
• OS_ScreenMode X4 – Release entity (head or overlay)

Further API details are TBD.

Display Capabilities & Configuration

There are many restrictions which govern how the different features of the OMAP3 display controller can be used. For example, the display timings affect the range of supported overlay scales; the colour format affects whether rotation is supported; scaling is only available with certain transparency methods; the colour phase rotation matrix is not available for TV-out; the limited number of transparency modes/overlay display orders places restrictions on the system if you want a YUV overlay to appear ‘under’ the desktop (i.e. so menus/windows and the mouse can appear ontop of it); PLL restrictions mean that the system isn’t capable of generating every pixel rate that lies between the min & max value; it may make sense to restrict configurations that use excessive memory bandwidth (e.g. having three fullscreen 32bpp overlays utilising transparency), to utilise hardware display rotation the driver will have to exert tight control over VRAM assignment, etc. Although creating an API that allows these restrictions (and any others that might apply to other hardware) to be communicated to programs is likely impossible, we can at the least strive for an API that:

1. Allows programs to make multiple changes at once. This allows programs to switch the hardware from one valid state to another without having to navigate through the maze of invalid states that exist between the two if bulk changes aren’t supported.
2. Allows programs to verify changes before attempting to apply them to the hardware. Existing APIs offer functionality similar to this (GraphicsV 7, OS_ScreenMode 2, etc.). E.g. if a video player wants to open a YUV overay it may try several different colour formats, transparency modes, overlay oderings, etc. until it finds the most satisfactory one supported by the hardware.
3. Allows drivers to give basic feedback about which configuration values are invalid and what can be done to correct them
4. Allows programs to tell the driver what configuration values are understood, and therefore allows the driver to program the ununderstood properties with best-guess values (e.g. colour phase rotation matrices, gamma controls, etc.)

As an attempt to satisfy the above criteria, all the configuration values of a driver will be contained within a table. Each column represents an entity within the driver (a head or an overlay), and each row represents a property. The values in the cells represent the configured values within the driver. By passing tables back and forth between programs and drivers, it is possible for programs to safely alter the configuration of the driver, to receive feedback about the driver’s current configuration, and to receive feedback about which configuration values are invalid.

Ordinarily the only rows that will be present in a settings table will be those that the program indicated to the system that it understood upon registration (via OS_ScreenMode X1). How to handle columns (i.e. whether all columns must always be present, or whether they are sometimes omitted) is TBD.

To use these new APIs, programs must first register themselves (see above)

New calls:

• OS_ScreenMode X5 – Get driver details
  • Returns number of heads & overlays, and list of supported properties
• OS_ScreenMode X6 – Get driver configuration table
  • Using a supplied property list or the one used upon registration
• OS_ScreenMode X7 – Verify/update driver configuration table
  • Used to verify a proposed configuration, and potentially apply that configuration to hardware
  • Driver will update the caller’s table with details of any changes that have been made to the entities that the program doesn’t own
  • It will then verify that the proposed settings are valid (automatically generating any values for properties that the program indicated it didn’t understand)
  • If the settings are valid (and the program wants to apply them to hardware) the hardware will be updated
  • If the settings are invalid, the driver will suggest changes by modifying the user’s table, and return those new settings to the program for the program to examine
• OS_ScreenMode X8 – Get lock state of driver’s variables
  • Returns bitmap indicating whether each table cell is configurable or locked in hardware
  • Returns list indicating the ownership of each column (free, system, program, your program)
• OS_ScreenMode X9 – Get supported values for a property
  • Parameters are driver, entity (table column index), property ID
  • For enum-based properties, returns a list of acceptable values (exact meaning of list items will be dependent upon the data type, e.g. colour formats vs. supported rotations)
  • For (flexible) numeric properties, returns three numbers – min, max, and smallest step

List of properties

Other notes

• GraphicsV 14 (for reading EDID data) needs revising. The implementation in the NVidia module doesn’t follow the spec with regards to error handling (and the current OMAP3 implementation mimics it for the sake of consistency), and there’s no way of reading anything other than EDID (e.g. DDC). Also the ‘card’ and ‘channel’ specification won’t be required if R4 is used to indicate the head.
• As part of the GraphicsV improvements, it’s not expected that RISC OS will gain a Geminus-like ability to allow the desktop to be used across multiple monitors. However the GraphicsV improvements will make the implementation of such software easier (a Geminus-like program could register itself as a display driver, claim sole use of all real hardware drivers for itself, and perform the necessary behind-the-scenes magic to configure the appropriately configure the displays to create a spanned desktop)
• For simplicity, OS_Byte 19 (wait for VSync) will only act on the display that’s currently displaying the default graphics overlay.
• It may be worth adopting ROL’s Service_DisplayChanged service call for when the default display is moved to a new display, or a new head/overlay configuration on that display
  • If the default display is being moved to a new driver, software should make sure the driver is fully configured before the change is made, in order to reduce the amount of ‘chatter’ as programs adjust to the constant reconfigurations
• We will likely need some kind of VideoGuard-style system to protect against situations where the default display is invalid
  • May have to cope with taking over control of claimed drivers/overlays – could be problematic for WIMP apps which have no easy way of receiving ‘device lost’ notifications
• Monitor/TV detection – need an API for that
  • Basic TV detection for TV-out heads
  • VIDC monitor lead ID reading for VIDC
  • EDID/DDC reading for newer hardware
• Control over widescreen signalling for TV out – need an API for that (extra vidclist3 control field? setting in settings table?)
• Need to sort out VIDCBandwidthLimit!
  • Should be down to the driver to decide which bandwidths are and aren’t sensible
• The settings table approach is able to prevent the driver from using invalid settings, and to allow the driver to (attempt to) automatically correct the settings, but doesn’t provide a good way of describing all the interdependencies between configuration values that controls whether the settings table is “good” or “bad”. An API to allow programs to select valid configurations without resorting to trial-and-error would be advantageous, but probably impossible due to the number of factors that could result in a set of settings being deemed invalid.
  • Because of this, any software such as MakeModes is likely to require custom routines for each supported display driver/hardware
• Display rotation on OMAP3 must typically be performed by utilising the VRFB module of the SDRAM controller. This allows sections of SDRAM to be treated as tiled images. The image is mapped to four physical address ranges (for 0, 90, 180, 270 degree rotation). Thus, to cleanly support display rotation on OMAP3, display drivers must have greater control over VRAM allocation than they do at present – each overlay will effectively require its own dynamic area. This will also have repurcussions for any software that wants to implement display spanning using one large framebuffer (if rotation is in use, then the other display must be capable of operating from the OMAP’s framebuffer, and the screen sizes will likely have to be a multiple of the tile size)

Document history

v1 – 31/1/2010

v2 – 28/3/2010

• Corrected some fallacies
• Attempted to make ‘default display’ discussion clearer
• Added Display Ownership and Display Capabilities & Configuration sections
• Added references to above sections in the appropriate places, deleting waffle where appropriate