A fast and easily customised operating system for ARM devices

Drag-And-Drop Functional Specification

Document Ref:	1309,419/FS
Project:	Shared Source RISC OS release (formerly Ursula and Java 1.2)
Revision:	F * DRAFT *
Date:	16 October 2007
Author(s):	Ben Avison

Overview

This Document

This document supersedes the cut-and-paste and drag-and-drop protocol application notes [1] and [2], and specifies the cut-and-paste / drag-and-drop abilities to be added to the Wimp, plus the Clipboard module that supports the Wimp (and whose facilities will also be of use to future applications intended to support the protocols).

Key terms are defined and some familiar terms are also redefined more precisely in the glossary (§13), and the reader is recommended to read this first.

Where information is fundamentally new in this specification - non-obvious consequences of and clarifications and extensions to the existing protocol, the relevant section is underlined to bring attention to the fact.

The document may be slightly weighted towards text-handling applications, but this is because the user interface is generally more complicated in such cases. It is expected that the reader be able to extrapolate meanings to apply to any possible fundamental type of data.

Cut-and-Paste

Global-clipboard-based or cut-and-paste data transfer involves data being removed or copied from any document in the desktop to a notional "clipboard", then pasted from the clipboard into the same document, or any other document in the desktop, whether managed by the same application or not. On the way, data translation is performed in such a way as to minimise the information loss about the data.

This interface involves three operations, cut, copy and paste, which may be performed in any order. Any one data transfer will require at least two of these operations, in addition to the choice of the original selection and the destination point; the process is somewhat clumsy (and unintuitive, since the clipboard is hidden from view), so this is the least preferred technique of the two described in this document. However, it must still be provided, as it does allow some operations which cannot be achieved in any other way, and can perform other operations faster than by drag-and-drop.

Drag-and-Drop

Drag-and-drop is similar to cut-and-paste, but with the cut/copy performed by pressing a mouse button, and the paste by releasing the button at the destination. Full drag-and-drop compliance combines the features of conventional, simple drag-and-drop, as commonly used in Save dialogue boxes, with the data translation abilities of the global clipboard, and overcomes the abstract nature of the global clipboard by the displaying throughout the drag a bounding box, dithered sprite/object or representation of the insertion point.

The user interface is simpler to use - consisting typically of two mouse drags (one to select and one to move or copy), with a requirement for at most one keypress (the Shift key swaps the meaning of copying and moving). However, the programming interface is more complicated, because continuously negotiated transferable data types, destination positioning and rendering of objects and pointers are required in addition to everything needed for cut-and-paste. This complexity is possibly responsible for the low implementation rate of the protocol to date.

General

The Clipboard module is primarily to enable cut-and-paste and drag-and-drop to be reliably implemented for writable icons, which are handled automatically by the Wimp. (The fundamental problem is that the established cut-and-paste and drag-and-drop systems are Wimp-message-based, and the Wimp itself is poorly equipped to send and receive messages.)

However, the module is secondarily to handle the complex protocols on behalf of any application that chooses to do so - and in the process, producing a more uniform user interface. This is to be the preferred method of implementing the protocols in future, although in special cases, the tasks it performs can be split into four separate areas, any combination of which can be taken advantage of by the same application:

clipboard management - supporting cut and copy operations
clipboard procurement - supporting paste operations
sending drag-and-drop data
receiving drag-and-drop data

Outstanding Issues

Bounding Box Discrepancies

It is possible that data may have different "real life" bounding boxes in different data types - for example, a DrawFile sprite object may be transformed and/or scaled, and thus have a different bounding box to the underlying sprite. Thus if a transformed sprite object were dragged from Draw to a sprite editor window, the bounding box would not represent the final position of the sprite.

Technical Background

This Document

Some important pre-existing technical terms are rigorously defined in the context of cut-and-paste / drag-and-drop in the Glossary in §13, along with some new terms introduced in this document.

Where technical background information is relatively straightforward (no more than one or two short paragraphs), it is included alongside the appropriate part of §4 or §5, for ease of reference.

Previous Documents

Simple drag-and-drop operations, such as those employed by Save dialogue boxes, do not employ any inter-task negotiation during the drag, and use the plain DataSave/DataLoad protocol during the drop, as described in the Programmer's Reference Manual [3].

The Style Guide [4] first indicates selection models, then describes an overview of cut-and-paste and drag-and-drop behaviour, before referring the reader to the relevant Support Group Application Notes [1] and [2].

Previous Applications

Old-style selection model / copy-and-move implementations are still in existence, especially so in the case of text editors, (e.g. Zap, StrongEd, SrcEdit) which have often followed Edit's example. Such schemes typically involve only three actions to perform an operation (select region, position caret, move/copy keypress) rather than the four (select region, cut/copy keypress, position caret, paste keypress), but have the disadvantage that they can present both a caret and a selection to the user at the same time, which is potentially confusing. Old behaviour will remain deprecated, but new cut-and-paste / drag-and-drop applications must be able to interact with tasks that use it.

Other applications may support just cut-and-paste, or just drag-and-drop. Drag-and-drop will be the favoured technique in future, due to the simpler actions required (select region, drag region), but cut-and-paste must also be supported as well, to cater for cases when drag-and-drop cannot be used (e.g. when copying on to a menu tree).

Some aspects (e.g. pointer shape, window) of applications already written to the application note guidelines (e.g. DataPower, EasiWriter/TechWriter) are already inconsistent, due to the lack of detail in the application notes. Applications will in future be encouraged to follow the more detailed specification herein.

User Interface

Selection

Protocol

Rendering

Selection is rendered either by a recolouring an object (with its photographic negative or otherwise) or by drawing a bounding box, typically in red, and optionally with one or more "handles" for resizing and/or rotating operations as appropriate:

It should be emphasised that a caret must never appear within the same window as a selection at any time, not even during the selecting drag. Placing a new caret or selection removes any caret or selection previously active in the same window. (The definition of a window for these purposes is a top-level window and its panes and all their child windows.)

However, if during (or immediately after) the selection process, the selection would have to be drawn with zero width (i.e. for text selections, when the two ends snap to the same character boundary), a caret must be displayed instead. It is helpful to consider the caret as a zero-width selection; only one selection may be present within one window at a time, so the exclusivity of the caret and selection is an extension of this concept.

When either a caret or a selection is placed in a document, the window must gain the input focus. This will happen automatically for a Wimp-drawn user caret, but in the case of application-drawn carets and selections, the application must position the Wimp caret in the window, but marked as invisible, in order to achieve the same effect. Simply changing the window border colour (as is possible in Wimps since RISC OS 4) is not acceptable.

When a caret or selection is placed in a different drag-and-drop window, the old selection must be redrawn as a shaded selection, not left as is or removed entirely. The caret must be removed entirely (or optionally redrawn as a shadow caret); if the application uses a Wimp-drawn user caret, the caret will be removed for it automatically. The LoseCaret and GainCaret events must not be used to determine when this is necessary, as the Wimp may "borrow" the caret temporarily while a menu is open. A Wimp message exists to indicate when removal or redraw of the caret and selection is necessary, and must be used in preference to the events (see §5.1.1).

Note that a selection may also be non-shaded but not have the input focus if an application not adhering to the cut-and-paste / drag-and-drop protocols had grabbed the Wimp caret (and therefore the input focus). Similarly, non-Wimp-drawn carets may be deprived of the input focus under similar circumstances while neither being removed, nor being replaced with a shadow caret.

Mouse Events

Non-contiguous selections (just about everything except text and DTP documents) will continue to be handled as described in the Style Guide [4], with the proviso that clicking Select over an already-selected object must not deselect anything, as a Select click event always precedes the Select drag event which initiates a drag-and-drop operation. Appropriate action must also be taken to un-shade shaded selections when necessary.

On the other hand, a more detailed behaviour for contiguous selections must be adhered to in future. In summary:

Select click outside the selection (or when there is no selection, or at one end of a selection): position the caret at the pointer position, and flag the next Select drag as creating a selection.
Select click on a selection: if the selection was shaded, un-shade it. Make sure the window has the input focus. Flag the next Select drag as being a drag-and-drop drag.
Select click in a "dead" region of a window (e.g. in a page border): un-shade any selection or replace any shadow caret with the caret, if either exists. Make sure the window has the input focus. Optionally, flag the next Select drag as causing an window scroll operation (as Impression, TechWriter etc. do as present), but certainly not as starting a selection or drag-and-drop operation.
Select drag-start event: remove the caret, and set the selection from the old caret position to the current pointer position. Alternatively, start the drag-and-drop operation (see §4.4.1). The exact meaning is determined by the flag that was set at the Select click stage.
During Select drag: if creating a selection, adjust the most recently touched end of the selection to the pointer position at regular intervals; autoscroll the window if necessary, using the SWI Wimp_AutoScroll introduced in the RISC OS 4 Wimp. For what to do during a drag-and-drop drag, see §4.4.1.
Select double-click: select a word (as defined in the Style Guide), irrespective of whether the click is on an existing selection or not.
Select click-drag (button pressed, then released, then pressed again within the double-click limits, then held or moved according to the drag limits): equivalent to a normal selection-delimiting drag, except that the selection limits are rounded to word boundaries (excluding whitespace at either end).
Adjust click when there is no caret or selection: position the caret at the pointer position, unless there was a shadow caret, in which case, position the caret where the shadow caret was. Set a flag to indicate that there was no caret or selection before the Adjust click (the shadow caret doesn't count); do not rely on the fact that there is no selection displayed when the drag event is generated to flag this, as a zero-width selection may have been displayed as a caret instead.
Adjust click when there is a caret: remove the caret, and set the selection from the old caret position to the current pointer position.
Adjust click when there is a selection: grow or shrink the selection so that the nearest end of the selection moves to the pointer position. (Remember to un-shade the selection and/or gain the input focus if necessary.)
Adjust click in a "dead" region of a window: the same as Select in these circumstances.
Adjust drag-start event: unless there was no caret or selection before the preceding Adjust click, adjust the most recently touched end of the selection to the pointer position.
During Adjust drag: unless there was no caret or selection before the preceding Adjust click, adjust the most recently touched end of the selection to the pointer position at regular intervals; autoscroll the window if necessary, using SWI Wimp_AutoScroll.

Other operations (including Adjust double-clicks, higher-multiple-clicks and combinations with shifting keys) are left to the application to respond to as it sees fit - they might select a line of text, or select something in another layer, or whatever. Typically, higher-multiple clicks of Select will select progressively larger blocks of text. For single-line items such as writable icons, three clicks means select the entire line. Once the maximum number of clicks is reached, the next click is interpreted as for a single click, so for writable icons, a quadruple-click sets the position of the caret.

In order to make selections over a larger area than can be displayed in a window, during selecting drags of contiguous selections and select box drags of non-contiguous selections, autoscrolling can be implemented. However, since there is rarely a meaning to making a selection spanning several windows, there is only one meaning to moving the pointer off the window, and so there must be no need for a pause over the autoscrolling zone to precede commencement of scrolling, as this would merely slow down the user's actions.

Keypresses

There are some special keypresses relating to cut-and-paste that affect the selection. Obviously, these only apply to selections that have the input focus (and therefore never apply to a shaded selection). The keypresses are:

Ctrl-Z: clear the selection (i.e. undraw the highlights), and place the caret (if appropriate) at the right-hand end of the old selection (or the left-hand end in a right-to-left language).
Ctrl-V or Insert: delete (not cut) the selected data, and place the caret (if appropriate) at the end of the newly inserted text.
Ctrl-X, Backspace or Delete: cut the selected data and place the caret (if appropriate) where the selection was.
Ctrl-K: delete (not cut) the selected data and place the caret (if appropriate) where the selection was.

Then there are a number of special behaviours for textual regions:

Left-arrow/up-arrow: clear the selection, and process the keypress as though the caret had been at the left of the selection.
Right-arrow/down-arrow: clear the selection, and process the keypress as though the caret had been at the right of the selection.
Any other repositioning keypresses (Home, Tab etc.) behave along similar lines, as appropriate to the application.
Any other keypresses that would normally insert one or more characters: perform a cut operation, then position the caret where the selection was, and process the keypress as normal.

Any other keypresses must not affect the selection.

During drags (both those that set a selection and those that copy or move one), no keypresses that would normally affect the selection must be acted upon.

Scope

When a caret or selection is placed in the same window where one already exists, the old one is removed (not just re-rendered as a shaded selection). In order for this to be consistent with the use of input focus colouring of windows, all carets and selections must be unique within a group of windows characterised thus: a top-level (non-nested) window, all its panes, and all windows nested within the window or one of its panes. If a task does its own selection handling but the window or one of its panes also uses writable icons, the task will need to monitor caret/selection updates to the writable icons in order to deselect its own selections.

Carets and selections must not be preserved when a window is closed, deleted or iconized (check for Open_Window_Requests with handle-to-open-behind of -3 to detect iconization). The Wimp takes care of everything for Wimp-drawn carets, and automatically removes the input focus in any case. If a selection can be made in a dialogue box opened from a menu, then the task must act as though the window were being closed when receiving Message_MenusDeleted, as tasks are not sent the usual Close_Window_Request for such windows.

When the window is being closed or deleted, application-drawn carets and selections must be marked as absent, but when it the window is being iconized, carets and selections drawn by the application must be flagged as a shadow caret (if supported) or a shaded selection, respectively, ready for the next redraw request.

Clipboard Module

The Clipboard is not involved in the selection process.

Writable Icons

Up to one writable icon selection may exist in each window, and the selection will only be un-shaded if the window has the input focus.

Rendering

Carets within writable icons will be Wimp colour 11 (red), irrespective of the background colour of the icon. This will be achieved by using (Wimp colour 11 EOR background colour), calculated in GCOL space, as the colour to EOR on to the icon.

Selections and shaded selections will be drawn by switching the foreground and background colours, then fading them if appropriate. A gap of 4 OS units will be left before the top and bottom borders (if any) of the icon. Therefore, a typical writable icon will look like this in its three states:

Using this method is better than EORing a block of colour, especially in the shaded selection case, where the anti-aliasing of the text is destroyed by an EOR operation. It also means that non-standard writable icons are catered for sensibly as well with no extra effort:

In the past, during writable icon redraws caused by scrolling of the icon (caused, for example, by repositioning of the caret), there has been a certain amount of flicker, both of the text, and of the caret itself, especially for large writable icons. This will be exaggerated substantially if the same technique is used to draw selections; to reduce flicker in both cases, a new algorithm will be written to deal with icon updates following caret / selection / ghost caret changes, utilising block copies wherever possible.

Scrolling

Icons where the text is less wide than the icon are relatively simple; the text has a fixed position, irrespective of caret and selection position. But it is likely that where the text is wider than the icon, occasions will arise where the user needs access to areas of the text string that are normally hidden, in order to set one or both ends of a selection. The matter is similar to the requirement for icon scrolling to position the ghost caret during a drag-and-drop selection (see §4.4).

So, while the user is delimiting a selection, or when a ghost caret is displayed in the icon, an autoscrolling scheme will be followed, directly analogous to that used for windows in Wimp_AutoScroll.

Note in particular:

The speed of scrolling is proportional to the distance the pointer has moved beyond the inside edge of the autoscrolling "pause" zone. This is because this scheme allows fine user control of both acceleration and deceleration.
When delimiting a selection, autoscrolling will start as soon as the pointer enters the "pause" zone - i.e. a pause time of zero is used. Conversely, to start autoscrolling during a drag-and-drop operation, the pointer must be held over the pause zone for the configured pause time. This matches the equivalent behaviour for autoscrolling of windows.
While document windows are generally of a comparable size, hence the similar pause zone widths, the size of writable icons can vary dramatically from icon to icon - compare, for example, a writable icon that is part of a numeric field, with the URL at the top of a web browser window. Scrolling speeds that would suit a small icon would be painfully slow for a very large one, and usable speeds for a large icon would scroll a small icon far too quickly. Therefore, the scrolling speed of a writable icon when the pointer is at one end will be proportional to its width. However, it is also desirable that the scrolling speed ramp up at the same rate, irrespective of icon size; these two constraints imply that a fixed proportion of the width of any icon needs to be allocated as the autoscroll pause zone - we will use 1/4 of the width at each end, as illustrated to scale below:

Below, the autoscrolling zone is cross-hatched; the autoscrolling pause zone is the intersection of the autoscrolling zone with the icon bounding box:

Mouse Events

Mouse events in writable icons will follow the general behaviour, as specified in §4.1.1.2, but with a couple of slight changes. The definition of a word will match that used for Shift-arrow navigation, i.e. treating both spaces and the '.' character as word delimiters.

On the second click of a double click with Adjust, the selection established by the first Adjust click will be extended outwards at both ends to include complete words.

Clicks with either Select or Adjust will not affect the text origin, unless they are setting the caret position (which will still be centred as far as possible, for consistency with old Wimps). Neither double-click operation will affect the text origin either, unless a scroll was caused by the first click of the two.

During a drag, while the pointer is over the central zone between the autoscrolling zones, no scrolling occurs. The autoscrolling zones act just like those of windows. After each scroll step (not before), the selection end is determined by the closest character boundary to the pointer.

When a drag starts, any movement of the text which was performed at the time of the click event is undone. This is necessary because otherwise we have introduced a relative movement between the text and the pointer which was not intended by the user, and the alternative (moving the pointer) is less in the style of the RISC OS user interface. Consider, for example, if the user clicks Select at the right hand end of an icon where there is a lot of text further to the right which is clipped out of view: if the user starts dragging to the left, but as a result of the initial click, the text had jumped quickly to the left of the pointer and so the user is now creating a selection to its right; worse still, if the pointer is still over the autoscrolling zone, the initial character may start scrolling off the left of the icon, leaving a large selection in the opposite direction to that intended by the user!

Keypresses

These will in general be handled as in §4.1.1.3. Note in particular:

Only keypresses as specified in the validation string would normally insert characters, so any others (except Ctrl-X and its synonyms, of course) will not cut any selected text.
Whenever a keypress (including Ctrl-X and synonyms) causes the caret to be repositioned, a traditional, centred caret will be used.
When a paste is performed, and so an entirely new selection is set, the selection will be centred within the icon (unless it is wider than the icon, in which it will be right-aligned).

Wimp Selections and Menus

When the pointer moves over a writable menu item, or when a dialogue box containing writable icons is opened from a menu, the Wimp automatically places the caret in the menu item, or the first writable icon, respectively. The Wimp remembers the position of the caret before it does this, and then returns the caret to its old location afterwards.

This behaviour will be extended to check for Wimp selections that have the input focus before the caret is placed. If the same selection still exists afterwards (i.e. a selection has not been made within the menu structure in the meantime), then the input focus will be returned to it.

Note that selections cannot be made in writable menu items, as any clicks are considered as choices from the menu tree before being considered as requests to set the caret position, let alone setting a selection. Also note that drags cannot be made to an icon in a menu structure, as the click that starts the drag will close the menu structure before the drag begins!

Cut and paste will work as specified for writable icons in dialogue boxes in menu trees, and pasting (but obviously not cut or copy) will work for writable menu items.

Password icons

Cutting, copying and dragging from a password icon, or pasting or dragging into one, is not permitted for security reasons. To give the user some feedback, the Wimp issues a system beep if the user attempts to do so. Selecting text in a password icon is permitted, although the only action that can be performed on it is deletion.

Application-altered Indirected Data

On occasions, the text of a writable icon is altered by code other than the Wimp's writable icon handling code (and as a prerequisite, the text data has to be indirected). A common example of this is the writable numeric range, where adjuster arrows may be used to alter the value inside the accompanying writable icon.

Altering the data does not, in itself, cause any screen updates to be done; applications have to force a redraw of the icon for the new value to be displayed. During the redraw, the caret is redrawn, but only using the last work-area-origin-relative co-ordinates calculated the last time the caret was positioned. If the new data requires a different text origin, the caret will then appear incorrectly positioned. To cater for this, nearly if not all applications set the position of the caret again, as well as forcing a redraw of the icon.

A similar situation could arise with selections (and even ghost carets) - but since no existing applications know about selections, they will not be able to cater for the "feature" in the API. For example, suppose the value 99 was selected in a centred numeric range, then the up-arrow was pressed; the result would be as below:

To work around this, separate checksums will be kept for the text of the icon currently containing the selection and the ghost caret. Each time an icon is redrawn, a new checksum is calculated, and if the checksum has changed, the selection or ghost caret will be removed. This is because the change to the text has probably invalidated the selected text anyway.

A variant on this approach will be used to fix the equivalent long-standing bug in the case of carets. One potential fix which we have to reject is to simply remove the caret when the text changes, because in many cases the application already has its own workaround whereby it reapplies the caret, so with each change of the text, the window's title bar would flicker due to losing and re-gaining the input focus. Instead, the Wimp will recalculate the caret position, assuming the same index into the string is required - unless the icon's numeric flag (icon flag bit 20) is set, in which the caret will be kept at the same index from the end of the string, to preserve the decimal place being edited. This way, future applications need not include the workaround at all.

Scope

In addition to the rules in §4.1.1.4., any combination of caret, ghost caret and selection must be removed when an icon is deleted. Also, when a menu is closed, any selection in a dialogue box linked to the menu must be removed (the caret already is removed in these cases).

It will not be possible for any caret, selection or ghost caret to be placed in a writable icon that is shaded. If any caret, selection or ghost caret is present in a writable icon when it becomes shaded, they will be removed.

The Wimp selection and both carets will be removed when the Wimp font is changed, but this will be the responsibility of the task that is changing the font - namely !Configure (or more precisely, a configure plug-in).

If an icon is resized using Wimp_ResizeIcon, any of the caret, ghost caret and selection which are present in the icon will be marked absent (although no redrawing will occur immediately, because Wimp_ResizeIcon expects to be followed by a separate redraw operation anyway).

Draggable-Writable (Type 14) Icons

Type 14 (draggable) writable icons are much rarer than standard, type 15 writable icons, and in the past, have only differed in that drag events are reported to the task. Some applications (such as Fresco) have taken advantage of these icons to implement a simpler form of drag-and-drop; such behaviour would be broken if the steps described above were employed for type 14 icons. Type 14 icons could also be a useful special case, where sub-units of the information in the icon have no meaning on their own, and where only the entire text can logically be dragged-and-dropped.

Therefore, all of the rest of §4.x.3 and §5.x.3 (with the exception of developments specific to carets, such as the bugfix in §4.1.3.7) will only apply to type 15 writable icons.