Filing System Update

RISC OS Filing System Update

This page describes plans for changes to update the RISC OS filing system stack.

Goals

• Support large files (64 bit offsets).
• Support foreign format discs (e.g. FAT32 USB drives) larger than 2GB.
• Support a modern disc format to use as a native filesystem, that will handle large discs more efficiently.
• Provide a standard method of partitioning discs.
• Provide support for inserting additional block driver layers to implement things like RAID.
• Implement file and directory cacheing.
• Provide an interface for storing arbitrary metadata with files on filing systems that support it.
• Add support for symlinks.
• Simplify interfaces (for new code) wherever possible.
• Retain backwards compatibility for userspace applications.
• Retain backwards compatibility for old filing systems.
• Provide a unified formatter which can write any supported formats to any hardware.
• Provide a standard way of creating filer iconbar icons.
• Some support for nonblocking/async IO?

Plan

The following is an initial plan on how the changes can be split into smaller steps that can be implemented and tested individually.
Some of the phases could be done in a different order, or in parallel.

Phase 1

• Design new userspace APIs
• Trivially implement them, mapping to the old ones.
• Provide a patch to do the same for older OSes.
• Ideally work with ROL so the same can be added to RO6.

Phase 2

• Design new interface between fileswitch and filesystems.
• Implement fileswitch changes (possibly as a separate module, so it can be in C).
• Non-disc filing systems (e.g. Sunfish) could then be ported to the new interface.

Phase 3

• Design new block driver interface (common to discs and image filing systems).
• Port some filing systems (e.g. Filecore, DOSFS) to the new interface, and test as an imagefs.

Phase 4

• Port or rewrite some hardware drivers to implement the new block interface.
• Can now test with a normal ADFS/Filecore hard disc using the new stack.

Phase 5

• Implement disc formatter.
• Implement disc partitioning.
• Implement file cacheing.

Phase 6

• Port a new filing system (e.g. ext3, ZFS) to RISC OS, using the new interfaces.

Userspace API changes

General concepts

Filetypes

Load and Exec addresses are only supported on old filing systems, new ones can only set the datestamp and the filetype separately. Therefore the new SWIs do not provide any method of reading or writing the load/exec addresses. Existing SWIs may give an error if used on a new filing system and the load/exec addresses written do not represent a filetype and datestamp. All new interfaces use 32bits for the filetype, and it is expected that new filesystems will store the filetype as 32bits where possible.

Symbolic links

New filing systems may support symbolic links. Symlinks in any part of a pathname other than the leaf will always be dereferenced by fileswitch (and an error will be returned if they are broken links). New calls have a flag to indicate if the symlink should be dereferenced or not when it is the leafname. Old calls that do not have this flag will always dereference (with the exception of OS_File 6, which will always delete the link and not the target). In new calls that return an object type (OS_File 26 and OS_GBPB 13) a value of 4 is used to signify a symlink. This value will only be returned if symlinks are not being dereferenced by the call, or if the leaf is a broken symlink).

Extended attributes

New filing systems may support extended attributes, to allow arbitrary metadata to be stored with any file or directory. An extended attribute consists of a name (null terminated, can contain any characters except null), and associated data (which can be any length, and may contain binary data). To avoid clashes with attribute names, they are organised into namespaces. A name should consist of its namespace, followed by a colon, followed by the attribute name. Applications can use a unique namespace for any data they store, to avoid overwriting data from another application. Should namespaces be allocated by the RISC OS allocations service? Two namespaces are currently reserved; riscos for any use by the OS, and user for any use the end user wishes. Any attribute name without a namespace (i.e. without any colon in the name) is reserved for the end user.
Filing systems may use extended attributes to store information in if needed, and this may be exposed to the user. For example the filetype may be stored as an extended attribute named riscos:filetype.

File Locking

OS_Find gains an extra flag on file opening to allow files to be opened by more than one process at once. OS_Args gains calls to lock byte ranges of files.

Changes to SWIs

Unless otherwise stated, all new calls behave the same as the ones they are replacing.

OS_Args
Use of OS_Args 0 with r1 != 0 is deprecated. OS_Args 10 should be used instead.
Use of OS_Args 1 is deprecated. OS_Args 11 should be used instead.
Use of OS_Args 2 is deprecated. OS_Args 12 should be used instead.
Use of OS_Args 3 is deprecated. OS_Args 13 should be used instead.
Use of OS_Args 4 is deprecated. Filing systems should be able to sort this out by themselves.
Use of OS_Args 6 is deprecated. Filing systems should be able to sort this out by themselves.

OS_Args 10 Read 64bit file pointer
  => r0 = 10
     r1 = file handle (must not be 0)
   r0 = 11
     r1 = file handle
     r2 = lower 32bits of file ptr
     r3 = upper 32bits of file ptr

OS_Args 12 Read 64bit file extent
  => r0 = 12
     r1 = file handle
   r0 = 13
     r1 = file handle
     r2 = lower 32bits of file extent
     r3 = upper 32bits of file extent

OS_Args 14 Lock file
  => r0 = 14
     r1 = file handle
     r2 = flags
     r3 = start lo
     r4 = start hi
     r5 = length lo
     r6 = length hi
   r0 = 26
     r1 = filename
     r2 = flags
   r0 = 28
     r1 = filename
     r2 = flags
   r0 = 29
     r1 = filename
     r2 = flags
     r3 = attr name (null terminated)
     r4 = data
     r5 = data length
Flags are:
  bit 0 don't resolve symlinks
Will create if doesn't exist, or overwrite is does.

OS_File 30 Read extended attribute
  => r0 = 30
     r1 = filename
     r2 = flags
     r3 = attr name
     r4 = data buffer
     r5 = data length
   buffer length if overflowed)
Flags are:
  bit 0 don't resolve symlinks

OS_File 31 Delete extended attribute
  => r0 = 31
     r1 = filename
     r2 = flags
     r3 = attr name
   r0 = 13
     r1 = directory name (not wildcarded)
     r2 = ptr to buffer
     r3 = num objs to read
     r4 = start
     r5 = buffer length
     r6 = flags
  C flag corrupted
Buffer
  0  file type
  4  file length lo
  8  file length hi
  12 attributes
  16 object type
  20 time/date
  25 name (0 terminated)
Flags are:
  bit 0 don't resolve symlinks

Should we just require the user to call GBPB 9 and then OS_File to get info?
With cacheing it wouldn't be such a penalty as now.

OS_GBPB 14 write bytes to given 64bit pointer
  => r0 = 14
     r1 = file handle
     r2 = pointer to buffer
     r3 = bytes to write
     r4 = file position lo
     r5 = file position hi
     r6 = flags?
   r0 = 14
     r1 = file handle
     r2 = pointer to buffer
     r3 = bytes to write
     r4 = file position lo
     r5 = file position hi
     r6 = flags