ResCommon Documentation
-----------------------

	The ResCommon utility has been altered for the Bethany project so
that one ROM image can contain more than one resource set, with the user
being able to select which resource set they wish to be able to use.
	This document outlines the changes that were made to the build system
to allow this to work. 

	These changes go hand-in-hand with changes to the ScanRes utility to
allow more than one resource block to be registered in the build.

	On boot-up the Messages module reads the current territory setting
and scans through all the available ResourceFS blocks for any containing that
territory number. These are then used as the available resources.

Fitting more than one resource set into a build
-----------------------------------------------
	During initial specification work for the Bethany project, we
calculated that a typical build had about 1.5 MB of resource data in it.
Since we had about 2.5 MB free after the UK, it was obvious that without some
form of compression we would only be able to fit two resource sets into a
build.
	It was also noted that there is a vast amount of commonality between
different the resource sets of different territories; only translated
resources change. We calculated that if files that are common between all
resource sets were placed in a 'common' reosurce block, then we could fit
three territories into the build and still have 0.5 MB of ROM space free.
	Unfortunately, it later became necessary to have five resource sets in
one ROM image. This forced a change in the scheme so that *any* commonality
between files in the resource sets are detected and placed in common
directories.
	This section outlines the scheme that was implemented to achieve this
goal. In the end we managed to fit Australia, France, Ireland, Italy and UK
resources in one ROM image and still have nearly 500K free ROM space.

        When comparing a number of files, there are (2^n)-1 possible
combinations that need to be checked. For instance for for 5 files there are
31 possible combinations, whulst for 3 there are only 8 combinations.
	These comparisons need to be performed for every byte of every file.
Obviously this would be very time-consuming, so an alogrithm that reduce the
number of comparisons that need to be made has been written.
	For the purposes of describing the process I will state that there
are 5 files to be compared; the same process holds true for any number from 1
to 5.
	The system works by having a series of lists (implemented as arrays
for speed) of identical files. If all files are identical, then there is one
list holding 5 files. If they are all different, then there are 5 lists each
holding one file.
	The process starts with all files being flagged as being identical.
The file contents are then read byte-by-byte, and the following process
occurs.
	All the lists are scanned, and the contents of this particular byte
compared. If any are different, then they get put into a free list, whlst if
they are the same they remain in the existing list.
	Because of this mechanism, the only comparisons occuring are those
where it knows that files are already identical.
	Once it is known that all files are different (i.e. there are five
lists, each with one file in it) then the process can be aborted and the next
file checked.

	If there is one difference in the list, then that goes into a list on
its own and gets ignored for the rest of the process. If there is more than
one file with identical changes, then the list gets split accordingly.

	Below there is an ASCII diagram showing how the process may work with
five files.

  File:       1 2 3 4 5
  Byte 0:     A A A A A    Lists  12345
       1:     B B C A A    Lists  12  3  45
       2:     B B C A A    Lists  12  3  45
       3:     A D C B B    Lists  1   2   3   45
       4:     A D C B A    Lists  1   2   3   4   5
       We do not need to continue in this file, as all lists are different.

  or:
  Byte 0:    A A A A A   Lists    12345         
       1:    C C E E E   Lists    12  345           (31 comparisons)
       2:    A C D F F   Lists    1   2   3   45    (1+7 comparisons)
       3:    D F E G G   Lists    1   2   3   45    (1 comparisons)
       4:    D F E G G   Lists    1   2   3   45    (1 comparisons)
       5:    D F E G G   Lists    1   2   3   45    (1 comparisons)
     <EOF>
     This means that all files are different, apart from 4 and 5 that are
identical.

	
Processed directory organisation
--------------------------------
	The files being processed by ResCommon are held in the 'Resources'
directory of the Messages module, and the processed files are placed into the
'Processed' directory.
	When a file has been processed, the lists are parsed through and the
files relating to that particular list are placed into the relevant
directory. If the directory does not exist, then it is created.
	For instance, if a file is common between UK and Germany, then it
gets placed into the 0107 directory (01 and 07 being the territory numbers
for the UK and Germany respectively).
	If a file is common between all territories in the territory list
then it gets placed into a directory called 'common'. This means that it will
be used in all resource sets.

Squash files
------------
	Because Squash file headers contain information about the time they
were compressed, the same file compressed at different times has different
header data, and hence would be registered as being different under the
scheme shown above. To remove this problem, if the file is of type Squash
then it ignores all the header bytes, and only compares the body data.


Limitations
-----------
	*) Because the system saves resources as a list of territories in
numeric form. RISC OS allows a maximum of 99 territories, and so a 2-digit
number can be used to represent any individual territory. Only five resource
sets can be included in a ROM on an old filecore disk (the ten character
filename limitation means that only 10/2 = 5 territories can be in a
filename). 
	*) This number is further reduced if one of the resource sets does
not include UK. Because ROM initialisation expects a UK resource set to be
present after a delete-poweron, if the territory list does not include UK we
make the first territory also include UK. For instance if the first territory
is Germany (07), then the directory will appear as 0701. Because we still
have to fit within the 10-character filename limitation, this means that
builds without UK resources can only have four different resource sets.
	*) Another limitation is the hard-coded value of MAX_TERRITORIES in
ResCommon and ScanRes. If we wish to have more than 10 territories in a
build, this number will have to be increased.
	*) All files must be present in all resource sets. For instance, you
cannot have a file present in the UK but not present in the French. However,
they can be empty files.

