BASIC allows redimensioning of a local array inside a FOR loop
Stuart Painting (5389) 709 posts |
The IOMD build of BASIC V allows a local array – i.e. one declared as LOCAL a%() – to be redimensioned, provided that the action takes place inside a FOR loop. Sample code:
This will fail on the third pass. Remove the LOCAL a%() line and the program will fail on the second pass. RPCEmu 0.9.3a for Mac, RISC OS 5.24 (BASIC 1.75) and RISC OS 5.29 (BASIC 1.84). The BCM2835 build does not exhibit this bug. (EDIT: Yes it does. See next post.) For the discussion on Stardot that prompted his post, see https://stardot.org.uk/forums/viewtopic.php?f=29&t=23021 |
Stuart Painting (5389) 709 posts |
I’ve worked out why I was getting different results on the Pi: I had retyped the program rather than copy it from machine to machine, and on the Pi I had used “NEXT i%” instead of “NEXT”. Change the line to “NEXT” and it fails on the third pass. That merely prompts a follow-up question: why does “NEXT i%” fail in this case? |
Martin Avison (27) 1479 posts |
The failure when i%=3 is after… Which is just as Richard suggested in his initial Stardot post. If the DIM is moved to just before the FOR, it runs without errors, which seems a more logical place to put it. The failure when i%=1 and NEXT i% is used is because the NEXT cannot find the expected FOR entry on the stack as the array gets in the way. As tested using |
Stuart Painting (5389) 709 posts |
It is possible to cause a one-off memory leak in slightly less contrived circumstances.
This will fail at the PRINT a%(1) line, illustrating that the array is no longer accessible. Inspecting the value of END at various points in the program will show that the BASIC heap increases substantially during the second call to PROCsetup, and the memory is not released when PROClocal exits. The memory leak is one-off (you only lose more memory if you create another array in the same fashion) so this bug may fall into the “not worth fixing” category. Tested using BASIC 1.81 on RISC OS 5.28 (Raspberry Pi 4). |
Steve Drain (222) 1620 posts |
I think this is an equivalent memory leak to: PROClocal PRINT a% END DEF PROClocal LOCAL a% PROCsetup PROCsetup ENDPROC DEF PROCsetup DIM a% 1000 ENDPROC where the pointers to the dimmed blocks are lost. This is more obvious, and I think it comes into the category of a programmer’s error. |
Martin Avison (27) 1479 posts |
Because the LOCAL version of the array from the first call of PROCsetup (which was on the stack) was removed when the stack was reduced at the end of PROCsetup.
Because the second PROCsetup allocated a normal array (not local) so went in the heap, and is there for evermore. It just serves to show that re-allocating arrays has to be done with care, and I agree with Steve that it comes into the category of a programmer’s error. Memory or Arrays allocated to LOCAL veriables vanishes at the end of the PROC they are allocated in when the stack is reduced. |
Stuart Painting (5389) 709 posts |
Sure, but BASIC isn’t exactly helping here, is it? It’s obvious in the stated examples what is going wrong, but in a real-life situation the programmer could spend a lot of time debugging before discovering that the array had been dimensioned twice. At the very least we need a modification to the BBC BASIC Reference Manual. Perhaps an admonition of the form “The corresponding DIM statement must appear in the same procedure/function as the LOCAL statement, and there should be as few intervening structures between the two statements as possible”. |
Julie Stamp (8365) 467 posts |
I had a look with *ReportTrace/Mem and saw this happening, but I still don’t understand why. I can see that the allocation from DIM in PROCsetup is let go from the stack at ENDPROC (setup), but I would have thought there’s something still on the stack frame for PROClocal ‘representing’ the LOCAL a%(). If we change it to
I’d’ve expected it to say array cannot be redimensioned, rather than heap allocating the DIM. How is it that PROCsetup’s ENDPROC causes the PROClocal stack frame to forget about the LOCAL a%()? |
Steve Drain (222) 1620 posts |
I am not sure where I got this from, but my StrongHelp BASIC manual has had this for a couple of decades: “However, if this is done in the So it has been know about for a long time. ;-) Martin explains it. |
Steve Drain (222) 1620 posts |
The manual says: “When you declare a local arrary, the LOCAL statement must be followed by a DIM statement to dimension the local array.” Is that not sufficient. Perhaps emphasis on “must be followed” is required. The scope of normal local variables is any routine deeper than the LOCAL statement. However, arrays are referenced and really do need to be passed as parameters. Do that in the example and all is well. Is this clear enough in the BASIC manual? Probably not. |
Martin Avison (27) 1479 posts |
If this case is intended, rather that a bug, then I agree that the BASIC manual needs clarification about the restriction.
Because a%() is defined as LOCAL in PROClocal, so is contained in the PROClocal stack. The array a%() is DIMmed in PROCsetup, so is contained in the PROCsetup stack. So at the PROCsetup ENDPROC, the stack containing the array is discarded, but a%() is still defined as LOCAL – although it appears now unusable for a LOCAL array to be defined. My theory is… If after the first PROCsetup, if the ‘l-value’ of a%() is set back to 1 (with great care!), the second DIM a%() allocates the array data on the stack, which is what I would expect. I have so far been unable to find where in the source the ‘l-value’ of a%() is reset on ENDPROC, or why there is a difference between one and zero (most checks seem to be <16). |
Steve Drain (222) 1620 posts |
I think I would, too. What documentation there is about how arrays work is under CALL, which is perhaps not the most obvious place to look. I have managed to get some strange problems like this in other circumstances, when re-using an array variable, eg a%(), at different points in a program. They probably have the same cause. The problems go away when arrays are passed as parameters. Martin is doing a sterling job tracking this down, so I will leave it to him. ;-)
That is what I would expect, as it is documented. I wonder if there is another sanity check that the array pointer is in the heap, ie <FSA, and if not, it is also treated as undimensioned. This would be so for a local array that has been released at ENDPROC. |
Clive Semmens (2335) 3274 posts |
But this doesn’t cover every use case. It’s perfectly reasonable to have a LOCAL array that can be different sizes on each call to the function or procedure in which it exists, for example. Which causes no problems at all as long as the array disappears when the function or procedure returns. |
Steve Drain (222) 1620 posts |
Of course, but that says nothing about using such an array at deeper levels than where it was made local. |
Clive Semmens (2335) 3274 posts |
Using them at deeper levels is no problem, as long as you don’t go and redimension one you’ve already dimensioned at whatever level. Even dimensioning them at lower levels is okay, as long as you only do it once. What I’m failing to understand is why it’s not a bug if BASIC fails to tell you you’re trying to redimension one. |
Martin Avison (27) 1479 posts |
I think I have found the cause of some of these issues, and I may have a fix. Here is a summary of my understanding of the processing involved… When an array is DIMmed, array storage is allocated for a list of sizes of dimensions, terminated by 0, followed by the total number of elements, then the values of each element. This storage is normally in the Heap, but for LOCAL arrays will be in the Stack. The array name and it’s l-value are allocated on the Heap, and the l-value is the address of the array storage. If an array variable is made LOCAL in a PROC (or FN), the previous l-value is stored (for restore when the PROC ends), and the l-value is set to one, indicating undimensioned. When an array is DIMmed and the variable name already exists, it is an error – unless it is a Local array. If l-value is one (undimensioned) then array storage is allocated on the Stack, the l-value set to point to it, and the array is added to a list of LOCAL arrays. When the PROC ends, stack entries for the PROC are discarded, including the LOCAL array data. When this happens, the list of Local arrays is checked, and any discarded have their l-values reset to “undim it” … but it is set to zero not one. Thus, if the array is DIMmed again while the variable is still LOCAL, it is allocated on the Heap and NOT the stack (because the l-value is zero not one). This I believe is a bug. That may appear to ‘work’, but is a memory leak, and if the array is DIMmed again while still LOCAL it will fail because the (global) array already exists on the Heap. This explains the error seen in the program in the first post in this thread. My possible fix is to set the l-value back to one when an array is lost from the stack, and the original test case runs without errors. However, nothing in the BASIC code is simple and straightforward (to me, anyway). There are other places which change an array l-value, sometimes to zero, and I am unsure what other effects there may be. More checking is required. |