TCP/IP Stage 2 bounty

Where’s GCC 8? NetSurf’s autobuilder is still using GCC 4.7 for RISC OS builds.

The latest I can see on the gccsdk site is gcc 4.7: https://www.riscos.info/index.php/GCCSDK_Releases

@Andrew : Timo – please drop me a line at the riscosdev.com email address => Done

Since it was announced by Andrew at the latest show that the port of the latest BSD source reached a state where someone just used !Netsurf with the port for the first time, why is the state of the Bounty still ‘Open’ instead of ‘Underway’ ?

Edit: I guess that’s because the donations didn’t reach the target yet.

I assumed that it was a ROD/Andrew organised effort, and so the bounty may not apply. I found Andrew’s talk about it most interesting, and was excited to hear that someone was working on it.

Maybe they will see how things pan out and throw some code (rather than cash) at the ROOL GitLab?

I guess that’s because the donations didn’t reach the target yet.

If I understood the bounty system correctly, that can never be the case. The “guide target” is just that: a value estimated by ROOL on how much effort the bounty might need.

If there is no developer interested in making the bounty happen, it will never change to “Underway”. If there is a developer interested in making the bounty happen even if it as only gathered 1 UKP of donations, it will change to “Underway”.

Andrew explained that the new TCP/IP stack he talked about was NOT funded via the ROOL Bounty system, but by a commercial customer.

Where’s GCC 8? NetSurf’s autobuilder is still using GCC 4.7 for RISC OS builds.

Newer GCC is in autobuilder. I think you need to build it.

Does this mean that we may see a bit newer compiler also? In other words if the Autobuilder is able to use GCC 8, how much other work is involved in getting it ported to run on RISC OS?

To be completely honest, I have never looked at what it takes to port GCC, where possible I use smaller compiler toolchains, and on RISC OS I currently use GCC because that is what we have other than the DDE, and I do not have a recent DDE. Though I may have to actually look at what it would take to get TCC going, good non-optimizing option for things that make sense not to need optimization.

Incidently, does this bounty include online documentation of all of the SWI interfaces (ie not C interfaces as in PRM 5a).

Or have I missed something? (again?)

Aren’t the SWIs basically just OS friendly wrappers around the standard C functions? A BASIC program can’t call listen() but it can call Socket_Listen()…

Aren’t the SWIs basically just OS friendly wrappers around the standard C functions?

Surely, it is the other way round. The SWIs came first. They are designed to be called by assembly language software, or even BASIC, both of which existed without C.

As I understand it, the socket stuff came from BSD and was originally in C, so the SWIs are indeed wrappers around C.

I stand corrected. Now I understand why this bounty exists.

In a number of topics, there have been requests, not just from me, asking for simple examples of socket software written in BASIC. There have no such simple examples given.

I suspect that this may well have been a major incentive for Justin to write his Easy Sockets module and for Roger to write SocketFS.

I have dusted off my copy of EasySockets and Aemulor and produced a simple example of an HTTP Client program.

SYS “XESocket_ConnectToHost”,host$,new_port% TO handle%

REPEAT

  < wimp poll >

  SYS “XESocket_CheckState”,handle% TO state%

UNTIL state%=4

SYS “XESocket_SendData”,handle%,request$,LEN request$

SYS “ESocket_ReadData”,handle%,reply_buffer%,200 TO ,,len%

SYS “XESocket_Forget”,handle%

Why do we still not have any other way to easily write socket software in Basic?

I have come to the conclusion that, at best, The Internet module SWI wrappers are very difficult to use; and, at worst, not really fit for purpose. Meaning that those of us trying to write socket software in BASIC have a very uphill struggle.

Why do we still not have any other way to easily write socket software in Basic?

Because using sockets isn’t easy. That’s not a feature of BASIC but of sockets.

It is possible to create a fairly simple app, if you accept that the operation will block whule waiting for a response from the socket. It’s also possible to use Easysockets to do some operations. However if you want a system where things communicate over TCPIP in a fully non-blocking fashion, and allow for broadcast messages (to estabish links for instance), link status checking etc, you will need to bite the bullet.

Understanding what needs to happen can be difficult. When I started doing this about ten years ago, Druck supplies me with a version of his socket library ported to BASIC, which was a great help, as it covered the lowest level bits. It’s still necessary however to uunderstand the steps needed to set up a link, to send messages across that link, and to check whether messages have been received (in non-blocking fashion) and read them.

Most of the work can be done in a null poll loop. If you also use Socketwatch, you can move this to a loop called from the pollword-changed return code. In practice I find the code for both is the same, so both returns call the same code. Socketwatch allows a faster response, otherwise you can only check every 10mSecs, which means a client-server dialog uses an average 10mSec for each message/response pair.

Detecting that a link has stopped working is an extra problem. If the far end client closes the link, the near end is notified. If the far end crashes, or the cable gets unplugged, there’s no indication until an attempt to send to it fails.

The only real complication due to the BASIC language is that you may be dealing with messages longer than 254 bytes, so using BASIC strings is not possible. Instead you need to use buffers, set up with DIM statements, and accessed by indirection. If you use TCP rather than UDP, then there is no guarantee that a message sent as one block of data will arrive in the same way. The packet may be split into two or more pieces, and you need to allow for this possibility. In my case, each message has a unique end character, so if received data doesn’t include it, I set up another read to get more and continue doing this until I have the end character. I use ring buffers, to avoid too much copying of data, but this does mean that a message may begin towards the end of a buffer, and wrap around so it ends near the start of the buffer. This produces edge cases that need careful testing, e.g. the message ends exactly at buffer end, or the only character wrapped is the final one, or the first character after the real message is an end character left from previous buffer contents.

Most of the above would also apply to C code, but the extra performance of a compiled language might mean some optimisations could be skipped. For example a message could be copied out of the input buffer, and the buffer reset to empty, eliminating the issues related to ring buffers. String processing in C might be possible where it is not in BASIC. (I’ve never written any C, so I don’t know what might be possible there.)

Yes, sockets are hard.

Not BASIC-related, nor even directly RISC OS-related, but for those people struggling with socket programming, may I recommend Beej’s Guide? I remember reading this aeons ago, when I was first learning network programming, and it remains, I think, an excellent tutorial on the hows and whys of socket programming. I’m pleased to see that it is still available, even updated.

https://beej.us/guide/bgnet/

I have come to the conclusion that, at best, The Internet module SWI wrappers are very difficult to use; and, at worst, not really fit for purpose.

Not really the internet module swis are just standard socket functions and the C library are just a wrapper to these swis. I think basic tends to get ignored because to learn to use sockets you need to read linux examples/tutorials and they are all in C so the easiest way to follow them is to use C on riscos which has a very similar socket library to linux. The problem with modules is that there is far more to a socket library than functions so half the information to use the module from basic is missing – you basically have to translate structures/constants from the C library. At this point having learnt sockets you give up on BASIC – you could translate back to BASIC but why bother.

I’ve not used EasySockets but generally find the problem with simplified libraries is lack of flexibility. You could just as easily have written a BASIC library using internet swis which does the same thing as your swis – once you now what you are doing :-). On the other hand I know how to use sockets and I would have questions about your code outline that I’d want answering before I used the module as you have shown – hopefully the documentation clears up my problems.

Sockets generally are an uphill struggle and it is much easier to rely on linux for documentation/tutorials/troubleshooting making C easier to use.

I wonder if Gerf would allow a 26/32 release of EasySockets?

When I started doing this about ten years ago, Druck supplies me with a version of his socket library ported to BASIC

I’m happy for anyone else to have it, if someone can suggest somewhere to put it (politely!)

I’m happy for anyone else to have it, if someone can suggest somewhere to put it (politely!)

I did have to make one correction that only shows up if the computer is using more than 32 sockets. There’s an offset of 1, expecting words, but interpreted as bytes, so should be 4. My errors in programming have on occasion taken the socket count up to the limit of 96. (Why 96 – I would have expected a power of 2, i.e. 128?)

I’m happy for anyone else to have it, if someone can suggest somewhere to put it (politely!)

May we have it sir? :)

It would be very nice to be able to download a copy.

Having a copy on Druck’s own web site – as well?

Has anyone done a ‘flow chart’ of internet connections?

[edit]

@Alan please can you let me have that correction.

@David
bq. please can you let me have that correction.

I’ve put the revised library here http://www.adamshome.org.uk/RiscOS/socketlib.zip

When I looked at the commented changed bit I couldn’t see what the original had been. Maybe using !SideDiff with your version would be the best way.

I changed max sockets to 128, not knowing at the time there is a system limit of 96.

@David and @Alan

Thank you for the library.

On the assumption that you are happy with comments here, so far I have PROCSocketLib called with DEBUG% previous set to zero.

With no further routines called, when exiting my task and calling PROCSocketLib_CleanUp, I get an error at line 173 because FNisvar does not appear to be avilable. Commenting the line out allows the Cleanup to complete.

Also File name ‘.LIBRARIES.HOSTS’ not recognised in “Libraries.SocketLib”

Sorry Folks,
This is no easier than calling the SWIs direct. I still need a simple working example.

@bryan Sorry if I’ve confused you. The link to the library was intended for David, so he could incorporate the bug fix into his library. (I don’t this forum supports private messages.)

My version, as you’ve discovered, has some other changes, although I had forgotten how many. FNisvar is in a different library, and is used to test if a variable exists before using it, so as to avoid “no such variable” errors. Hosts is a copy of the system hosts file, used to resolve names when the environment isn’t fully set up for name resolution. It may actually be redundant in my system now, so I need to see whether it can be removed.