TCP/IP bounty beta release

How many of those pre-3.5 versions had a working TCP/IP implementation as part of the OS?

There was “Universal Boot” for RISC OS 3.1x, which included the Internet module version 5, so pretty much up to date. As well as e.g. PPP, URL/HTTP fetcher module and AcornSSL, because of course !Browse needed it and Acorn seemed to be keen to provide “modern Internet access” to RISC OS 3.1 machines – after all, Toolbox and nested WIMP and even Java all ran on 3.1.

About the only thing I could imagine where a HTTPS fetcher would be useful would be for a 3.1 port of PackMan to provide an easy way for 3.1 machines (inlcuding potentially emulators) to fetch software. Well, someone would need to build a RISC OS 3.1 software distribution then, the only thing currently available is probably ADFFS along with its Game JFDs.

There was “Universal Boot” for RISC OS 3.1x, which included the Internet module version 5, so pretty much up to date.

I interpreted “part of the OS” to mean actually built in. A lot of stuff can be loaded afterwards (as the nested Wimp) but everything like that takes precious memory…

Acorn seemed to be keen to provide “modern Internet access” to RISC OS 3.1 machines

Of course, in 1997/1998 there would be rather more RISC OS 3.10 machines in use than there would be twenty years down the line.

and even Java all ran on 3.1.

Are you sure about that? I seem to recall Java wanting an 8MB WimpSlot before it would even start up.

after all, Toolbox and nested WIMP

There was also a lot of controversy over the Toolbox, again due to perceived bloat and memory restrictions. It was becoming quite clear that the 4MB which was mind-blowing in 1987 was titchy in 1997… and think about the sorts of sites that existed back then versus those which exist today.

Google’s main page – a 238K HTML document (that Firefox’s Page Info incorrectly reports as 67.75K), supported by four script files with gibberish names totalling 792K, a 4K Privacy Shield icon that’s as laughably small as the shield itself is useless. Oh, and a nice Christmas animation adding up to 169K.
So that’s ~1204K just to load the empty looking Google entry page. Or if we accept that JavaScript on older machines will be rubbish (even Fresco will be way out of date), that’s still 411K… On a machine where you might have something like 600K free after loading the stuff to get you online in the first place…maybe less. There’s a reason I wrote QuickVoy – internet on an A5000 was…cramped.

If someone was real keen to run it on ro3.11 – what about the module ‘DummyDynamicAreas’ by J. Fletcher v1.23 (23 Dec 1998).

Merry Xmas all – and a Healthy New Year.

If someone was real keen to run it on ro3.11 – what about the module ‘DummyDynamicAreas’ by J. Fletcher v1.23 (23 Dec 1998).

I’m not at all sure you can download Justins stuff from anywhere¹, plus as he says himself in his Rambles

“The error handling for the module was very poor – which was fine with me because this was intentionally hacky, just to make something that I could use with SpriteExtend.”
and in the JPEGSprite section.
“The downside to this was that you had to use DDA, which could be a little flakey when memory got low.”

Nice idea though.

¹ I think some moron was abusive enough to him² to cause him to obliterate as much of his online content as he could recall.
² When people speak of “the RISC OS wars” they, and we, probably don’t know the half of it. Some nasty stuff and casualties.

I’m not at all sure you can download Justins stuff from anywhere

Should have checked the Wayback machine before I posted because a good chunk is still in existence there for anyone with a nostalgic turn.

Given the origin of our nice, new shiny SSL provision and peoples comments about using the appropriate socket this release note is interesting reading don’t you think?

“Secure sockets
With secure sockets, your Mbed OS applications can securely connect and send data to any cloud or server through any TCP-based protocol. By introducing native secure sockets to Mbed OS we enable TLS functionality in the background, simplifying secure connections. For more information, please visit secure sockets.”

For more information

quote
// First talk with the server without encryption
connection.send(“STARTTLS\r\n”, 10);
unquote
I have seen this a few times now, what gets sent unencrypted when using STARTTLS?
It seems to vary from server to server but for smtp.mail.yahoo.com
port 465 works if I have (openssl) STARTTLS disabled otherwise port 587 can work.
I came across a Windows implementation via a search claiming that 587 was an insecure connection also which got me interested.

what gets sent unencrypted when using STARTTLS?

For POP3 and SMTP, the unencrypted bit is the server greeting and capacity listing (POP3 response to the CAPA command or the SMTP EHLO response). The client then knows the server can handle secure connections and the handshake is started after the client sends STLS (POP3) or STARTTLS (SMTP).
This is over port 110/587 (POP3/SMTP – sometimes SMTP port 25 as well). Ports 995/465 (POP3/SMTP) require secure connections straight away – no STLS/STARTTLS.

There was a time using port 587 for SMTP was considered the way to go and port 465 was ‘decommissioned’. However, ‘implicit TLS’ is now preferred again (see RFC 8314 – https://tools.ietf.org/html/rfc8314), so port 465 has been ‘reinstated’.

However, ‘implicit TLS’ is now preferred again

Been hunting and looking at the results, a wiki described port 25 and 587 as a normal port and 465 as an ssl port.
The result appears to be the same, apart from the extra lines

[<-] 250 STARTTLS
[->] STARTTLS
[<-] 220 2.0.0 Ready to start TLS

Edit: The ports I have been using select implicit TLS OK but the !SMTPS that Alexander has was STARTTLS only.
Almost as simple as adding

if (port!=465) { 
existing starttls section
}

Hoorah! Our final beta (release 6) is now available. Changes are as below and unless there’s anything shocking found, we’re going to mark this one as closed and roll it into our ROM builds…

• Updated to use the latest mbedTLS 2.14.1 from Arm, specifically to fix
  CVE-2018-19608
• Further update to use the latest mbedTLS 2.16.0 from Arm
• Now attempts to start the module task itself when RMLoad’ed, documentation updated to that effect too
• The Send and Recv SWIs now propagate MSG_WAITALL/MSG_DONTWAIT flags on a per call basis
• When a socket lacks buffer space or data, AcornSSL backs off rather than hammering the Internet module

Sounds like we’re on the home stretch! I’ll give this a go when I get home, although I’m not doing anything strenuous with it :)

Congratulations to everyone involved with getting this far.

Edit: Confirmed working with my app, and no Filer_Run hackery needed!

Sorry to dredge this topic back up, but have just had a problem report from a RiscPC (StrongARM) user. Has anyone tested AcornSSL “properly” (ie. doing multiple fetches etc) on a RiscPC class machine? The user is reporting lots of timeouts and lost connections.

We’ve already had to increase a lot of timeouts anyway, as AcornSSL seems to take a lot longer at various points to negotiate or return data. However, it is (give or take) fine on my machines, both VRPC 4.39 and ARMX6. Of course, these are both relatively nippy compared to a RiscPC!

Before I “dig deep” and fund yet more programmer time on tracking this down, I’d appreciate any feedback from people running AcornSSL on RiscPCs as to success/failure/timeouts/speed. My gut is saying that perhaps the RiscPC simply lacks the grunt to do the job, but on the flip side, there are much “weedier” devices out there doing SSL transfers, and they can’t all be using out of date SSL!

Hi Andrew,

I can’t do any testing until Sunday but if you can wait then will check the AcornSSL module out on a Kinetic enabled RiscPC.

Do you know what processor the person who is having issues has in their machine and what OS?

I believe it was an ordinary SA but it may have had 4.39 OS.

Hi Andrew

Well I have given the latest beta I have a whirl, it includes the a fixed version of Hermes 54903, and it took two stabs at downloading 1581 emails on one of my accounts.

This is on a Kinetic RiscPC with 136Mb Ram, including 2MB VRam, and using a Unipod network/IDEFS and VPod graphics adaptor.

It gave a connection lost error but I’ve also seen this on my ARMX6 on occasions as well when doing large numbers or simultaneous account downloads so I don’t think it is specific to older hardware.

It could equally be the email server disconnecting if a download is taking a while.

Anyway I’ll leaving it running for the rest of the day and let you know if I see any other issues.

OK so done some more testing and made 4 other email accounts live on the RiscPC and there is a definite increase in the number of lost connections on trying to transfer even when an account has no emails to fetch.

I also got a System heap error but think that was as a result of a particular large download trying to debatch and failing.

So seems there is something there as an issue.

Andrew,

Detailed log of email session that resulted in a lost connection message.

I have blanked off certain elements for security

Hope this helps

Doug

Connecting to NB***** Blocked for security
Receiving response: +OK POP3 PROXY server ready
Sending command: CAPA
Logging in as NB ******(Blanked user)
Receiving response: +OK Capability list follows
Receiving response: TOP
Receiving response: RESP-CODES
Receiving response: USER
Receiving response: SASL PLAIN
Receiving response: PIPELINING
Receiving response: UIDL
Receiving response: .
Receiving response: +OK Password required
Sending command: PASS ********
Receiving response: +OK Maildrop ready, (JPOP server ready).
Sending command: STAT
Receiving response: +OK 0 0
Sending command: QUIT
Receiving response: +OK Server signing off.
Lost connection to NB * Blocked for security

My POP3 is rusty, but shouldn’t it send a USER xxxxxx command? Did you edit that line out?

Hi Rick

The only editing I have done, apart from removing time stamps, is the first line as I’ve blocked out the mail service I’m connecting to.

Logging in as line, just after the CAPA command, where I have placed NB blanked out user in place of actual user details.

And the final line where I have again blanked out the mail service.

I have checked a good logging in and the only thing different is after the server signing off command at the end I get 3 additional lines/responses:

Connected to NB **** blanked out mail service
Logged in as NB **** blanked out user
Disconnecting from NB **** blanked out mail service

Again edited to remove service and user details.

Edit:

OK so got the same Connection lost error on another user and again the difference between a good mail connection is the last three lines after receiving the +OK Server signing off response.

Thanks for this.

Do you have any other software that uses the module? I’d like to know if these timeouts are due to AcornSSL being demanding (ie. slow) or Hermes issues. The fact they seem more common on slow machines (coupled with no issues for non-SSL accounts) makes me think that AcornSSL is just abnormally slow. It could be a quirk of the module, exasperated by increasing number of connections and slower CPUs. Or, it could be that Hermes isn’t waiting long enough for responses.

However, ROOL probably won’t appreciate me saying that without proof (quite reasonably), so other applications that use the module (eg. RiscOSM?) would be useful data points.

It could be a quirk of the module, exasperated by increasing number of connections and slower CPUs.

:)
Exacerbated maybe, unless of course the increasing number is irritating, annoying and frustrating the module.

More usefully, how well do both the elements do on the co-operative element of CMT? i.e. Is there a CPU time availability issue? Not releasing control of the wimp frequently enough.
The NIC driver on the machine could be the root cause – a lot of the Acorn era NIC’s were 10Mb and half duplex, yet still faster than the users ISP connection, but internet connections have improved a lot and those NIC’s are still around with 20 year old driver software.

To test that kind of theory you’d need a simple SSL capable server instance and a RiscPC matching the problem machine sitting on a decent switch to throw files around locally and measure throughput (also do wireshark capture looking for re-transmits)

However, ROOL probably won’t appreciate me saying that without proof

Well, if it’s an underlying issue with the behaviour of the NIC driver or similar then the user problem being on a non-ROOL derived OS might have a bearing on their view but if it’s at NIC level then finding and fixing would benefit all irrespective of code branch.

Back to non-work related stuff Steve, you’re on leave. Keep saying it…

Do you have any other software that uses the module? I’d like to know if these timeouts are due to AcornSSL being demanding (ie. slow) or Hermes issues.

The new Making Tax Digital extension to Prophet uses AcornSSL for its data transfer – for both the initial OAuth log in, and subsequent JSON data exchanges.

This has been extensively tested on a StrongARM Risc PC with RISC OS 4.02 (as well as a Titanium, naturally, and the interview with HMRC was done using a Raspberry Pi screen sharing over VNC). At no point have any timeouts been seen, probably between 70-100 transactions were needed in testing.

It must be said though that the amount of data for a VAT return is relatively small, of the order of 1k.

It must be said though that the amount of data for a VAT return is relatively small, of the order of 1k.

Which as you suggest isn’t likely to stress things, particularly buffers.
Really the testing needs to be:

1. Within local switches so that external delays and packet drops are eliminated
2. File transfers of two sizes – many small files(1-2k) and one larger (40-50MB)
3. All connections captured – either a tap or a port mirror¹² feeding a capture device running wireshark.
   Dropped packets or timeouts should be quickly apparent.

¹ That’s SPAN for the Cisco droids.
² A managed switch with the facility helps on the mirror. Not sure how many people have that.

Do you have any other software that uses the module?

I have RiscOSM and Recce and in the case of the latter even on the ARMX6 it is a bit hit and miss but I’ve put that down to it being version 1.00.

The NIC driver on the machine could be the root cause – a lot of the Acorn era NIC’s were 10Mb and half duplex

Well the one RiscPC has a Unipod plus 10Mb NIC which has v4.59 of the EtherH driver so I could test using both to see if there is any difference.

All connections captured – either a tap or a port mirror feeding a capture device running wireshark.

Well long , long time since I did anything like that and no managed switch here so I’ll have to pass on that but I agree it is the way to go on testing this.

The only issue I have at the moment is time and therefore I will not have a big enough window on that until at least Friday or the weekend to do just to some testing.

I have RiscOSM and Recce and in the case of the latter even on the ARMX6 it is a bit hit and miss but I’ve put that down to it being version 1.00.

Sorry, I’ve not been keeping an eye on the forum the last week.

If you could describe how Recce is “hit and miss” that would be helpful, because it may or may not be a problem I know about. I am aware of a difficulty with the AcornSSL module version 1.04. We distributed 1.03 with Recce but you may have 1.04 and I think that’s what Andrew is distributing with Hermes. The problem Recce is having won’t affect software that uses AcornSSL direct — it only affects fetching https content via the URL_Fetcher module. It seems that in some circumstances the contents is returned corrupted. In Recce this means pictures from Google StreetView come out slightly odd, and there may be an error about a failure to read licence descriptions for Flickr.

I have reported the problem to the developers.