RISC OS Open
A fast and easily customised operating system for ARM devices
ROOL
Home | News | Downloads | Bugs | Bounties | Forum | Documents | Photos | Contact us
Account
Forums → Announcements →

C library for easier HTTP(S)

Subscribe to C library for easier HTTP(S) 50 posts, 12 voices

Posts per page:

Pages: 1 2

 
Sep 5, 2018 7:20am
Avatar Chris Mahoney (1684) 1704 posts

The other day I mentioned that I was working on a library that made it easier to use the URL, AcornHTTP and AcornSSL modules. I said at the time that it wasn’t quite ready but that I’d share something in “a few days”. It’s been more than a few days so I figured that I should share what I have.

Download HTTPLib – Click the HTTPLib tab. This should be considered “alpha” quality. Source is linked from the same page.

Quick overview
Call httplib_get and pass in a URL, any desired headers (can be NULL), any desired request body (can be NULL), a function to call once the transfer is complete, and optionally a function for reporting download progress (can be NULL). For example:

httplib_get("https://server/path/file.txt", "Content-Type: application/json", "{ my-json }", downloadcomplete, progress);

void downloadcomplete(unsigned int status, char *content)
{
    if (status == 200) // OK
    {
        printf(content);
    }
}

void progress(unsigned int bytes_transferred, unsigned int total_size)
{
  printf("%d of %d bytes downloaded", bytes_transferred, total_size);
}

status will contain the HTTP status code if there is one, or 0 if it didn’t get that far (e.g. if you click “Reject” on an unverified certificate).

You will also need to add some code to your polling loop using httplib_isbusy and httplib_dowork so that everything can multitask. You will need to allow null events if HTTPLib is doing something:

while (!quit)
{
    if (httplib_isbusy())
    {
        event_set_mask(0);
        event_poll(&event_code, &poll_block, 0);
        httplib_dowork();
    }
}

Remember to set the mask back to something appropriate afterwards :)

Downloading files
You can use httplib_download to save a file instead of creating a char *. To do this:

httplib_download("https://server/path/file.zip", "local-filename", downloadcomplete);

The downloaded file will have its file type set based on its MIME type.

As of HTTPLib 0.06 you can cancel an in-progress download by calling httplib_stop.

Setting user agent
You can set a custom user agent via httplib_useragent. You must call httplib_useragent_free once you have finished your HTTP session.

Other functions
httplib_head does a HEAD request. Use it in exactly the same way as httplib_get.

Limitations

  • Only one transfer can be active at a time

Future plans

  • Fix the limitations!
  • Add support for caching (with an option to bypass)

Licence
HTTPLib is available under a slightly modified BSD licence. As of 0.02, a copy of the licence is available in the Zip (and no longer cluttering up this post!).

Upgrading to 0.09
All verbs now have header and body parameters. Pass in NULL to replicate the behaviour of previous versions.

I’ll post any updates in this thread. Hopefully someone finds this useful and easier than massaging the SWIs directly.

And almost forgot to say: I think you’ll need AcornHTTP 0.98 or newer. Earlier versions will probably not work. You’ll also need the URL module loaded, and you will of course need AcornSSL if you’re using HTTPS.

 
Sep 5, 2018 1:14pm
Avatar Rick Murray (539) 10813 posts

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS [blah blah]

Why does nobody seem to understand THAT A BIG WALL OF TEXT IN CAPITAL LETTERS IS HARDER TO READ? Imagine reading the licence aloud AND SHOUTING the large chunk at the end…

 
Sep 5, 2018 1:22pm
Avatar Steve Pampling (1551) 6783 posts

Imagine reading the licence aloud AND SHOUTING the large chunk at the end…

Imagine a user that emails the third line techs direct with BOLD UPPERCASE AND RED.
Imagine the same user having their emails drop into the junk mail folder(s) :)

 
Sep 6, 2018 12:36am
Avatar Chris Mahoney (1684) 1704 posts

Well, for 0.02 I’ll put the licence inside the Zip so you don’t need to be blasted with it (in this case I forgot to include it and I’d already copied the Zip onto my server). With that said, I think the licence file in my other apps is probably also full of caps because I just copied a template. I’ll make a note to change that…

 
Sep 6, 2018 7:17am
Avatar Steve Pampling (1551) 6783 posts

Well, for 0.02 I’ll put the licence inside the Zip so you don’t need to be blasted with it

I’m pretty sure Rick’s comment was a criticism of the person/people that wrote the original.
My comment was about one of a small set of users within the work domain and the effect her abuse had, which is probably not dissimilar to other wall of caps instances (ignore it)

 
Sep 6, 2018 7:57am
Avatar Colin Ferris (399) 1125 posts

With ref to the code – might be nice if it could be put into a module – so it could be used from BASIC etc.

Thanks for the work.

 
Sep 6, 2018 8:31am
Avatar Chris Mahoney (1684) 1704 posts

I’m pretty sure Rick’s comment was a criticism of the person/people that wrote the original.

I know, but still :)

As for work, we have someone who writes every single email subject line in all caps (but thankfully not the rest of the message!)

With ref to the code – might be nice if it could be put into a module – so it could be used from BASIC etc.

While that would be possible, I’m not sure how much benefit it would be. The URL module already exists, and HTTPLib is basically a simplified C interface into that. If there are specific features you’d find useful in BASIC then it might be more logical to add them to URL rather than have a module “wrapping” another module (which, in turn, wraps the AcornHTTP module!)

If you disagree then please give some examples of what you’d like to accomplish so I can understand better.

 
Sep 6, 2018 3:34pm
Avatar Clive Semmens (2335) 2268 posts

My impression was the whole purpose of C was to get everyone playing an endless game of Pass The Parcel…

 
Sep 6, 2018 4:57pm
Avatar Steffen Huber (91) 1687 posts

With ref to the code – might be nice if it could be put into a module – so it could be used from BASIC etc.

Sensible future RISC OS project number 87536: teach BASIC to load and call a user-mode library (ELF, ALF) so that we can stop putting standard application code into the RMA and excecute it with OS privileges.

I am sure Steve will be around in a minute and tell us that Basalt can do this already :-)

 
Sep 6, 2018 5:26pm
Avatar patric aristide (434) 427 posts

I am sure Steve will be around in a minute and tell us that Basalt can do this already

Lol!

 
Sep 6, 2018 6:03pm
Avatar Steve Drain (222) 1442 posts

I am sure Steve will be around in a minute and tell us that Basalt can do this already

Not yet. ;-)

I did play around with calling SharedCLib routines within Basalt, to enhance printing with printf for instance, but that was a long while ago.

I understand the point about privileged modes, but I still like the parsimony of shared routines called as SWIs.

 
Sep 6, 2018 6:32pm
Avatar Rick Murray (539) 10813 posts

I understand the point about privileged modes,

We all do, but this is one of the things we’re stuck with – unless somebody fancies having a crack at running user (and most system) modules in SYS mode . . . and dealing with the fallout of pretty much everything that assumes “modules run in SVC mode”.

but I still like the parsimony of shared routines called as SWIs.

Indeed. While SWIs have efficiency issues (recalling the SWI number from something that should be in the code cache, not the data cache etc), there is a benefit to a set of routines that can be called via a standardised interface from just about any language worth knowing the name of1.
One could argue that the Desktop is exactly that – a set of routines with a SWI interface.
And Sockets.
And FileSwitch/FileCore.
And………

1 Obviously I am referring to the subset of languages available on RISC OS, not languages in general everywhere (and especially not the vagaries of human languages!)…

 
Sep 14, 2018 9:09pm
Avatar Chris Mahoney (1684) 1704 posts

HTTPLib 0.02 is now available, and the first post has been updated accordingly. But in summary:

  • Handles failure better (better ≠ well!)
  • Added feature to download files instead of save to char *
  • Removed the user agent parameter from the various functions; a future version will offer a “global” way of setting this.
 
Sep 23, 2018 7:50am
Avatar Chris Mahoney (1684) 1704 posts

HTTPLib 0.03 is now available. Changes in this version:

  • Fixed a malloc bug, which means that httplib_useragent (which was present in 0.02 but not documented) now works and is supported
  • Downloaded files will now be typed according to their MIME type, instead of being hardcoded to Zip.
 
Oct 8, 2018 6:11am
Avatar Chris Mahoney (1684) 1704 posts

HTTPLib 0.04 is now available. This one fixes an embarrassing bug which was intermittently corrupting downloaded files (because some numpty – me – was calling the wrong SWI…) and adds an httplib_head function for doing a HEAD request. There is still an issue with some servers not “liking” HTTPLib, which I’ll hopefully get to the bottom of in 0.05.

 
Nov 23, 2018 9:50pm
Avatar Chris Mahoney (1684) 1704 posts

Version 0.05 is now available (see first post). It supports the rest of the HTTP methods, and is also “better” because it doesn’t have the same code repeated all through it anymore :)

The issue I mentioned in the previous post appears to be a problem with AcornHTTP rather than HTTPLib. I haven’t had a chance to investigate it properly yet.

 
Nov 25, 2018 10:07pm
Avatar John Sandgrounder (1650) 574 posts

The URL module already exists,

where? How do we use it?

 
Nov 26, 2018 4:00am
Avatar Chris Mahoney (1684) 1704 posts

It’s included in the nightly HD4 image and the docs are here.

 
Nov 26, 2018 2:16pm
Avatar John Sandgrounder (1650) 574 posts

Thank you.

 
Jan 8, 2019 7:20pm
Avatar Chris Mahoney (1684) 1704 posts

HTTPLib has been updated to version 0.06 (link in first post).

  • Implemented “progress” function so you can see how far through a download you are.
  • Added httplib_stop function to abort an in-progress download.
  • No longer uses a fixed 1024-byte block size. It can now download in blocks of up to 1 MB, resulting in a significant performance boost on fast connections.
 
Mar 8, 2019 3:41am
Avatar Chris Mahoney (1684) 1704 posts

0.07 released. This version has some minor reliability improvements and is now known to work when called from a module.

 
Oct 12, 2019 6:31pm
Avatar Sergey Lentsov (8268) 28 posts

The link from first message return 404 error.
Can anyone to send a correct link or upload the sources on Google drive or something?

 
Oct 12, 2019 7:09pm
Avatar nemo (145) 2136 posts

This link seems to be working.

 
Oct 12, 2019 7:54pm
Avatar Chris Mahoney (1684) 1704 posts

Apologies! I moved the site yesterday and completely forgot that I had direct links to some of the Zips. Links in first post fixed, although I’ve broken nemo’s link in the process :)

 
May 1, 2020 11:13am
Avatar Michael Gerbracht (180) 104 posts

Does anybody have a C or C++ example app that uses HTTPLib? I can’t get it to work although I tried to implement it as shown in the initial post. But I am new to C so the problem is very likely me ;-)

Next page

Pages: 1 2

Reply

To post replies, please first log in.

Forums → Announcements →

Search forums

Social

Follow us on and

ROOL Store

Buy RISC OS Open merchandise here, including SD cards for Raspberry Pi and more.

Donate! Why?

Help ROOL make things happen – please consider donating!

RISC OS IPR

RISC OS is an Open Source operating system owned by RISC OS Developments Ltd and licensed primarily under the Apache 2.0 license.

Description

Announce and discuss new hardware and software releases.

Voices

  • Chris Mahoney (1684)
  • Rick Murray (539)
  • Steve Pampling (1551)
  • Colin Ferris (399)
  • Clive Semmens (2335)
  • Steffen Huber (91)
  • patric aristide (434)
  • Steve Drain (222)
  • John Sandgrounder (1650)
  • Sergey Lentsov (8268)
  • nemo (145)
  • Michael Gerbracht (180)

Options

  • Forums
  • Login
Site design © RISC OS Open Limited 2018 except where indicated
The RISC OS Open Beast theme is based on Beast's default layout

Valid XHTML 1.0  |  Valid CSS

Powered by Beast © 2006 Josh Goebel and Rick Olson
This site runs on Rails

Hosted by Arachsys