P2P Distribution of Tarballs

We’re considering using P2P distribution for our source and binary tarballs in order to reduce overall bandwidth requirements for this site. However, there are a couple of problems we have:

1. selecting a P2P network/client
2. getting the thing set up at our end

So first of all, can anyone suggest a P2P client which will work on RISC OS, Unix, Linux, Mac and Windows? If you can, then what is involved from our end in getting a tarball onto the P2P network and what is involved from the user end in downloading it (note: we will need to document this, particularly from a RISC OS user’s point of view on our site).

If we don’t have a suitable client available already, does anyone out there have a suggested client which could be ported to RISC OS without too much effort?

All comments welcome!

There is only one choice: BitTorrent. You don’t want to involve a “network”, as you run the risk of people downloading non-official versions, possibly with malware. With BitTorrent, they download a small .torrent file from this site, and this acts as a signature so they know they’ve got something you’ve signed off. It’s also the most reliable and open of the systems. It does mean that you’ll have to run a tracker on your co-lo box, but this is really simple if it’s only really supporting a handful of files.

There’s a client for almost every platform – and if there isn’t one for RISC OS, porting one shouldn’t be too difficult.

1. User selects item to download from your site
2. Rather than giving them the actual file, you give them a .torrent file, which contains where the “tracker” is (ie, how the client discovers other users), and checksums/hashes for chunks of the file, so it can verify it’s got the real thing.
3. Either their web browser is integrated to their BitTorrent client somehow and the download just happens as if they’d clicked on a normal download link, or they’re prompted to save the torrent file.
4. If they’re prompted to save the torrent file, they do so, and feed it to their BitTorrent client, which then connects to the tracker, connects to the various peers it tells them about, and starts swapping pieces until it has the whole file downloaded.

In any case, surely this will only become a problem when there is a sizeable amount of the OS to release, rather than just what’s in Batch 1, etc?

So are you saying that Gnutella isn’t an option (seeing as there is a Gnutella client available for RISC OS, although I don’t know how current that is)?

If rogue versions of our tarballs are a problem, we could always sign them with gpg or publish the md5 hash on our web site, etc. Although that’s a bit technical for some, it’s probably not beyond the people who care about access to RISC OS sources.

Does anybody fancy porting a BitTorrent client to RISC OS? ;)

And look whats apeard :)

http://www.riscos.info/pipermail/gcc/2007-March/003773.html

Nothing to do with me, guvnor :)

Actually I did it ages ago, but only got around to testing it last night (and hadn’t seen this thread). It’s not hugely complex, so should be possible to bolt a frontend to, if anyone’s so inclined.

Great work. Thanks Theo! :)

I’ll be doing some work on the ROOL site this week, hopefully, though I’ve got a few other more urgent tasks to clear first. My efforts will be directed towards updating site components to newer versions of the respective supporting applications. However, if I can squeeze in some time to generate (ideally automatically) torrent files for the TGZ archives, then I’ll do so.

I can recommend BitTornado as a BitTorrent tracker, I use it on my co-location boxes.

I’ve not forgotten about BitTorrent for .tar.gz files BTW, it’s still there in amongst the other urgent things.