Updated port of SQLite
Chris Mahoney (1684) 2162 posts |
There was some discussion last month around SQLite and getting it updated. I’ve taken the old port of SQLite 3.3.6 created by The Really Small Software Company and have updated it to the latest version of SQLite (3.42). It’s built with the DDE and should work back to RISC OS 3.1. “Beta 8” is available here or via PackMan. Source is here (you may wish to read the BuildNotes file). Changed in beta 8
Changed in beta 7
Changed in beta 6
Changed in beta 5
Changed in beta 4
Changed in beta 3
Changed in beta 2
Changed in beta 1
Fixed in alpha 13
Fixed in alpha 12
Changed in alpha 11
Changed in alpha 10
Changed in alpha 9
Fixed in alpha 8
Changed in alpha 7
Changed in alpha 6
Fixed in alpha 5
Fixed in alpha 4
Changed in alpha 3
Fixed in alpha 2
Notes
Please report any bugs in this thread. There are certain to be some I haven’t found! Thanks to Really Small for the original port; I probably wouldn’t have got this far without that as a baseline. |
Andrew Rawnsley (492) 1440 posts |
IIRC the main issues we had with latest sqlite in Iris (it uses it for various local storage things) related to re-enterancy and handling of multiple instances. I guess for non-ported apps (generally DDE’s strength) it wouldn’t matter, but is there anything here that might help with such things? |
Chris Mahoney (1684) 2162 posts |
It’s a pretty basic port at the moment, so doesn’t have much in the way of handling RISC OSisms. The SQLite docs actually advise against having multiple instances so I haven’t done anything special to handle them. Presumably Iris is using UnixLib and the “Unix version” of SQLite. Functionally it’s probably fairly similar to this new one. |
Chris Mahoney (1684) 2162 posts |
Alpha 2 is now available, complete with source code. The first post has been updated with details. |
Paolo Fabio Zaino (28) 1821 posts |
very nice job @ Chris! :) |
Chris Mahoney (1684) 2162 posts |
As a little interim update, here is an up-to-date version of the command-line shell. This is shell 3.35.5, as opposed to the older version included with alpha 2. The underlying SQLite library is the same as alpha 2. Next on the agenda for the main library itself is to get the automated tests working. I’ve run some of them manually and none of them have failed, but I haven’t yet automated them. There are a lot, so automation is a necessary step! |
Chris Mahoney (1684) 2162 posts |
Alpha 3 is now available (see first post). This is based on SQLite 3.36, and released on the same day as the non-RISC OS version :) |
Alan Robertson (52) 420 posts |
You’re on fire. Nice work. |
Chris Mahoney (1684) 2162 posts |
I haven’t had much free time over the past couple of months, but today I finally got this up and running. Unfortunately there are some failed tests, so this library shall retain the “alpha” tag for now. I’m not sure when I’ll have time to fix up the issues, but will post any updates in this thread. |
Chris Mahoney (1684) 2162 posts |
I fixed1 an issue and a number of the failed tests now pass. Unfortunately it didn’t fix all of them, so I’ll have to delve deeper at some point. The sorts of things that are failing are edge cases with SQL that nobody in their right mind would actually write:
Nevertheless, I want to get all the tests working for the confidence that everything behaves correctly. 1 It’s really a workaround rather than a fix; I had to disable CSE optimisation to get it to work. I’m not sure whether it’s something non-standard in the code doing it, or whether it’s an issue with the compiler. I simply noticed that it worked fine2 in DDT but not when running a ‘real’ build. 2 And by “fine” I mean “the file’s so large that DDT starts truncating line numbers once you pass line 99999 and then crashes on quit”… |
Rick Murray (539) 13756 posts |
In my experience, that is often something using a value that it expects to have a default “unset” value of null, which isn’t the case in the real build. For example, if you have: int x; int y; Then at runtime, until I always declare as: int x = 0; int y = 0; even if three lines down I set it to be something different. Always begin with a known state, or relying upon any sort of state that you can’t guarantee will eventually give you headaches. |
David J. Ruck (33) 1591 posts |
FTFY: When built for debug, they will be zeroed. On platforms where you can run debug builds without a debugger, this catches out a lot of people when they finally make a release build. At least with Norcroft, having to run under DDT ensures no one ever builds for debug if they don’t need to. |
Chris Mahoney (1684) 2162 posts |
The problematic code is (or rather, appears to be) part of SQLite itself and not my ‘interface’ code, so I don’t want to make any changes to it. The last thing I want to do is introduce strange new bugs :) |
Chris Mahoney (1684) 2162 posts |
I finally stopped procrastinating and had a look into this. First of all, here’s the actual failing SQL test case:
This is supposed to return nothing, but instead returns -544.4. The underlying issue is that SQLite calls Even more interestingly, if I separate the code off into its own file (nantest.c) then it works as expected! Can anyone shed any light on this? I don’t really want to modify the core sqlite3.c file if I can help it, so I’d love to understand what’s going on here. |
Sprow (202) 1150 posts |
The reason the smaller/faster version doesn’t work is because the order of words in an FPA I’m also not convinced when you say
as -280366 looks pretty number-y, and not at all nan-y to me. |
Chris Mahoney (1684) 2162 posts |
You’re quite right! Post updated to match reality :) Thanks for the explanation. |
Chris Mahoney (1684) 2162 posts |
Upon further investigation, I’ve had to add a new entry to the Known Issues list.
Hopefully this will be something I can fix, but it probably won’t be any time soon. I expect to release Alpha 4 shortly, which makes use of |
Chris Mahoney (1684) 2162 posts |
Alpha 4 is now available. This is a milestone release, as SQL Logic Test now passes in its entirety. I forgot to update the BuildNotes text file in the source archive: sqlite3-in is documented as being an untouched copy from SQLite 3.36. This is not correct, as it contains the isnan() fix mentioned above. This fix should be included in SQLite 3.37 so a hand-edited file should no longer be required in the future. |
Chris Mahoney (1684) 2162 posts |
It ended up being trivial. So, hot on the heels of alpha 4 comes alpha 5. Details in the first post. Please note that this version will not correctly read databases created with alpha 4. If you need to retain existing data then you should open your database with alpha 4, |
Chris Mahoney (1684) 2162 posts |
While some simple actions (creating a DB, adding data, reading it back) are successful under 3.1, some of the test scripts are failing under ArcEm when using HostFS. I don’t yet know whether this is a SQLite issue, or an ArcEm issue. I also tried using ADFS under ArcEm but this crashed. The SQLite tests all pass under OS 3.6, but at this stage I’m not ‘certifying’ the port on earlier OS versions. I tried running under Arculator on Windows 10 but couldn’t figure out how to get the mouse working. There don’t seem to be any settings for it and Google didn’t return anything useful! I can see the mouse pointer, but it moves in steps of ~64 pixels and therefore I can’t click anything on the icon bar as the pointer just skips right past it. Has anyone else run into this? |
Chris Mahoney (1684) 2162 posts |
I can now confirm that this is an ArcEm/HostFS issue. I’ve only run a limited number of tests under 3.1 but it’s now returning correct results when using ADFS :) |
Paolo Fabio Zaino (28) 1821 posts |
Nice! BTW Chris, would it be ok if I upload both the HTTPLib and the SQLite ports on the RISC OS Community on GitHub? I am working on create a cross platform build (that runs on RISC OS too) to build code that also pulls down dependencies. Given that I am using your HTTPLib and the SQLite port for a couple of projects it would be nice to have the source there (mostly handy). Thanks for this effort BTW, really cool to see SQLite working on RISC OS 3.1 as well! |
Chris Mahoney (1684) 2162 posts |
That’s fine, but please put a note on there making it clear that that’s not their official home. One of these days I need to finish off HTTPLib… |
Paolo Fabio Zaino (28) 1821 posts |
thanks and of course I’ll make sure it’s displayed that those are only mirror repositories, thanks a lot! :) |
Chris Mahoney (1684) 2162 posts |
Alpha 6 of SQLite for RISC OS is now available from the usual place, based on SQLite 3.37. There are no RISC OS-specific changes. If you’re building from source, please note that the “amalgamate” utility has been updated, and that the new version is required when using 3.37. It’s also compatible with 3.36. |