I dispute this claim. The underlying artifact format for Fossil is compatible
back to the beginning. There have been enhancements, but nothing that would break. And there have been no reports of breakage among the countless users on the Fossil Forum.
I suspect what the OP encountered was that he checked in some things using a newer version of Fossil that had enhanced capabilities. (For example, Fossil originally only use SHA1 hashes, but was enhanced to support both SHA1 or SHA3 after the SHAttered attack.) Then the OP tried to extract using an older Fossil that didn't understand the new feature and returned an error. I'm guessing at this, of course, but that seems like the most likely scenario.
I have never once made a backup of SQLite repo or the Fossil self-hosting repo, or any of the other 100+ Fossil repositories that I have at hand. I've cloned the repos to other machines as disaster protection. In fact, I have cron jobs running on machines all over the world that "sync" critical Fossil repositories (such as SQLite) once an hour or so. But I have never even once made a pure backup.
I did have the primary SQLite repo go corrupt on my once, years ago. Somehow, file descriptor 2 got closed. Then when the SQLite database that is the repository was opened, it opened on file descriptor 2. Then some bug in Fossil caused an assert() to fire which wrote on file descriptor 2, overwriting part of the database. I restored the repo from a clone, fixed the assertion fault in Fossil, and enhanced SQLite so that it refuses to use a file descriptor less than 3.
Earlier in it's history, Fossil didn't have a separate cherry-pick command, but rather just a --cherrypick option to the "merge" command. See https://fossil-scm.org/home/help?cmd=merge. Perhaps that is where you got the idea that Fossil did not cherry-pick.
Fossil has always been able to cherry-pick. Furthermore, Fossil actually keeps track of cherry-picks. Git does not - there is no space in the Git file format to track cherry-picks merges. As a result, Fossil is able to show cherry-picks on the timeline graph. It shows cherry-pick merges as dashed lines, as opposed to solid lines for regular merges. For example the "branch-3.42" branch (https://sqlite.org/src/timeline?r=branch-3.42) consists of nothing but cherry-picks of bug fixes that have been checked into trunk since the 3.42.0 release.
The important point to keep in mind is that SQLite will read JSON5, but it never writes it. The JSON that SQLite generates is canonical JSON that is fully compliant with the original JSON spec.
It turns out that there is a lot of "JSON" data in the wild that is not pure and proper JSON, but instead includes some of the extensions of JSON5. The point of this enhancement is to enable SQLite to read and process most of that wild JSON.
This feature was requested by multiple important users of SQLite.
Off topic: would you mind sharing any info on potential timing of begin-concurrent-pnu-wal2 branch being merged into main (or consideration of forking sqlite to have a "client/server" version)?
While I don't care for JSON5 it's hardly "virtually any sequence of text", it rather hews closer to JavaScript content, rather than the very limited subset (ish) that is JSON.
Virtually any sequence is not the same as adopting a specification five years after it’s been published and adopted by industry: https://spec.json5.org/
I appreciate that SQLite can’t write the format, because those changes are human afordances
Anyone is allowed to contributed to the SQLite code base. There is no religious test, nor even any code-of-conducts requirements for being able to contribute to SQLite. This has always been the case. But the barrier to making contributions is high - higher than many other projects. There are two main reasons for this:
(1) Any contributions need to be able to demonstrate, with legal rigor, that they are in the public domain. Otherwise, if copyrighted code were introduced, SQLite itself would cease to be in the public domain. The SQLite project places a lot of emphasis on provenance of the code.
(2) Contributions need to demonstrate that they will be useful to a very wide audience, and that they will not diminish our ability to maintain the code for decades into the future. Most of the effort in a project like SQLite is long-term maintenance. People might be really proud of the work they have done on some patch over a day, or week, or month. But the amount of work needed to generate the patch is nothing compared to the amount of work they are asking the developers to put into testing, documenting, and maintaining that patch for the life of the project (currently projected to be 27 more years).
Many people, and even a few companies, have contributed code to SQLite over the years. I have legal documentation for all such contributions in the firesafe in my office. We are able to track every byte of the SQLite source code back to its original creator. The project has been and continues to be open to outside contributions, as long as those contributions meet high standards of provenance and maintainability.
The server is on Linode. Looking at the stats, they appeared to have had an
outage of some kind last night at about the time you got this message.
Everything seems to be running fine now. Please try again.
The CLI does use the relevant APIs, however I don't know of a way to reach this bug using a script input to the CLI. If there is such a path that I don't know about, it seems like it would require at least a 2GB input script.
This, and also there is the saying: "Never trust the frontend"
So in any case a string length check in the Backend should help here. Also if you get strings from a Browser Frontend, it seems unlikely you could create a payload of 2GB due to several reasons, one being string length limitations on the Browser side: https://stackoverflow.com/questions/34957890/javascript-stri...
There is a lot of static content on https://sqlite.org/ but also a lot of SQLite-backed dynamic content. I just checked the logs. Over the past 5 days, 12.03% of non-robot HTTP requests were against dynamically generated pages.
Fossil is hosted on the same machine as SQLite. Fossil is self-hosting and the Fossil website is 100% dynamically generated. Every HTTP request against https://fossil-scm.org/ does about 200 SQLite queries (give or take - depending on the page).
Fossil creator here: Fossil was created for one purpose - to support the development of SQLite, a job at which it has been successful beyond all expectation. Any other use of Fossil (and there is a lot of that, though still a lot less than there is for Git) is just gravy.
Fossil was designed to support the SQLite workflow. Git was designed to support the Linux Kernel workflow. Both systems seem amazingly well-suited for the projects for which they were designed.
Should you use Fossil or Git on your project? I suppose that depends a lot on whether your project workflow more closely matches the Linux Kernel or SQLite. There are other considerations, but I think that is the main differentiator of the two systems.
Side note: I've had a lot of help writing Fossil over the past 15 years, and especially a lot of help on the documentation. The article being discussed by this thread originated with me, but has been extensively edited by others with different ideas about the advantages and disadvantages of Fossil vs. Git. See https://fossil-scm.org/home/finfo/www/fossil-v-git.wiki?ubg for a complete timeline of changes to that one file, color-coded by committer. (BTW: Can Git or GitHub generate such a timeline?) My color is brown. If you scroll through the history, you can see that most of the edits to the article are by others. Which is fine. Just don't attribute everything you read there to me.
Thanks for your efforts! Both SQLite and Fossil are brilliant products. Fossil may not meet everyone's needs but its a great tool to be aware of as an option.
I suspect what the OP encountered was that he checked in some things using a newer version of Fossil that had enhanced capabilities. (For example, Fossil originally only use SHA1 hashes, but was enhanced to support both SHA1 or SHA3 after the SHAttered attack.) Then the OP tried to extract using an older Fossil that didn't understand the new feature and returned an error. I'm guessing at this, of course, but that seems like the most likely scenario.
I have never once made a backup of SQLite repo or the Fossil self-hosting repo, or any of the other 100+ Fossil repositories that I have at hand. I've cloned the repos to other machines as disaster protection. In fact, I have cron jobs running on machines all over the world that "sync" critical Fossil repositories (such as SQLite) once an hour or so. But I have never even once made a pure backup.
I did have the primary SQLite repo go corrupt on my once, years ago. Somehow, file descriptor 2 got closed. Then when the SQLite database that is the repository was opened, it opened on file descriptor 2. Then some bug in Fossil caused an assert() to fire which wrote on file descriptor 2, overwriting part of the database. I restored the repo from a clone, fixed the assertion fault in Fossil, and enhanced SQLite so that it refuses to use a file descriptor less than 3.
See also: https://fossil-scm.org/home/doc/trunk/www/selfcheck.wiki