The TiddlyWiki Loader
This is a small collection of things to help with public TiddlyWikis. The idea is to prevent so much re-downloading of the whole wiki and to show progress to the reader---rather than just having the browser sit there while it loads.
Using the Loader (index.html)
So the first piece is the index.html---which shows progress of loading the wiki and then replaces itself with the wiki. There is no redirection, so the URL won't change.
To use this, put the
index.html in a directory, next to your TiddyWiki---which
must be named
┣━ index.html ┗━ wiki.html
NOTE: There might be a delay of several seconds while the wiki is loaded from memory. But it also seems that this happens when the file is loaded directly from the network---just be advised that there is probably still some work to do getting this to go faster.
Reduce Bandwidth with Deltas (kicksnap)
kicksnap script copies your wiki and the
index.html to a directory. If
that directory exists already, then some patching files are generated in the
same directory---these will be used by the loader above to patch the wiki file.
NOTE: You cannot just delete the directory generated by this script! It must
be preserved so that subsequent runs of
kicksnap will update the patch files.
To run it, supply the path to your wiki and the path to the output directory.
$ ./kicksnap ~/Downloads/MyWiki.html /var/www/html
Once the script starts generating patches, the directory will look like this:
┣━ index.html ┣━ wiki.html ┣━ wiki.html.diff ┣━ wiki.html.json ┗━ wiki.html.snap
wiki.html is a copy of your full wiki, if you find that you need to
This delta strategy has the nice effect of making it very simple for us to generate archives of the wiki. So... that's built into the tool as well.
Just add the
archive command at the end:
$ ./kicksnap ~/Downloads/MyWiki.html /var/www/html archive
This will generate an
archives directory that will contain numbered archives.
wiki.html.lst file will also be created, containing a simple directory
┣━ archives/ ┣━ index.html ┣━ wiki.html ┗━ wiki.html.lst
The presence of these files will enable access to the archive list in the
index.html. You can add "?archives" to the URL to the index.html and the
archive list will appear.
Other Attempted Strategies
You might be wondering if there are other ways to do this---I have tried a few other ways and just want to mention them before you offer them up as suggestions for improving this setup.
Using Split Files
My first strategy was to split up the wiki file into anywhere from 20 to 200
separate files (split on line endings) and then using a diff to update only
these 'chunks'. You can find the script
kicksplit.py in the commit history of
The trouble with this strategy is that it was:
- Not efficient: Daily diffs of a wiki are generally only 20k-100k. However, if twenty (of 200) chunks get touched (generally the case) then we are looking at 2 megs of download. By the end of a month, things would get up to 8 MB for a 20 MB wiki---so things got pointless really fast.
- Overly complicated: Deleted lines would cause some chunks to disappear---I had to start rebalancing the data as time would pass, meaning that at least 10% of chunks were getting updated even with an increasingly complex algorithm.
Using Multiple Diffs
Another strategy was to add diffs for each time the file is changed. This wasn't a bad approach---but once the files began to build up, they became double the size of a single diff anyway. It wasn't worth the extra complexity.
Okay, Nothing Further
Except that there is a bit further.
Here is an article discussing my reasons behind this project: