rendertiddlers command doesn't work correctly on wiki files ending in .htm

[dullroar]

The following is an issue even with an empty TiddlyWiki template file downloaded from tiddlywiki.com. I have been testing it on Windows 8.1 using TW 5.0.13beta and node v0.10.29.

If the TW file is saved with a suffix of .htm (such as mylife.htm) then running the following command (adapted from this post - #168) produces incorrect output:

tiddlywiki --load mylife.htm --verbose --rendertiddler $:/core/templates/static.template.html static.html text/plain --rendertiddler $:/core/templates/static.template.css static/static.css text/plain --rendertiddlers [!is[system]] $:/core/templates/static.tiddler.html static text/plain

The output that is produced is this:

C:\Temp>dir /s output
 Volume in drive C is Windows8_OS
 Volume Serial Number is DCCD-A5B5

 Directory of C:\Temp\output

07/24/2014  08:45 AM    <DIR>          .
07/24/2014  08:45 AM    <DIR>          ..
07/24/2014  08:45 AM    <DIR>          static
07/24/2014  08:45 AM            14,021 static.html
               1 File(s)         14,021 bytes

 Directory of C:\Temp\output\static

07/24/2014  08:45 AM    <DIR>          .
07/24/2014  08:45 AM    <DIR>          ..
07/24/2014  08:45 AM         1,311,344 mylife.htm.html
               1 File(s)      1,311,344 bytes

     Total Files Listed:
               2 File(s)      1,325,365 bytes
               5 Dir(s)  53,141,127,168 bytes free

In other words, the entire TW is reproduced in a single mylife.htm.html file.

If I change the file name to mylife.html (simply adding an "l" to the file suffix), and change the --load command to read in mylife.html, then the output produced is structured as I would expect, like this:

C:\Temp>dir /s output
 Volume in drive C is Windows8_OS
 Volume Serial Number is DCCD-A5B5

 Directory of C:\Temp\output

07/24/2014  08:35 AM    <DIR>          .
07/24/2014  08:35 AM    <DIR>          ..
07/24/2014  08:36 AM    <DIR>          static
07/24/2014  08:35 AM            28,885 static.html
               1 File(s)         28,885 bytes

 Directory of C:\Temp\output\static

07/24/2014  08:36 AM    <DIR>          .
07/24/2014  08:36 AM    <DIR>          ..
07/24/2014  08:35 AM             4,797 %40done.html
07/24/2014  08:35 AM             4,797 %40todo.html
07/24/2014  08:35 AM             5,146 .NET%20Bookmarks.html
07/24/2014  08:35 AM             5,474 A%20few%20thoughts.html
07/24/2014  08:35 AM             5,536 ARM%20Architecture%20Bookmarks.html
07/24/2014  08:35 AM             6,176 Bookmarks.html
07/24/2014  08:35 AM             5,766 Buena%20Vista%20Bookmarks.html
07/24/2014  08:35 AM             6,684 Computers%20and%20Programming%20Bookmarks.html
07/24/2014  08:35 AM             6,070 Cooking%20Bookmarks.html
07/24/2014  08:35 AM             5,257 Deep%20Learning%20Bookmarks.html
07/24/2014  08:35 AM             5,109 Diet%20Bookmarks.html
07/24/2014  08:35 AM             5,346 Education%20Bookmarks.html
07/24/2014  08:35 AM             5,436 Entertainment%20Bookmarks.html
07/24/2014  08:35 AM             5,352 Exercise%202014-07-16.html
07/24/2014  08:35 AM             5,105 Exercise%20Bookmarks.html
07/24/2014  08:35 AM             9,129 favicon.ico.html
07/24/2014  08:35 AM             5,240 Gardening%20Bookmarks.html
07/24/2014  08:35 AM             5,390 Government%20Bookmarks.html
07/24/2014  08:35 AM             5,211 Health%20Bookmarks.html
07/24/2014  08:35 AM             5,575 Hobbies%20and%20Home%20Bookmarks.html
07/24/2014  08:35 AM             4,820 Home.html
07/24/2014  08:35 AM             4,985 House%20Maintenance.html
07/24/2014  08:35 AM             6,207 Humor%20Bookmarks.html
07/24/2014  08:35 AM             4,844 Jim.html
07/24/2014  08:35 AM             5,268 Linux%20Bookmarks.html
07/24/2014  08:35 AM             5,141 Maps%20Bookmarks.html
07/24/2014  08:35 AM             5,122 Math%20Bookmarks.html
07/24/2014  08:35 AM             5,123 Meditation%20Bookmarks.html
07/24/2014  08:35 AM             5,283 Mobile%20Bookmarks.html
07/24/2014  08:35 AM             5,526 MOOCs%20Bookmarks.html
07/24/2014  08:35 AM             5,103 Movies%20Bookmarks.html
07/24/2014  08:35 AM             5,821 Music%20Bookmarks.html
07/24/2014  08:35 AM             5,120 News%20and%20Politics%20Bookmarks.html
07/24/2014  08:35 AM             5,216 Node.js%20Install%20Notes%20on%20Windows.html
07/24/2014  08:35 AM             5,289 OCaml%20Bookmarks.html
07/24/2014  08:35 AM             5,147 Operating%20Systems%20Bookmarks.html
07/24/2014  08:35 AM             5,130 Photography%20Bookmarks.html
07/24/2014  08:35 AM             5,146 Physics%20Bookmarks.html
07/24/2014  08:35 AM             5,160 Programming%20Bookmarks.html
07/24/2014  08:35 AM             5,169 Projects%20Bookmarks.html
07/24/2014  08:35 AM             6,050 Python%20Bookmarks.html
07/24/2014  08:35 AM             5,863 Raspberry%20Pi%20Bookmarks.html
07/24/2014  08:35 AM             6,270 Reading%20Bookmarks.html
07/24/2014  08:35 AM             5,266 Religion%20and%20Philosophy%20Bookmarks.html
07/24/2014  08:35 AM             5,802 Religious%20Teachings%20Bookmarks.html
07/24/2014  08:36 AM             5,094 School%20Bookmarks.html
07/24/2014  08:36 AM             5,547 Science%20and%20Nature%20Bookmarks.html
07/24/2014  08:36 AM             7,377 Security%20Bookmarks.html
07/24/2014  08:36 AM             5,412 Self%20Improvement%20Bookmarks.html
07/24/2014  08:36 AM             5,371 Source%20Control%20Bookmarks.html
07/24/2014  08:36 AM             5,134 Space%20Bookmarks.html
07/24/2014  08:36 AM             4,803 Stuff%20To%20Do.html
07/24/2014  08:36 AM             5,219 TiddlyWiki%20Install%20Notes.html
07/24/2014  08:36 AM             5,117 Vacation%20Bookmarks.html
07/24/2014  08:36 AM             6,736 Weather%20Bookmarks.html
07/24/2014  08:36 AM             7,817 Web%20Programming%20Bookmarks.html
07/24/2014  08:36 AM             5,336 Windows%20Bookmarks.html
              57 File(s)        315,430 bytes

     Total Files Listed:
              58 File(s)        344,315 bytes
               5 Dir(s)  53,139,976,192 bytes free

Each non-system tiddler is now broken out as its own HTML file under the static directory as I would expect.

I have looked in the rendertiddlers source but not too deeply yet, so at this point I am filing the bug (at least _I_ think it is a bug) to get it documented before delving into determining the cause.

[Jermolene]

Hi @dullroar thanks for the report.

I'm afraid it is as simple as the file extension mechanism currently only recognising a full .html extension. It's not a one-line fix because the current code incorporates the assumption that there will be a one-to-one mapping between file extensions and content types. Fixing this ticket requires us to support content types that have more than one associated file extension.

[dullroar]

That's a bummer. Just spent some time hacking around in the boot tasks and couldn't get these hacks to work:

1. During the registration of file types adding the following right after registering .html, using the optional charset parameter on the MIME type to disambiguate from the default text/html:

$tw.utils.registerFileType("text/html;charset=utf-8","utf8",".htm",{deserializerType:"text/html"});

1. Right after all the registerFileType calls trying to just hack the fileExtensionInfo list directly:

$tw.config.fileExtensionInfo[".htm"] = {type: "text/html"};

1. --savetiddlers causes an error #1 and actually none of the command line examples seem to work #2 combined.

Ah, well, looks like I will have to study the code harder, when I am not at work. :)

[Jermolene]

Ha! It turns out that there is a quick workaround, see c738532

[Jermolene]

Oops sorry @dullroar I just noticed that your second suggestion is the same as what I'd done - it works for me, does it not work for you?

[dullroar]

It is not, @Jermolene (and I just compared your code and mine side-by-side, which was identical, and then even copied and pasted your code in to make sure - same results). Would it be because I am hacking a downloaded 5.0.13beta .htm file directly and not doing some sort of build process and hence perhaps missing some other changes that is allowing it to work for you?

Either way, I feel happy that I was on the right track!

[Jermolene]

Hi @dullroar yes I'd recommend building the source as described in http://tiddlywiki.com/#Working%20with%20the%20TiddlyWiki5%20repository

Anyhow, I've posted a prerelease to http://tw5test.tiddlyspot.com/ for you to check.

[dullroar]

I was able to build the source and confirm it now works. Thanks. I am closing this issue.