Added mechanism for treating .txt file as .opml, with documentation; …

…released as 1.0.2.

The core of this change is the :treatasopml directive, which is the signal in a .txt file to convert the body of the page to an Opml object. This change involved, first, a modification to Opml.new (initialize), since I had foolishly assumed that the parameter would always be a pathname string; whereas now it must be either a Pathname (to a file to be read, consisting of OPML) or a string (consisting of OPML). Second, a class method Opml.textToOpml was added, to convert indented text (very elegantly!) to OPML. In PageMaker, runOutlineDirectives no longer converts adrObject to a string before passing it along to Opml.new, since the fact that it is a Pathname is now the sign that it indicates a file to be read. And, finally, in PageMaker, in runDirectives, having read the scalar directives from the start of a .txt file, we immediately look to see whether :treatasopml is true and, if so, convert the rest of the page to OPML (using Opml.textToOpml) and hand it off to Opml.new to get an Opml object just as would be the case if this were a .opml file; notice that we also set :treatasopml to false to prevent the same thing happening again later when template directives are read (skanky but simple).

RubyFrontier.tmbundle/HISTORY

@@ -1,7 +1,7 @@
 VERSION
 =======
 
-This is version 1.0.1.
+This is version 1.0.2.
 
 HISTORY
 =======
@@ -40,4 +40,5 @@ In version 1.0, cleaned up cruft in the CSS processing processing routines, espe
 
 In version 1.0.1, expanded the user.rb mechanism to allow inclusion of a #user.rb file at the top level of the site folder; also improved the error message when a user.rb file fails to load. Documentation updated.
 
+In version 1.0.2, introduced a mechanism to allow a `.txt` file to function as an outline (like a `.opml`) file, by using indentation and setting the `:treatasopml` directive to `true`. Documentation updated.
 

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney.rb

@@ -2,7 +2,7 @@
 # utilities: myrequire, myraise, Memoizable, and various modifications to existing classes
 require 'longestJourneyUtilities.rb'
 
-myrequire "pathname", "yaml", "erb", "pp", "uri", "rubygems", "exifr", "enumerator", "kramdown", "haml", "sass"
+myrequire "pathname", "yaml", "erb", "pp", "uri", "rubygems", "exifr", "enumerator", "kramdown", "haml", "sass", "nokogiri"
 
 =begin make 'load' and 'require' include folder next to, and with same name as, this file 
 that is where supplementary files go

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/docs/RubyFrontierDocumentation/defaultfolder/techfolder/directiveobjects.txt

@@ -50,7 +50,7 @@ As a starting point, we'll use <%= xref "FIGsourcefolder", :fignum, true %>. Not
 
   * If a file in a `#tools` folder is a `.txt` file, it is a *snippet*. A [snippet](snippets) is a named stretch of text: the name is the name of the file (minus the `.txt` suffix), the text is the contents of the file. As the Web page is [built](howsapage), the text will be substituted for the phrase `[[snippetName]]` in your page object. This provides a convenient mechanism for inserting the same text in multiple places in your Web site. (Frontier users: this is not a Frontier feature, but it is based on one use of the Frontier glossary feature.)
 
-  * If a file in a `#tools` folder is a `.rb` file defining a subclass of `UserLand::Renderers::SuperRenderer`, it is an *outline renderer*. If the page object to be rendered is an outline (`.opml`), it needs a [renderer](outlinerenderers) to transform it into text. The renderer to be used is specified by the `:renderoutlinewith` directive. For example, if you have set the `:renderoutlinewith` directive to `"mycoolrenderer"`, RubyFrontier will look for a class `Mycoolrenderer` which is a subclass of `UserLand::Renderers::SuperRenderer`, and the first place it will look for the definition of such a class is in a file in a `#tools` folder.
+  * If a file in a `#tools` folder is a `.rb` file defining a subclass of `UserLand::Renderers::SuperRenderer`, it is an *outline renderer*. If the page object to be rendered is an outline, it needs a [renderer](outlinerenderers) to transform it into text. The renderer to be used is specified by the `:renderoutlinewith` directive. For example, if you have set the `:renderoutlinewith` directive to `"mycoolrenderer"`, RubyFrontier will look for a class `Mycoolrenderer` which is a subclass of `UserLand::Renderers::SuperRenderer`, and the first place it will look for the definition of such a class is in a file in a `#tools` folder.
 
   * If a file in a `#tools` folder is a `.rb` file and is *not* an outline renderer, it is a *macro script*, meaning that it contains a method that can by called by name from a [macro](macros) (essentially a use of [ERB](ERB) in your page object or template). A macro script is expected to define one or more top-level methods, which you can call in a page object or template. For example, if your page object or template says `<%%=homelink()%>`, RubyFrontier will look for and call a top-level `homelink()` method in a macro script. It is usual for the name of the macro script file to match the top-level method, just to make your source folder easy to maintain — the file in the `#tools` folder would be called `homelink.rb` — but at present this is not strictly required.
 

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/docs/RubyFrontierDocumentation/defaultfolder/techfolder/howsapage.txt

@@ -38,7 +38,7 @@ This is a lot of information, and it may seem at first like too much to take in.
 
     * If the page object is an `.rb` script file, we **run the script**. In particular, the script is expected to define a top-level method, `render()`, taking one parameter; that method is called, supplying one parameter — the `UserLand::Html::PageMaker` object that's rendering the page. The script is thus in a very powerful position. It can obtain the page table (for example, if the `PageMaker` object is called `pm`, the page table is `pm.adrPageTable`) and can do whatever it wishes. The result of the script becomes the new value of `:bodytext`.
 
-    * If the page is an `.opml` file, it consists of an *outline* (expressed as XML); you must *designate and supply* an **[outline renderer](outlinerenderers) to transform it into text**. To *designate* the outline renderer, you must, prior to this moment, have defined the `:renderoutlinewith` [scalar directive](scalardirectives); its value must be the string name of the outline renderer class (which, as for any class name, must begin with a capital letter). To *supply* the outline renderer, you will usually have placed an `.rb` file in a `#tools` folder; or, if you want an outline renderer to be available to all your Web sites, you can keep it in [the `user.rb` file](user). The renderer's `render()` method is called, as described [here](outlinerenderers), using [macro scoping](macros); the result of this call becomes the new value of `:bodytext`.
+    * If the page is an `.opml` file, it consists of an *outline* expressed as XML; alternatively, it might be a `.txt` file with the `:treatasopml` [scalar directive](scalardirectives) set to `true`, in which case RubyFrontier has translated the text to XML for you, based on indentations. Either way, you must *designate and supply* an **[outline renderer](outlinerenderers) to transform it into text**. To *designate* the outline renderer, you must, prior to this moment, have defined the `:renderoutlinewith` [scalar directive](scalardirectives); its value must be the string name of the outline renderer class (which, as for any class name, must begin with a capital letter). To *supply* the outline renderer, you will usually have placed an `.rb` file in a `#tools` folder; or, if you want an outline renderer to be available to all your Web sites, you can keep it in [the `user.rb` file](user). The renderer's `render()` method is called, as described [here](outlinerenderers), using [macro scoping](macros); the result of this call becomes the new value of `:bodytext`.
 
 1. Information about the page object is **added to the [autoglossary](autoglossary)**. This is so that other pages in the site (and in other sites) can link easily to this page. In particular, the page object is keyed in the autoglossary under (1) its simple filename (the name of the page object file, minus the extension) and (2) the `:title` if there is one (and at this point, if this page object is a renderable, there should be one).
 

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/docs/RubyFrontierDocumentation/defaultfolder/techfolder/howsapagefolder/outlinerenderers.txt

@@ -1,6 +1,10 @@
 #title "Outline Renderers"
 
-A cool feature of Frontier is that it's an [outliner](http://www.outliners.com/). As part of the Frontier Web framework, therefore, it is possible for a [page object](types) to be an outline. RubyFrontier, on the other hand, is *not* an outliner (chiefly because RubyFrontier is not, itself, a GUI — it's just a script — and [TextMate](http://macromates.com/) is not an outliner). Nonetheless, an outliner is a very good editing tool, and many of my own Frontier page objects were outlines. In order to keep this feature in RubyFrontier, therefore, the following mechanism is used. A page object is allowed to be an `.opml` file — OPML is a form of XML, describing an outline. You can't really edit such a file with TextMate directly (well, you can, but you probably shouldn't, since it defeats the point, and anyhow it is all too easy to generate bad XML that way); instead, you are expected to edit an outline page object using an outliner application that can read and save OPML. My choice for this purpose is [OmniOutliner](http://www.omnigroup.com/applications/omnioutliner/).
+A cool feature of Frontier is that it's an [outliner](http://www.outliners.com/). As part of the Frontier Web framework, therefore, it is possible for a [page object](types) to be an outline. RubyFrontier, on the other hand, is *not* an outliner (chiefly because RubyFrontier is not, itself, a GUI — it's just a script — and [TextMate](http://macromates.com/) is not an outliner). Nonetheless, an outliner is a very good editing tool, and many of my own Frontier page objects were outlines. In order to keep this feature in RubyFrontier, therefore, and thus to provide compatibility for those converting their Frontier Web sites to RubyFrontier, the following mechanism is used:
+
+* A page object is allowed to be an `.opml` file — OPML is a form of XML, describing an outline. You can't really edit such a file with TextMate directly (well, you can, but you probably shouldn't, since it defeats the point, and anyhow it is all too easy to generate bad XML that way); instead, you are expected to edit an outline page object using an outliner application that can read and save OPML. My choice for this purpose is [Opal](http://a-sharp.com/opal/).
+
+* Alternatively, a page object that is a text file (a `.txt` file) can effectively be an outline, indicating the outline hierarchy by indentation (i.e. the number of spaces at the start of each paragraph). In this case, to let RubyFrontier know that this is an outline, the page must use the `:treatasopml` [scalar directive](scalardirectives), setting it to `true`. RubyFrontier will convert the body of the page to OPML. This format has the disadvantage that TextMate is not an outliner, beyond the crude ability to increase or decrease the indentation level of selected lines, but it has the advantage that you can do your editing entirely in TextMate itself.
 
 Okay, so let's say you decide to do this. Then you will also need an outline renderer. An **outline renderer** is a Ruby script that transforms an outline into text. Every outline page object must have a corresponding outline renderer, which will be called upon at the [appropriate moment](howsapage) during the rendering process; RubyFrontier will apply the outline renderer to the contents of the `:bodytext` entry of the [page table](pagetable), turning it from outline to text.
 
@@ -62,7 +66,7 @@ When using the `Opml` instance methods, there is always a "current line" of the
 
 > NOTE: These are the only Frontier `op` verbs I have implemented because they are the only ones my renderers need. Others can be implemented in future if required.
 
-> ANOTHER NOTE: The `Opml` class is designed to be implemented either using the built-in Ruby library `REXML` or the Ruby gem `libxml`. To set which of the two is used, edit the value of the boolean constant `USELIBXML` in `opml.rb`. I find that `libxml` is a bit faster, but it has some memory bugs on my machine (though these do not prevent the `Opml` class from working properly). Since `libxml` may not be present on your machine, you will have to [install it](http://libxml.rubyforge.org/install.xml) if you want to use it.
+> ANOTHER NOTE: The `Opml` class is designed to be implemented either using the built-in Ruby library `REXML` or the Ruby gem `libxml`. To set which of the two is used, edit the value of the boolean constant `USELIBXML` in `opml.rb`. I find that `libxml` is a bit faster, but it has some memory bugs on my machine (though these do not prevent the `Opml` class from working properly). Since `libxml` may not be present on your machine, you will have to [install it](http://libxml.rubyforge.org/install.xml) if you want to use it. <i>I intend eventually to abandon use of `REXML` and `libxml` entirely and use [Nokogiri](http://nokogiri.org/) instead.</i> This change should not matter to users with outline renderers (if any exist apart from myself), as this should be a mere behind-the-scenes implementation detail as far as you're concerned.
 
 </div>
 

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/docs/RubyFrontierDocumentation/defaultfolder/techfolder/scalardirectives.txt

@@ -94,10 +94,12 @@ If a [page object](types) file is a renderable, RubyFrontier uses the page objec
 
 * **:renderoutlinewith**. The name of the [outline renderer](outlinerenderers) to be used if the page object is an outline (`.opml`).
 
+* **:treatasopml**. If `true`, a `.txt` file will be treated as if it were an outline (`.opml`), converting the text to XML in accordance with its indentation structure.
+
 #### Template
 
 * **:template**. The name of the [template](template) into which the page object should be poured. If not defined, the file `#template.txt` will be used. If defined, the file in question will be sought by appending ".txt" and looking in the source folder's `#templates` folder and in the [user templates](user) folder.
 
 #### Macro processing
 
-* **:processmacros**. A boolean (the default is `true`): should [macro](macros) ([ERB](ERB)) expressions be processed? It would be a very rare thing to turn this off.
+* **:processmacros**. A boolean (the default is `true`): should [macro](macros) ([ERB](ERB)) expressions be processed? It would be a very rare thing to turn this off.

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/docs/RubyFrontierDocumentation/defaultfolder/whyrf.txt

@@ -28,7 +28,7 @@ And what about Frontier's "tables" of "scalars"? Well, Ruby has internal "tables
 
 So, it was starting to look like the project might be possible after all.
 
-Still I hesitated, worried about one final piece of the puzzle — outlines. [Outlining](http://www.outliners.com/) is one of Frontier's great strengths, and not something I wanted to lose. But then, once more, I got to thinking: Where in the Web site framework are outlines *really* needed? The outline representation of scripts is taken care of by TextMate's code folding feature. The outline representation of the hierarchy of file and folders on disk is handled by the TextMate project drawer. The one remaining place where outlines are important is this: in Frontier, an object to be turned into a Web page can *be* an outline, where a "renderer" transforms the outline into HTML. After some hesitation over this issue, I decided that I could use [OmniOutliner](http://www.omnigroup.com/applications/omnioutliner/) to open and save outlines as OPML, which Ruby could then parse. True, this introduces an inconvenience in the writing/editing process: if a Web page is constructed as an outline, it must be edited using OmniOutliner (not TextMate directly). But such a slight inconvenience seemed insufficient to bar usability.
+Still I hesitated, worried about one final piece of the puzzle — outlines. [Outlining](http://www.outliners.com/) is one of Frontier's great strengths, and not something I wanted to lose. But then, once more, I got to thinking: Where in the Web site framework are outlines *really* needed? The outline representation of scripts is taken care of by TextMate's code folding feature. The outline representation of the hierarchy of file and folders on disk is handled by the TextMate project drawer. The one remaining place where outlines are important is this: in Frontier, an object to be turned into a Web page can *be* an outline, where a "renderer" transforms the outline into HTML. After some hesitation over this issue, I decided that I could use [Opal](http://a-sharp.com/opal/) to open and save outlines as OPML, which Ruby could then parse. True, this introduces an inconvenience in the writing/editing process: if a Web page is constructed as an outline, it must be edited using Opal (not TextMate directly). But such a slight inconvenience seemed insufficient to bar usability. (Later, I introduced a mechanism for converting indented text to OPML, thus allowing an outline to be maintained as pure text using TextMate alone.)
 
 ## A Voyage of Discovery
 

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/opml.rb

@@ -2,6 +2,8 @@
 Simulate some Frontier op.* verbs using OPML as the outline source.
 =end
 
+# TODO: now that I trust Nokogiri, should eliminate use of libxml and rexml and rely on Nokogiri entirely
+
 # superclass's "new" factory method lets us substitute different subclass implementations at will
 # also container for methods that don't vary between implementations
 class Opml
@@ -117,14 +119,58 @@ def getLineTextRaw
     @curline
   end
 
+  # class method, utility: translate line-indented text to OPML
+  # at the moment we basically assume line-indent by spaces (probably two spaces per level)
+
+  def self.doThisLevel(lines, level, doc, curnode) # private loop/recurse helper for next method
+    # lines is the array of lines
+    # level is the level we are at:
+    # a deeper level means add a child, another at this level means add a sibling
+    # doc is a reference to the xml document
+    # curnode points to an already processed node; level is its level
+    # =======
+    # keep processing lines as long as level doesn't go shallower
+    # if it does, do nothing and let unwinding of recursion deal with it
+    while (lines.length > 0) && ((nextlevel = lines[0][:level]) >= level)
+      if nextlevel > level
+        newnode = curnode.add_child(Nokogiri::XML::Node.new("outline", doc))
+        newnode['text'] = lines[0][:text]
+        lines.shift
+        doThisLevel(lines, nextlevel, doc, newnode) # dive dive dive
+      else
+        newnode = curnode.add_next_sibling(Nokogiri::XML::Node.new("outline", doc))
+        newnode['text'] = lines[0][:text]
+        curnode = newnode
+        lines.shift
+      end
+    end
+  end
+  class << self; private :doThisLevel; end
+  def self.textToOpml(s)
+    doc = Nokogiri::XML::Document.new
+    doc.root = Nokogiri::XML::Node.new("opml", doc)
+    doc.root['version'] = '1.0'
+    body = doc.root.add_child(Nokogiri::XML::Node.new("body", doc))
+    lines = s.split("\n")
+    # separate level from content up front
+    lines = lines.map do |line|
+        line =~ /^(\s*)/
+        level = $1.length
+        rest = line[level..-1]
+        {:text => rest, :level => level}
+    end
+    doThisLevel(lines, -1, doc, body)
+    doc
+  end
+
 end
 
 class Opmlrexml < Opml
   myrequire ['rexml/document', :REXML]
 
   # ivars: doc, top, curline
-  def initialize(f)
-    @doc = Document.new(File.read(f))
+  def initialize(f) # f can be pathname or string
+    @doc = f.kind_of?(Pathname) ? Document.new(File.read(f)) : Document.new(f)
     @top = @doc.root.elements["body"]
     self.firstSummit()
   end
@@ -251,8 +297,8 @@ def previous_element
   end
 
   # ivars: doc, top, curline
-  def initialize(f) # f is a file
-    @doc = Document.file(f)
+  def initialize(f) # f can be pathname or string
+    @doc = f.kind_of?(Pathname) ? Document.file(f) : Document.string(f)
     @top = @doc.root.find_first("body")
     self.firstSummit()
   end

RubyFrontier.tmbundle/Support/bin/RubyFrontier/longestJourney/userland_pagemaker.rb

@@ -378,12 +378,26 @@ def runDirectives(adrObject, adrPageTable=@adrPageTable)
       while line = io.gets and line[0,1] == "#"
         runDirective(line[1..-1], adrPageTable)
       end
-      line + (io.gets(nil) || "") # read all the rest
+      rest = line + (io.gets(nil) || "") # read all the rest
+      if adrPageTable[:treatasopml]
+        # new feature: keep outline-structured text in .txt file
+        # we use #treatasopml directive to alert us
+        # we already have the directives, so now just create and return the Opml object
+        opml = Opml.textToOpml(rest)
+        op = Opml.new(opml.to_s)
+        # problem! runDirectives will be called *again* on the template
+        # but we don't want *it* treated as opml
+        # the only thing I can think of right now is to turn the switch back off
+        adrPageTable[:treatasopml] = false
+        return op
+      end
+      return rest
     end
   end
   def runOutlineDirectives(adrObject, adrPageTable=@adrPageTable)
+    # start with .opml file: extract to Opml object
+    op = Opml.new(adrObject)
     # extract directives from start of outline, return rest of outline
-    op = Opml.new(adrObject.to_s)
     while aline = op.getLineText and aline[0,1] == "#"
       runDirective(aline[1..-1], adrPageTable)
       op.deleteLine