Examples

[deerpig]

The documentation is a little too simple for someone at my level of programming. It would be very helpful to have a couple of complete concrete examples to be able to use templatel when getting started.

Are there any examples out there already?

I'm looking for a a way to create a set of templates with inheritance and then generate a file from an alist of variables. The two snippets provided in the documentation for inheritance do not make it clear to me how to do this.

So far this is what I have, but it doesn't work -- not sure if I am going in the right direction:

(templatel-env-new
 :importfn (lambda(env name)
             (templatel-env-add-template env name
              (templatel-new-from-file
               (expand-file-name name "./templates")))


   (templatel-env-render env "template.html"
                       '(("title" . "A nice web page")
                         ("users" . ((("url" . "https://chenla.la")
                                      ("name" . "link"))
                                     (("url" . "https://gnu.org")
                                      ("name" . "Gnu!!"))))))))

[deerpig]

I based the code above on the following, which does work:

(with-temp-buffer
    (insert 
        (templatel-render-file "./templates/template.html"
                       '(("title" . "A nice web page")
                         ("users" . ((("url" . "https://chenla.la")
                                      ("name" . "link"))
                                     (("url" . "https://gnu.org")
                                      ("name" . "Gnu!!")))))))
(write-file "out.html")
(kill-buffer "out.html"))

[clarete]

Hi @deerpig 👋🏾 thank you for taking the time to open the issue. That's a great point, things are a little scattered around in the API documentation and we don't really have a good tutorial there. The code you provided won't really work with inheritance because it depends on an import function. To kick things off, I'd love to get your feedback on what I'd probably think of using to write a more thorough tutorial for that.

A small example

This is an excerpt from our test file. I'll use it as a starting point to try to get to the more involved case we're looking for.

(let ((env (templatel-env-new
              :importfn (lambda(e name)
                          (templatel-env-add-template
                           e name
                           (templatel-new "{% block greeting %}Hello {{ text }}{% endblock %}"))))))
    (templatel-env-set-autoescape env t)
    ;; The override that calls super also didn't escape anything
    (templatel-env-add-template
     env "page.html"
     (templatel-new "{% extends \"nav.html\" %}{% block greeting %}{{ super() }} and world{% endblock %}"))
    ;; The output should be escaped
    (should (equal (templatel-env-render env "page.html" '(("text" . "<h1>Title</h1>")))
                   "Hello &lt;h1&gt;Title&lt;/h1&gt; and world")))

The first thing to notice is that you can't really use templatel-render-file when you need inheritance, and that might be a bit of a design flaw that. I might revisit that at some point in a new version of templatel, but let's just say for now that inheritance requires the template-env-* API to be used.

The next point to notice is that you need to create an environment with templatel-env-new with an import function attached to it, so that it can be called for templates that use the extend statement.

I'll go into more details on how to write an import function in a bit, let's just also mention that once the template environment is created, you can configure it to your liking (as the example does with templatel-env-set-autoescape) and then you need to add the main template to the environment (in the example, we name it "page.html", and notice that it contains the use of the extends keyword). Once the main template is added to the environment, you can render it with templatel-env-render).

---

The :importfn parameter

On the example I mentioned in the beginning, the :importfn always adds the same template to the environment, no matter what name it is given. That works pretty well for that test, because we only ever call extend once. That wouldn't work so well for most cases though. The project that made me think of writing templatel (weblorg) searches for a template name through a list of paths, resembling what Django and Flask usually do as well.

To write such a function, we can break the problem into two steps, the actual import function that will be passed to the :importfn parameter of templatel-env-new:

Excerpt extracted from here but simplified.

 (defun a-nice-import-function (en name)
    (templatel-env-add-template
     en name
     (templatel-new-from-file (weblorg--template-find '("/some/path" "/some/other/path")
                                             name)))))

And then the implementation for weblorg--template-find would look like this. The following excerpt was extracted from here.

(defun weblorg--template-find (directories name)
  "Find template NAME within DIRECTORIES.
This function implements a search for templates within the
provided list of directories.  The search happens from left to
right and returns on the first successful match.
This behavior, which is intentionally similar to the PATH
variable in a shell, allows the user to override just the
templates they're interested in but still take advantage of other
default templates."
  (if (null directories)
      ;; didn't find it. Signal an error upwards: (must be caught with `condition-case` or will raise an error to the user)
      (signal
       'file-missing
       (list "" "File not found" (format "Template `%s' not found" name)))
    ;; Let's see if we can find it in the next directory
    (let* ((path (expand-file-name name (car directories)))
           (attrs (file-attributes path)))
      (cond
       ;; doesn't exist; try next dir
       ((null attrs) (weblorg--template-find (cdr directories) name))
       ;; is a directory
       ((file-attribute-type attrs) nil)
       ;; we found it
       ((null (file-attribute-type attrs))
        path)))))

Wrap up

I'm curious to hear what you think about the above, and once we have a version that is good enough, I can make a lil page on the templatel website for it.

[deerpig]

Quite good -- wasn't able to spend much time with it yet -- the end of the month and year is a pain in the ass at the University. You've already cleared up some of my confusion but I want to play with the code and think this through carefully before I give you a proper reply.

To give you an idea of what we're trying to do with templatel, we are launching a research journal similar to the AI Journal Distill (http://distill.pub); an open access journal which uses a separate git repo for each article. This approach is a good fit for reproducible research since all lab notes, data and code for the article can be included in the repo for collaboration between scientists, as well as between the editors of the journal and the authors, not just for publication.

They use a template (https://github.com/distillpub/template) which is written in javascript which generates the final published paper. I hope to do something similar but use orgmode and elisp instead of javascript. The idea is that we can combine literate programing for organizing research, writing the paper and integrating it into org-roam as well as eventually generating the html and latex for a pdf. Templatel seems like a very good fit for this.

My question for you is if you think it's better to do this using weblorg or a lightweight version which uses templatel directly. It's a bit of a different use-case than generating a blog but I think it is a good fit.

I'll dive into your code and notes tomorrow morning (it's already 8pm here -- Phnom Penh) and give you more substantial feedback.

Cheers

[deerpig]

I've been trying to put everything from your post together and see how it works, but it is still only pulling templates in once. If you try to insert a file {{ file.html | safe }} or if you try to move the embedded template into a file and then call it using something like:

(templatel-env-add-template
     env "page.html"
     (templatel-new-from-file "page.html"))

You get a can't find file error.

As the doc string says. weblorg--template-find looks for a template and stops... Does this have to be called every time that you want to load a template?

Obviously this works in weblorg but I don't understand how... Is there a way for templatel to find any template in the import directory? A template system that can't find templates is somewhat limited :)

;; create new environment
(let ((env (templatel-env-new
            :importfn (lambda (e name)
    (templatel-env-add-template
     e name
     (templatel-new-from-file (weblorg--template-find '("./templates")
                                             name)))))))
    (templatel-env-add-template
     env "page.html"
     (templatel-new "{% extends \"nav.html\" %}{% block greeting %}{{ super() }}
     ...and Bingo was it's Name-o.{% endblock %}"))
    ;; Render template and save to file
     (with-temp-buffer
       (insert
       	  (templatel-env-render env "page.html" '(("text" . "<h1>B-I-N-G-O</h1>"))))
       (write-file "out.html")
       (kill-buffer)))

[clarete]

Quite good -- wasn't able to spend much time with it yet -- the end of the month and year is a pain in the ass at the University. You've already cleared up some of my confusion but I want to play with the code and think this through carefully before I give you a proper reply.

To give you an idea of what we're trying to do with templatel, we are launching a research journal similar to the AI Journal Distill (http://distill.pub); an open access journal which uses a separate git repo for each article. This approach is a good fit for reproducible research since all lab notes, data and code for the article can be included in the repo for collaboration between scientists, as well as between the editors of the journal and the authors, not just for publication.

They use a template (https://github.com/distillpub/template) which is written in javascript which generates the final published paper. I hope to do something similar but use orgmode and elisp instead of javascript. The idea is that we can combine literate programing for organizing research, writing the paper and integrating it into org-roam as well as eventually generating the html and latex for a pdf. Templatel seems like a very good fit for this.

My question for you is if you think it's better to do this using weblorg or a lightweight version which uses templatel directly. It's a bit of a different use-case than generating a blog but I think it is a good fit.

I'll dive into your code and notes tomorrow morning (it's already 8pm here -- Phnom Penh) and give you more substantial feedback.

Cheers

This is pretty awesome. I think weblorg could actually serve as a good base, because. It will provide a few utilities off the shelf, and if you feel like things are going well, you can extend weblorg to do implement features you're missing.

That being said, I thought it was still worth trying to share an example with just templatel in case you decide taking a stab it again. Here we go: https://github.com/clarete/templatel/tree/main/examples/inheritance
you'll find a script called gen.el and it's basically just the assembling of the lose pieces of code we've been discussing.

That should serve as a starter for you to try different things with different template and directory settings! let me know how that goes and what you decide! 🙇🏾

[deerpig]

This is fantastic! I've been playing around with it on and off all day and indeed it works as advertised and templates nest quite nicely. I think it will work well for the journal (a second journal will be launched next year that will use the same backend). I'm also building a companion blog which will use weblorg.

I'll send you a link when everything is up, both for the journal and the blog (and to the repos as well).

I haven't really coded anything serious since 2006 -- I wrote a major mode in emacs; sort of a poor man's org-roam which was shown at the Extreme Markup Conference in Montreal but I couldn't get working well enough to release it. I don't even know if I still have the source code for it any more.

The journal and our fledgling institute will launch and demo at the Cyber Civilization Research Center at Keio University in Tokyo in february. Ted Nelson (the guy behind Xanadu) spent a couple of years at Keio. The CCRC is run by the Dave Farber (who is called the Grandfather of the Internet). I've been at this a while -- I founded the second ISP in Hong Kong back in 1991 running on a bunch of Sparc 10's and Silicon Graphics servers -- I can't believe that was over 30 years ago!

I now will jump back into coding again, in elisp and guile, to build out the reproducible research backend which will extend the git filesystem and treat content as packages which can be installed using the Guix package manager. All of my planned work for the next ten years depends on my getting this to work by September. So no pressure :)

Thank you for your help -- I hope to have a very long relationship with both templel and webblorg. These are small but important steps towards that.