Feature idea: exporting via org-publish-project

[EFLS]

Org-mode provides strong exporting features via org-publish-project. I'm wondering how difficult it would be to extend it so that it includes the links between notes.

I currently use a basic setup, shared in zd-tutorial here, but would love to expand this so that links between notes work.

To make something like this work, all Zetteldeft type links would need to be resolved first.
One option might be to replace them with Org-mode type links right before export, like this example on integrating org-links

Now I'm not sure how easy it is to plug in to org-publish, so that Zetteldeft can resolve the links.
The Org manual on publishing links is rather sparse.

All input and ideas welcome.

[TRSx80]

I very excitedly wrote out some ideas, then I deleted it all. 😄

I will come back to this after I have read all the links above and thought about it some more, and have something actually productive/useful to add, as it's clear you have already put some thought into the potential ways forward.

Until then, cheers!

[TRSx80]

Once again I manage to ramble on and on. So I insert some headings for you "high time preference" types. 😉

Background / Motivation

OK, I been thinking about this quite a lot. I will admit to never really getting my head around the complicated beast that is Org publishing to HTML, even though I am very interested in it and I have tried at least once or twice before.

I do think it would be another "killer feature" of Zetteldeft to be able to have some sort of nice, automated way to export all Zettel to HTML, convert links, etc. and dump the generated HTML into some specified folder which you could then easily sync to mobile (with Syncthing, or whatever). With some appropriate responsive css, I think is the best possible interface ( at least for viewing) on mobile. Also cross platform (Android, iPhone, or anything, really, as it's just HTML of course).

I have been interested in learning the Org publishing stuff on a personal level for a long time, as I want to publish a blog or something easily from my preferred editing environment in Emacs/Org. And I think this could be a really good and easy way to do that.

Converting Zetteldeft links to Orgmode links

I realized that a vanilla Zetteldeft setup are not actually using Org links. So I guess this adds one more conversion step. Not sure how to handle this yet, but is just one of many reasons why I have been so keen to get more vanilla Org functionality working in (or on top of) Zetteldeft. So most of what follows really will be an Org link based implementation. Which I still think is best approach (unless you know something I don't).

Org links in Zetteldeft

I have actually last night finished a (mostly) complete vanilla Org link implementation for Zetteldeft. Which I will make a different topic about (EDIT: here it is: #79). However it led me even closer to what I had suspected from the beginning. And so finally we get to the actual topic. 😄

org-link-set-parameters :export

If I am understanding correctly, all we should have to do would be to implement the :export "method" (for lack of a better term) of org-link-set-parameters which would resolve the Zetteldeft ID out of zdlink: to an (I think relative would be best) file location, which would in turn ultimately be converted to a (relative) HTML link upon export.

Single char Org link type (§:) not possible (?)

Oh, and while we are on this topic. I could not seem to create an Org link type for §:, only for zdlink:. I think the problem is that it is a single character, and thus not allowed to define for a link type in Org. I went digging through the ol.el and other sources, but I could not find what I was looking for. I think I will probably ask the Orgmode mailing list for confirmation at some point after maybe trying to search some more. Any help on this will be appreciated however.

So, if the above is true, it seems zdlink: (or similar) Org links should not be too hard to implement. Not a problem for me, personally, but I know a lot of you guys love your "funny little S" (§). 😄

All jokes aside, for a plain, non bracketed link (eg. zdlink:2020-08-26-1234) I think it would actually look quite nice with the "funny little S" (ex.: §:2020-08-26-1234) so perhaps this means I am becoming a convert. 😄

Directory to export to?

Only other part would be deciding what directory we want to export to, should we hard code that and how would that affect our conversion function.

Closing

Anyway, I think that is enough idiotic ramblings to at least convey the idea. What do you think about that approach?

I know you are busy, do not stress out about getting back to me. I will keep shouting into the ether here and working on things on my end. I know you will reply when you can. I am just lucky to have some time on my hands at the moment and so I am "making hay while the sun shines" so to speak. Cheers!

[EFLS]

I realized that a vanilla Zetteldeft setup are not actually using Org links.

That's the main challenge indeed. Neat idea to use a §: type of link, I hadn't thought of that, but it would not cover the "vanilla" Zetteldeft setup.

I'm thinking along the lines of creating a copy of the Zetteldeft directory, and then doing a sort of find-and-replace on all link, to replace them with hard file links so that org-export takes care of them.

[TRSx80]

That sort of conversion could be a one liner in awk perhaps... Or, whatever tool you feel comfortable in of course. Perhaps even on the fly as part of the HTML export pipeline... I am not familiar enough with the Org export to HTML (yet), so I can't really say which way might work best.

[EFLS]

I've added an exporting feature. It allows exporting Zetteldeft notes written in Org via org-publish.

Read about it in the literate code: https://www.eliasstorms.net/zetteldeft/zetteldeft.html#export-setup

Or check it out in the exported version of zd-tutorial: https://www.eliasstorms.net/zd-tutorial/2020-10-06-1014%20Exporting%20notes.html

If you check it out, please let me know whether it works!

[patmaddox]

I have another approach to offer, that I believe is probably more in line with the org-mode. I am still in the early stages - I just cobbled this together from some documentation, your org-link integration code, and a random site I found online.

The main shortcoming I've found so far is that you can't set a EXPORT_FILE_NAME with a subdir - it just throws the dir away and only keeps the base file name, I don't know why. So basically this will produce a flat site with no directories. I haven't tried it with assets yet - haven't gotten that far.

It does avoid the whole copy-and-replace thing described in the export setup. Essentially it uses org-link's resolving mechanism to generate a link to the right file. ID-based export is something I've wanted for a long time. And after just a couple days of plugging away at emacs and zetteldeft, I have it! So thank you for all the work that's gone into zetteldeft.

I hope this is helpful as a starting point for a different approach to org export.

(setq org-publish-project-alist
      '(("pat.local"
         :base-directory "~/code/writing/notes"
         :publishing-directory "~/pat.local"
         :publishing-function org-html-publish-to-html
         :base-extension "pat-local\.org"))) ; this lets me only publish files such as "202012300805 home.pat-local.org"

; from http://kitchingroup.cheme.cmu.edu/blog/2013/05/05/Getting-keyword-options-in-org-files/
(defun jk-org-kwds ()
  "parse the buffer and return a cons list of (property . value)
from lines like:
#+PROPERTY: value"
  (org-element-map (org-element-parse-buffer 'element) 'keyword
    (lambda (keyword) (cons (org-element-property :key keyword)
                            (org-element-property :value keyword)))))

(defun jk-org-kwd (KEYWORD)
  "get the value of a KEYWORD in the form of #+KEYWORD: value"
  (cdr (assoc KEYWORD (jk-org-kwds))))

(org-link-set-parameters
 "z"
 :follow #'zetteldeft--search-filename
 :complete 'efls/zd-complete-link
 :help-echo "Searches provided ID in zetteldeft"
 :export #'zetteldeft--export-link)

(defun zetteldeft--export-link (link description _format _)
  (save-excursion
    (zetteldeft--search-filename link)
    (let ((path (jk-org-kwd "EXPORT_FILE_NAME")))
      (format "<a href=\"%s\">%s</a>" path (or description link)))))

(defun efls/zd-complete-link ()
  "Link completion for `tslink' type links"
  (let* ((file (completing-read "File to link to: "
                                (deft-find-all-files-no-prefix)))
         (link (zetteldeft--lift-id file)))
    (unless link (user-error "No file selected"))
    (concat "z:" link)))

[EFLS]

Thanks for your comment, Pat!

Essentially it uses org-link's resolving mechanism to generate a link to the right file.

As you state, this requires that every link between notes is an org-link. While this is possible (and I can imagine some people prefer it this way), it is different from the "default" Zetteldeft-type links. The reason Zetteldeft doesn't use org-mode links by default, is that I want Zetteldeft to work with non-org files as well (e.g. Markdown files).

In any case, your setup is a nice addition for those who use Org-links, so thanks for that! We could include it in the zd-tutorial & knowledge base later on.

[patmaddox]

The reason Zetteldeft doesn't use org-mode links by default, is that I want Zetteldeft to work with non-org files as well (e.g. Markdown files).

Ah, that makes perfect sense. I've been using The Archive and markdown for the last year, so I have lots of those links (as alluded to in #99). After a few days on org-mode, it's clear to me that it's a lot more powerful than markdown, and I might as well just use it for everything.

The final nail in the markdown coffee was precisely this functionality. I've had "add ID-based linking to Jekyll" on my mind for a while now (it's plausible there's a plugin, I haven't looked into it much). Now that I can get that behavior cheaply with org-link, I don't think I'll be looking back at all.

[TRSx80]

After a few days on org-mode, it's clear to me that it's a lot more powerful than markdown, and I might as well just use it for everything.

So much this!

Unfortunately Markdown has become quite popular in the meantime.

Even though I (greatly!) prefer Orgmode, I still admire EFLS principled commitment to maintaining support for both.