This file is the source for all new and updated content on my website since March 2018. Content here may be in progress or incomplete. This file gets converted to Hugo files by the excellent ox-hugo.
You really should not read it here but at zzamboni.org.
- Pages
- Ideas
- Posts
- Literate config files
- Emacs
- Hammerspoon
- Getting Started With Hammerspoon
- Using Spoons in Hammerspoon
- Just Enough Lua to Be Productive in Hammerspoon, Part 1
- Just Enough Lua to Be Productive in Hammerspoon, Part 2
- First release of “Learning Hammerspoon”
- New release of “Learning Hammerspoon” is out!
- August 2020 release of “Learning Hammerspoon” is out!
- Elvish
- Security 🔐
- Other
- My blogging setup with Emacs, Org mode, ox-hugo, Hugo, GitLab and Netlify
- New release of /Publishing with Emacs, Org-mode and Leanpub/
- New book: Publishing with Emacs, Org-mode and Leanpub
- New book: Literate Config
- Nuevo libro “Utilerías de Unix”
- Automating Leanpub book publishing with Hammerspoon and CircleCI
- Hosting a Ghost Blog in GitHub - the easier way
- The Big Website Reboot
- Reviving an old Mac with Linux (part 1 - installing Linux)
- Reviving an old Mac with Linux (part 2 - prettifying boot)
- Website redesign!
- Fixing wake-from-sleep in Linux on a Mac when using Ethernet
- Locally hosting my Twitter archive
- Porting my CV from Org+LaTeX to JSON Resume
- resume-toolkit: from personal CV pipeline to reusable toolkit
This section contains all the static pages.
- Who:
- My name is Diego Zamboni. I am an IT architect, computer scientist, security expert, consultant, project and team leader, programmer, sysadmin, author and overall geek. I am from Mexico but live in Switzerland with my beautiful wife and our two awesome daughters.
- What:
- I currently am the Governance CISO at Avaloq. Previously I worked as Global Security Architect at AWS. I am also the author of “Learning CFEngine”, “Learning Hammerspoon” and a few other books.
- Where:
- I was born in Argentina, but have moved around all my life. When I was very young I moved to Mexico, where I grew up. I studied Computer Engineering at UNAM before moving to the U.S. to pursue my Ph.D. at Purdue University under the direction of Gene Spafford. Upon finishing my studies, my wife and I decided to go to Switzerland, where I worked at the IBM Zurich Research Lab. Eight years and two kids later, we moved to Mexico in late 2009. In 2015, we moved back to Switzerland.
- Long version:
- If you are interested, here’s my Curriculum Vitae. For other useless trivia about me, see here.
You can find my full Curriculum Vitae, also in PDF format.
This entire site is generated by Hugo and served by Netlify.
The source is stored in my zzamboni/zzamboni.org GitLab repository. I use ox-hugo to generate the content from a single source file in org-mode format, although all the older articles and pages are still stored in their original source Markdown files (I gradually convert them whenever I update them). Some of my project pages are stored in the gh-pages branch of their own GitHub repositories. You can read more about my setup here: My blogging setup with Emacs, Org mode, ox-hugo, Hugo, GitLab and Netlify.
I think it’s incredible that all of this infrastructure is so easy to use and available for free.
- 2008-2024 (Twitter):
- I was on Twitter fairly early (created my account in 2007, though my first tweet is from January 2008), and although I was never a super prolific user, I used it regularly for about 15 years, until its decline after its acquisition and conversion to X. My account is still there so that my username cannot be reused, but my posts are protected. You can find an archive of my tweets in this site: https://zzamboni.org/twitter/ courtesy of Tweetback. I’m on Bluesky these days, though I don’t post too much.
- 2009-2016:
- During this time my blog went through several iterations, hosted/powered by Posterous, Jekyll, Octopress, Postach.io and Enwrite. Most blog entries from this period have been merged into my current blog;
- 2005-2009:
- My blog titled BrT, powered mainly by a self-hosted Wordpress installation, with some intermixed use of Posterous. What you find at the link is a static archive copy;
- 1997-2001:
- My hand-maintained web page at Purdue University (here’s a mirror in case it disappears).
- C128 Code (code header background) is from the source code listing from my Commodore 128 program Supercataloguer 128.
- Scrabble letters (Blog background) and Tags (Tags background) from Pexels, licensed under CC0.
- Curriculum Vitae background image is generated by OpenAI’s Sora.
- All other site-page header and background photos were taken either by my wife or me.
If you have any concerns or questions about the images used in this site, please let me know.
I have decided to close the comments form that used to be here, because the only messages I ever got through it were spam.
If you want to reach me, please use your imagination (I’m not hard to find) or follow some of the social media icons above.
I have a few self-published books and courses, all of them available at {{{icon(leanpub)}}} Leanpub.
You can get each book individually, or get them all as a bundle!
@@hugo:{{<figure src=img/book-bundle-cover.png href=https://leanpub.com/b/diego nozoom=true >}}@@
{{{leanpubbook(learning-cfengine,style=”float:right” height=340)}}}
The best book for learning CFEngine!
The book has its own webpage at http://cf-learn.info, please visit it for more information, code samples, etc.
You can buy the book at https://leanpub.com/learning-cfengine or by clicking the link on the right.
{{{leanpubbook(learning-hammerspoon,style=”float:right”)}}}
Automate all the things! From window manipulation to automated system settings depending on your current location, Hammerspoon makes it possible.
In this book you will learn how to get started with Hammerspoon, how to use pre-made modules, and how to write your own, to achieve an unprecedented level of control over your Mac.
Learn more, read a free sample and get the book (you choose how much you pay!) at https://leanpub.com/learning-hammerspoon/, or click on the banner on the right.
{{{leanpubbook(utilerias-unix,style=”float:right” height=”340”)}}}
(this book is in Spanish)
¡Automatiza tus tareas e incrementa tu eficiencia! Realiza tareas de todo tipo usando los comandos más útiles en un sistema Unix/Linux.
Puedes obtener una muestra gratis y comprar el libro (¡tu eliges cuánto pagas!) en https://leanpub.com/utilerias-unix, o haz click en la imágen de la derecha.
{{{leanpubbook(lit-config,style=”float:right”)}}}
Learn about Literate Configuration and become a master of readable, maintainable and scalable configuration files!
Literate Configuration is the application of Literate Programming to configuration files. Literate Programming can be especially applicable to configuration files for the following reasons:
- Configuration files are inherently focused, since they correspond to a single application, program or set of programs, all related. This makes it easier to draw a narrative for them;
- Most configuration files are self-contained but their structure and syntax may not be immediately evident, so they benefit from a human-readable explanation of their contents;
- Configuration files are often shared and read by others, as we all like to learn by reading the config files of other people. Applying Literate Programming to config files makes them much easier to share, since their explanation is naturally woven into the code.
Org-mode is a powerful and simple markup language for general writing, but with unique features that make it easy to include code within the text, and even further, to easily extract that code into stand-alone source files which can be interpreted by their corresponding programs.
Whether you already use Emacs and org-mode or not, you will find value in this book by seeing how uniquely Literate Programming can help you better write, maintain, understand and share your config files.
{{{leanpubbook(emacs-org-leanpub,style=”float:right” height=”380”)}}}
Learn the workflow and tools to easily self-publish books using Emacs, GitHub and Leanpub.
Publishing your words has never been easier than it is today. Blogging means you can have your words read by thousands of people within minutes of writing them. Even publishing a book has become considerably easier through self publishing. There are many tools and publishers that allow you to get started for little or no money. Still, getting started can be confusing, and that is what this book is about.
In this book, I will show you the workflow and tools I use to publish my books. The three main tools involved are:
- The GNU Emacs editor together with Org-mode for writing, editing and exporting your text;
- GitHub or Bitbucket to store your book files.
- Leanpub for typesetting, previewing, publishing and selling your work.
To illustrate the process and provide you with a starting point, the source repository for this book is available at https://github.com/zzamboni/emacs-org-leanpub. I am populating the repository live as I write this book.
Check it out, and happy writing!
Prepare for passing the CISSP certification exam!
There are multitude of CISSP courses, books and materials out there, but this one is special: it is a collection of topics I found useful when preparing for my own certification, complemented with examples, exercises and additional information based on my own experience and knowledge. The course is content-complete for all 8 (ISC)2 Common Body of Knowledge domains, although I continue improving formatting, structure and adding exercises (quizzes and exercises for the CBK domains 1-3 are there, domains 4-8 are coming).
If you are preparing for taking the CISSP exam, I am sure you will find this course useful. Get it now:
CISSP Training: Prepare to pass the CISSP!
Here you can find a sample of some programs I have written over the years. For a full list of my public projects please check my GitHub and GitLab profiles.
{{{leanpubbook(emacs-org-leanpub,style=”float:right” height=”380”)}}}
Ox-leanpub is a Leanpub book exporter for Org mode. It allows you to write your material entirely in Org mode, and manages the production of the files and directories needed for Leanpub to render your book. I use this package to publish my books.
For a comprehensive introduction to publishing with Org mode and Leanpub, check out my book Publishing with Emacs, Org-mode and Leanpub!
Enwrite is a tool I wrote some time ago to publish a Hugo blog using Evernote. I don’t use it anymore since I switched to publishing my blog using Org-mode and ox-hugo, so it may be broken, but feel free to give it a try.
GrabCartoons is a comic-summarizing utility. It is modular, and it is very easy to write modules for new comics. It’s one of my oldest open-source projects, and still in use!
You can find all the information at http://github.com/zzamboni/grabcartoons
resume-toolkit is a tool for converting JSON Resume (and optionally, BibTeX files) into HTML and PDF formats.
jsonresume-theme-eventide is a JSON Resume theme, derived from jsonresume-theme-even, which I extended for use with resume-toolkit with features like a Table of Contents, section reordering and custom labels, certificate and organization logos, and much more.
Ideas for new blog posts and posts in progress.
https://github.com/barunespadhy/wingpanel-indicator-appicontray <write here>
<write here>
- Preventing lockup when waking from sleep with Ethernet connected.
- Fixing low-speed Ethernet negotiation when waking from sleep.
- Automatically disabling/enabling WiFi when Ethernet is connected/disconnected.
Blog posts.
I group here the posts about my documented config files, which include the live files from my current configuration.
:export_hugo_custom_front_matter+: :series ‘(“Literate Config Files”)’ :series_order 1{{{leanpubbook(lit-config,style=”float:right”)}}}
Last update: {{{updatetime}}}
In my ongoing series of literate config files, I am now posting my Doom Emacs config. I switched to Doom from my hand-crafted Emacs config some time ago, and I have been really enjoying it. Hope you find it useful!
As usual, the post below is included directly from my live doom.org file.
If you are interested in writing your own Literate Config files, check out my book Literate Config on Leanpub!
:export_hugo_custom_front_matter+: :series ‘(“Literate Config Files”)’ :series_order 2{{{leanpubbook(lit-config,style=”float:right”)}}}
Last update: {{{updatetime}}}
I have enjoyed slowly converting my configuration files to literate programming style style using org-mode in Emacs. I previously posted my Elvish configuration, and now it’s the turn of my Emacs configuration file. The text below is included directly from my init.org file. Please note that the text below is a snapshot as the file stands as of the date shown above, but it is always evolving. See the init.org file in GitHub for my current, live configuration, and the generated file at init.el.
If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!
:export_hugo_custom_front_matter+: :series ‘(“Literate Config Files”)’ :series_order 3{{{leanpubbook(lit-config,style=”float:right”)}}} {{{leanpubbook(learning-hammerspoon,style=”float:right”)}}}
Last update: {{{updatetime}}}
In my ongoing series of literate config files, I present to you my Hammerspoon configuration file. You can see the generated file at https://github.com/zzamboni/dot-hammerspoon/blob/master/init.lua. As usual, this is just a snapshot at the time shown above, you can see the current version of my configuration in GitHub.
If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!
:export_hugo_custom_front_matter+: :series ‘(“Literate Config Files”)’ :series_order 4{{{leanpubbook(lit-config,style=”float:right”)}}}
Last update: {{{updatetime}}}
In this blog post I will walk you through my current Elvish configuration file, with running commentary about the different sections.
This is also my first blog post written using org-mode, which I have started using for writing and documenting my code, using literate programming. The content below is included unmodified from my rc.org file (as of the date shown above), from which the rc.elv file is directly generated.
If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!
Without further ado…
(this is a slightly modified extract from my Doom Emacs configuration)
While writing with Org mode, I frequently need to insert links to other headings within my local document. I started by doing this manually, inserting a CUSTOM_ID property in the destination headline, and then creating the link.
Later, I discovered and now normally use counsel-org-link (part of counsel, which is included and enabled by default with Ivy in Doom Emacs) for linking between headings in an Org document. It shows me a searchable list of all the headings in the current document, and allows selecting one, automatically creating a link to it. Since it doesn’t have a keybinding by default, let’s start by giving it one (C-c l l is the default +links section in Doom Emacs):
(map! :after counsel :map org-mode-map
"C-c l l h" #'counsel-org-link)I also configure counsel-outline-display-style so that only the headline title is inserted into the link, instead of its full path within the document.
(after! counsel
(setq counsel-outline-display-style 'title))counsel-org-link uses org-id as its backend, which generates IDs using UUIDs and stores them in the ID property. I prefer using human-readable IDs stored in the CUSTOM_ID property of each heading, so we need to make some changes.
First, configure org-id to use CUSTOM_ID if it exists. This instructs org-id to grab those IDs when using the org-store-link function (funny that org-id knows how to recognize and use CUSTOM_ID, but not how to generate them).
(after! org-id
;; Do not create ID if a CUSTOM_ID exists
(setq org-id-link-to-org-use-id 'create-if-interactive-and-no-custom-id))Second, I override counsel-org-link-action, which is the function that actually generates and inserts the link, with a custom function that computes and inserts human-readable CUSTOM_ID links. This is supported by a few auxiliary functions for generating and storing the CUSTOM_ID.
(defun zz/make-id-for-title (title)
"Return an ID based on TITLE."
(let* ((new-id (replace-regexp-in-string "[^[:alnum:]]" "-" (downcase title))))
new-id))
(defun zz/org-custom-id-create ()
"Create and store CUSTOM_ID for current heading."
(let* ((title (or (nth 4 (org-heading-components)) ""))
(new-id (zz/make-id-for-title title)))
(org-entry-put nil "CUSTOM_ID" new-id)
(org-id-add-location new-id (buffer-file-name (buffer-base-buffer)))
new-id))
(defun zz/org-custom-id-get-create (&optional where force)
"Get or create CUSTOM_ID for heading at WHERE.
If FORCE is t, always recreate the property."
(org-with-point-at where
(let ((old-id (org-entry-get nil "CUSTOM_ID")))
;; If CUSTOM_ID exists and FORCE is false, return it
(if (and (not force) old-id (stringp old-id))
old-id
;; otherwise, create it
(zz/org-custom-id-create)))))
;; Now override counsel-org-link-action
(after! counsel
(defun counsel-org-link-action (x)
"Insert a link to X.
X is expected to be a cons of the form (title . point), as passed
by `counsel-org-link'.
If X does not have a CUSTOM_ID, create it based on the headline
title."
(let* ((id (zz/org-custom-id-get-create (cdr x))))
(org-insert-link nil (concat "#" id) (car x)))))Ta-da! Now using counsel-org-link inserts nice, human-readable links.
(tip of the hat: I got a lot of inspiration and some code for this from Emacs Org-mode: Use good header ids!)
Over the last few months, I have used org-mode more and more for writing and programming in Emacs. I love its flexibility and power, and it is the first literate programming tool which “feels right”, and I have been able to stick with it for a longer period of time than in my previous attempts.
Recently I started thinking about how I could make my editing environment more visually appealing. I am in general very happy with my Emacs’ appearance. I use the Gruvbox theme (in the meantime I have switched to the light Spacemacs theme) and org-mode has very decent syntax highlighting. But as I write more and more prose in Emacs these days, I started thinking it might be nice to edit text in more visually-appealing fonts, including using a proportional font, which makes regular prose much more readable. I would like to share with you what I learned and my current Emacs configuration.
In the end, you can have an Emacs setup for editing org documents which looks very nice, with proportional fonts for text and monospaced fonts for code blocks, examples and other elements. To wet your appetite, here is what a fragment of my init.org file looked like with the Gruvbox theme:
And this is how it looks now with the light Spacemacs theme:
My first step was to make org-mode much more readable by using different fonts for headings, hiding some of the markup, and improving list bullets. I took these settings originally from Howard Abrams’ excellent Org as a Word Processor article, although I have tweaked them a bit.
First, we ask org-mode to hide the emphasis markup (e.g. /.../ for italics, *...* for bold, etc.):
(setq org-hide-emphasis-markers t)Then, we set up a font-lock substitution for list markers (I always use ”-” for lists, but you can change this if you want) by replacing them with a centered-dot character:
(font-lock-add-keywords 'org-mode
'(("^ *\\([-]\\) "
(0 (prog1 () (compose-region (match-beginning 1) (match-end 1) "•"))))))The org-bullets package replaces all headline markers with different Unicode bullets:
(use-package org-bullets
:config
(add-hook 'org-mode-hook (lambda () (org-bullets-mode 1))))Finally, we set up a nice proportional font, in different sizes, for the headlines. The fonts listed will be tried in sequence, and the first one found will be used. My current favorite is ET Book, feel free to add your own:
(let* ((variable-tuple
(cond ((x-list-fonts "ETBembo") '(:font "ETBembo"))
((x-list-fonts "Source Sans Pro") '(:font "Source Sans Pro"))
((x-list-fonts "Lucida Grande") '(:font "Lucida Grande"))
((x-list-fonts "Verdana") '(:font "Verdana"))
((x-family-fonts "Sans Serif") '(:family "Sans Serif"))
(nil (warn "Cannot find a Sans Serif Font. Install Source Sans Pro."))))
(base-font-color (face-foreground 'default nil 'default))
(headline `(:inherit default :weight bold :foreground ,base-font-color)))
(custom-theme-set-faces
'user
`(org-level-8 ((t (,@headline ,@variable-tuple))))
`(org-level-7 ((t (,@headline ,@variable-tuple))))
`(org-level-6 ((t (,@headline ,@variable-tuple))))
`(org-level-5 ((t (,@headline ,@variable-tuple))))
`(org-level-4 ((t (,@headline ,@variable-tuple :height 1.1))))
`(org-level-3 ((t (,@headline ,@variable-tuple :height 1.25))))
`(org-level-2 ((t (,@headline ,@variable-tuple :height 1.5))))
`(org-level-1 ((t (,@headline ,@variable-tuple :height 1.75))))
`(org-document-title ((t (,@headline ,@variable-tuple :height 2.0 :underline nil))))))My next realization was that Emacs already includes support for displaying proportional fonts with the variable-pitch-mode command. You can try it right now: type M-x variable-pitch-mode and your current buffer will be shown in a proportional font (you can disable it by running variable-pitch-mode again). On my Mac the default variable-pitch font is Helvetica. You can change the font used by configuring the variable-pitch face. You can do this interactively through the customize interface by typing M-x customize-face variable-pitch. At the moment I like Source Sans Pro ET Book.
As a counterpart to variable-pitch, you need to configure the fixed-pitch face for the text that needs to be shown in a monospaced font. My first instinct was to inherit this from my default face (I use Inconsolata Fira Code), but it seems that this gets remapped when variable-pitch-mode is active, so I had to configure it by hand with the same font as my default face.
What I would suggest is that you customize the fonts interactively, as you can see live how it looks on your text. You can make the configuration permanent from the customize screen as well. If you want to explicitly set them in your configuration file, you can do it with the custom-theme-set-faces function, like this:
(custom-theme-set-faces
'user
'(variable-pitch ((t (:family "ETBembo" :height 180 :weight thin))))
'(fixed-pitch ((t ( :family "Fira Code Retina" :height 160)))))Tip #1: you can get the LISP expression for your chosen font (the part that looks like ((t (:family ... ))) from the customize-face screen - open the “State” button and choose the “Show Lisp Expression” menu item.
Tip #2: if you use a Mac, you can get the value to use for the :family attribute by looking at the “Family” attribute in the Font Book application for the font you want to use.
You can enable variable-pitch-mode automatically for org buffers by setting up a hook like this:
(add-hook 'org-mode-hook 'variable-pitch-mode)One thing you will notice right away with proportional fonts is that filling paragraphs no longer makes sense. This is because fill-paragraph works based on the number of characters in a line, but with a proportional font, characters have different widths, so a filled paragraph looks strange:
Of course, you can still do it, but there’s a better way. With visual-line-mode enabled, long lines will flow and adjust to the width of the window. This is great for writing prose, because you can choose how wide your lines are by just resizing your window.
There is one habit you have to change for this to work: the instinct (at least for me) of pressing M-q every once in a while to readjust the current paragraph. I personally think it’s worth it.
You can enable visual-line-mode automatically for org buffers by setting up another hook:
(add-hook 'org-mode-hook 'visual-line-mode)After all the changes above, you will have nice, proportional fonts in your Org buffers. However, there are some things for which you still want monospace fonts! Things like source blocks, examples, tags and some other markup elements still look better in a fixed-spacing font, in my opinion. Fortunately, org-mode has an extremely granular face selection, so you can easily customize them to have different elements shown in the correct font, color, and size.
Tip: you can use C-u C-x = (which runs the command what-cursor-position with a prefix argument) to show information about the character under the cursor, including the face which is being used for it. If you find a markup element which is not correctly configured, you can use this to know which face you have to customize.
You can configure specific faces any way you want, but if you simply want them to be rendered in monospace font, you can set them to inherit from the fixed-pitch face we configured before. You can also inherit from multiple faces to combine their attributes.
Here are the faces I have configured so far (there are probably many more to do, but I don’t use org-mode to its full capacity yet). I’m showing here the LISP expressions, but you can just as well configure them using customize-face.
(custom-theme-set-faces
'user
'(org-block ((t (:inherit fixed-pitch))))
'(org-code ((t (:inherit (shadow fixed-pitch)))))
'(org-document-info ((t (:foreground "dark orange"))))
'(org-document-info-keyword ((t (:inherit (shadow fixed-pitch)))))
'(org-indent ((t (:inherit (org-hide fixed-pitch)))))
'(org-link ((t (:foreground "royal blue" :underline t))))
'(org-meta-line ((t (:inherit (font-lock-comment-face fixed-pitch)))))
'(org-property-value ((t (:inherit fixed-pitch))) t)
'(org-special-keyword ((t (:inherit (font-lock-comment-face fixed-pitch)))))
'(org-table ((t (:inherit fixed-pitch :foreground "#83a598"))))
'(org-tag ((t (:inherit (shadow fixed-pitch) :weight bold :height 0.8))))
'(org-verbatim ((t (:inherit (shadow fixed-pitch))))))Update (2019/10/24): updated the settings above based on my latest config.
Update (2019/02/24): thanks to Ben for figuring out the fix to the vertical spacing issue noted below. The trick is to set the org-indent face (see above) to inherit from fixed-pitch as well.
One minor issue I have noticed is that, in =variable-pitch-mode=, the fixed-pitch blocks have a slight increase in inter-line spacing. This is not a deal breaker for me, but it is a noticeable difference. This can be observed in the following screenshot, which shows the block of code above embedded in the org-mode buffer and in the block-editing buffer, which uses the fixed-width font. If you know a way in which this could be fixed, please let me know!
The setup described above has considerably improved my enjoyment of writing in Emacs. I hope you find it useful. If you have any feedback, suggestions or questions, please let me know in the comments.
As I’m taking notes or writing in Org-mode, I often want to insert screenshots inline with the text. While Org supports inserting and displaying inline images, the assumption is that the image is already somewhere in the file system and we just want to link to it.
The org-download package eases the task of downloading or copying images and attaching them to a document, and it even has an org-download-screenshot command, but this assumes you want to initiate the screenshot from within Emacs, whereas the workflow I prefer is like this:
- Capture screenshot using the macOS built-in screenshot tool ({{{keys(Shift ⌘ 5)}}}) and leave it in the clipboard.
- Paste the image into the document I’m working on.
Fortunately, org-download allows customizing the command used by the org-download-screenshot command. Together with the pngpaste utility, this can be used to make org-download-screenshot store the image from the clipboard to disk, and insert it into the document. This is my configuration:
(use-package org-download
:after org
:defer nil
:custom
(org-download-method 'directory)
(org-download-image-dir "images")
(org-download-heading-lvl nil)
(org-download-timestamp "%Y%m%d-%H%M%S_")
(org-image-actual-width 300)
(org-download-screenshot-method "/usr/local/bin/pngpaste %s")
:bind
("C-M-y" . org-download-screenshot)
:config
(require 'org-download))With this configuration, images are stored in a directory named images under the current directory, in a flat directory structure and each file is prepended with a timestamp (I would prefer not to use timestamps, but org-download uses a fixed filename for screenshots, which makes it difficult to insert multiple screenshots in the same document). You may want to check the org-download documentation and configure these settings to your liking.
Finally, I bind org-download-screenshot to {{{keys(Ctrl ⌘ y)}}} to keep it similar to the default {{{keys(Ctrl y)}}} for pasting the clipboard and to easily perform step 2 of the workflow described above.
Now when I want to insert a screenshot in a document, I simply press {{{keys(Shift ⌘ 5)}}}, capture the screenshot, switch back to Emacs, press {{{keys(Ctrl ⌘ y)}}}, and done. It looks like this:
And without inline image display, we can see that the screenshot is automatically stored inside the images/ directory:
This is the first installment of a series of posts about Hammerspoon, a staggeringly powerful automation utility which gives you an amazing degree of control over your Mac, allowing you to automate and control almost anything. In the word of Hammerspoon’s motto: Making the runtime, funtime.
Hammerspoon is a Mac application that allows you to achieve an unprecedented level of control over your Mac. Hammerspoon enables interaction with the system at multiple layers–from low-level file system or network access, mouse or keyboard event capture and generation, all the way to manipulating applications or windows, processing URLs and drawing on the screen. It also allows interfacing with AppleScript, Unix commands and scripts, and other applications. Hammerspoon configuration is written in Lua, a popular embedded programming language.
Using Hammerspoon, you can replace many stand-alone Mac utilities for controlling or customizing specific aspects of your Mac (the kind that tends to overcrowd the menubar). For example, the following are doable using Hammerspoon (these are all things I do with it on my machine - each paragraph links to the corresponding sections in my config file):
- Add missing or more convenient keyboard shortcuts to applications, even for complex multi-step actions. For example: automated tagging and filing in Evernote, mail/note archival in Mail, Outlook and Evernote, filing items from multiple applications to OmniFocus using consistent keyboard shortcuts, or muting/unmuting a conversation in Skype.
- Open URLs in different browsers based on regular expression patterns. When combined with Site-specific Browsers (I use Epichrome), this allows for highly flexible management of bookmarks, plugins and search configurations.
- Replace Spotlight, Lacona and other launchers with a fully configurable, extensible launcher, which allows not only to open applications, files and bookmarks, but to trigger arbitrary Lua functions.
- Manipulate windows using keyboard shortcuts to resize, move and arrange them.
- Set up actions to happen automatically when switching between WiFi networks–for example for reconfiguring proxies in some applications.
- Keyboard-triggered translation of selected text between arbitrary human languages.
- Keep a configurable and persistent clipboard history.
- Automatically pause audio playback when headphones are unplugged.
Hammerspoon is the most powerful Mac automation utility I have ever used. If you are a programmer, it can make using your Mac vastly more fun and productive.
Hammerspoon acts as a thin layer between the operating system and a Lua-based configuration language. It includes extensions for querying and controlling many aspects of the system. Some of the lower-level extensions are written in Objective-C, but all of them expose a Lua API, and it is trivial to write your own extensions or modules to extend its functionality.
From the Hammerspoon configuration you can also execute external commands, run AppleScript or JavaScript code using the OSA scripting framework, establish network connections and even run network servers; you can capture and generate keyboard events, detect network changes, USB or audio devices being plugged in or out, changes in screen or keyboard language configuration; you can draw directly on the screen to display whatever you want; and many other things. Take a quick look at the Hammerspoon API index page to get a feeling of its extensive capabilities. And that is only the libraries that are built into Hammerspoon. There is an extensive and growing collection of Spoons, modules written in pure Lua that provide additional functionality and integration. And of course, the configuration is simply Lua code, so you can write your own code to do whatever you want.
Interested? Let’s get started!
Hammerspoon is a regular Mac application. To install it by hand, you just need to download it from https://github.com/Hammerspoon/hammerspoon/releases/latest, unzip the downloaded file and drag it to your /Applications folder (or anywhere else you want).
If you are automation-minded like me, you probably use Homebrew and its plugin Cask to manage your applications. In this case, you can use Cask to install Hammerspoon:
brew cask install hammerspoonWhen you run Hammerspoon for the first time, you will see its icon appear in the menubar, and a notification telling you that it couldn’t find a configuration file. Let’s fix that!
Let us start with a few simple examples. As tradition mandates, we will start with a “Hello World” example. Open $HOME/.hammerspoon/init.lua (Hammerspoon will create the directory upon first startup, but you need to create the file) in your favorite editor, and type the following:
hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "h" },
function()
hs.notify.show("Hello World!", "Welcome to Hammerspoon", "")
end
)Save the file, and from the Hammerspoon icon in the menubar, select “Reload config”. Apparently nothing will happen, but if you then press {{{keys(Ctrl ⌘ Alt h)}}} on your keyboard, you will see a notification on your screen welcoming you to the world of Hammerspoon.
Although it should be fairly self-explanatory, let us dissect this example to give you a clearer understanding of its components:
- All Hammerspoon built-in extensions start with
hs.In this case, {{{hsapi(hs.hotkey)}}} is the extension that handles keyboard bindings. It allows us to easily define which functions will be called in response to different keyboard combinations. You can even differentiate between the keys being pressed, released or held down if you need to. The other extension used in this example is {{{hsapi(hs.notify)}}}, which allows us to interact with the macOS Notification Center to display, react and interact with notifications. - Within
hs.hotkey, the {{{hsapi(hs.hotkey,bindSpec)}}} function allows you to bind a function to a pressed key. Its first argument is a key specification which consists of a list (Lua lists and table literals are represented using curly braces) with two elements: a list of the key modifiers, and the key itself. In this example,{ { "ctrl", "cmd", "alt" }, "h" }represents pressing {{{keys(Ctrl ⌘ Alt h)}}}. - The second argument to
bindSpecis the function to call when the key is pressed. Here we are defining an inline anonymous function usingfunction() ... end. - The callback function uses {{{hsapi(hs.notify,show)}}} to display the message. Take a quick look at the {{{hsapi(hs.notify)}}} documentation to get an idea of its extensive capabilities, including configuration of all aspects of a notification’s appearance and buttons, and the functions to call upon different user actions.
Try changing the configuration to display a different message or use a different key. After every change, you need to instruct Hammerspoon to reload its configuration, which you can do through its menubar item.
As you start modifying your configuration, errors will happen, as they always do when coding. To help in development and debugging, Hammerspoon offers a console window where you can see any errors and messages printed by your Lua code as it executes, and also type code to be evaluated. It is a very useful tool while developing your Hammerspoon configuration.
To invoke the console, you normally choose “Console…” from the Hammerspoon menubar item. However, this is such a common operation, that you might find it useful to also set a key combination for showing the console. Most of Hammerspoon’s internal functionality is also accessible through its API. In this case, looking at the {{{hsapi(hs,,documentation for the main hs module)}}} reveals that there is an {{{hsapi(hs,toggleConsole)}}} function. Using the knowledge you have acquired so far, you can easily configure a hotkey for opening and hiding the console:
hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "y" }, hs.toggleConsole)Once you reload your configuration, you should be able to use {{{keys(Ctrl ⌘ Alt y)}}} to open and close the console. Any Lua code you type in the Console will be evaluated in the main Hammerspoon context, so you can add to your configuration directly from there. This is a good way to incrementally develop your code before committing it to the init.lua file.
You may have noticed by now another common operation while developing Hammerspoon code: reloading the configuration, which you normally have to do from the Hammerspoon menu. So why not set up a hotkey to do that as well? Again, the {{{hsapi(hs)}}} module comes to our help with the {{{hsapi(hs,reload)}}} method:
hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "r" }, hs.reload)Another useful development tool is the hs command, which you can run from your terminal to get a Hammerspoon console. To install it, you can use the {{{hsapi(hs.ipc”,cliInstall)}}} function, which you can just add to your init.lua file to check and install the command every time Hammerspoon runs.
Now you have all the tools for developing your Hammerspoon configuration. In the next installment we will look at how you can save yourself a lot of coding by using pre-made modules. In the meantime, feel free to look through my Hammerspoon configuration file for ideas, and please let me know your thoughts in the comments!
In this second article about Hammerspoon, we look into Spoons, modules written in Lua which can be easily installed and loaded into Hammerspoon to provide ready-to-use functionality. Spoons provide a predefined API to configure and use them. They are also a good way to share your own work with other users.
See also the first article in this series.
As a first example, we will use the MouseCircle spoon, which allows us to set up a hotkey that displays a color circle around the current location of the mouse pointer for a few seconds, to help you locate it.
To install the spoon, download its zip file from https://github.com/Hammerspoon/Spoons/raw/master/Spoons/MouseCircle.spoon.zip, unpack it, and double-click on the resulting MouseCircle.spoon file. Hammerspoon will install the Spoon under ~/.hammerspoon/Spoons/.
Once a Spoon is installed, you need to use the hs.loadSpoon() function to load it. Type the following in the Hammerspoon console, or add it to your init.lua file and reload the configuration:
hs.loadSpoon("MouseCircle")After a spoon is loaded, and depending on what it does, you may need to configure it, assign hotkeys, and start it. A spoon’s API is available through the spoon.<SpoonName> namespace. To learn the API you need to look at the spoon documentation page. In the case of MouseCircle, a look at http://www.hammerspoon.org/Spoons/MouseCircle.html reveals that it has two methods (bindHotkeys() and show()) and one configuration variable (color) available under spoon.MouseCircle.
The first API call is spoon.MouseCircle:bindHotkeys(), which allows us to set up a hotkey that shows the mouse locator circle around the location of the mouse pointer. Let’s say we wanted to bind the mouse circle to {{{keys(Ctrl ⌘ Alt d)}}}. According to the MouseCircle documentation, the name for this action is show, so we can do the following:
spoon.MouseCircle:bindHotkeys({
show = { { "ctrl", "cmd", "alt" }, "d" }
})Once you do this, press the hotkey and you should see a red circle appear around the mouse cursor, and fade away after 3 seconds.
Each actionX is a name defined by the spoon, which refers to something that can be bound to a hotkey, and each keySpecX is a table with two elements: a list of modifiers and the key itself, such as { { "ctrl", "cmd", "alt" }, "d" }.
The second API call in the MouseCircle spoon is show(), which triggers the functionality of showing the locator circle directly. Let’s try it – type the following in the console:
spoon.MouseCircle:show()Most spoons are structured like this: you can set up hotkeys to trigger the main functionality, but you can also trigger it via method calls. Normally you won’t use these methods, but their availability makes it possible for you to use spoon functionality from our own configuration, or from other spoons, to create further automation.
spoon.MouseCircle.color is a public configuration variable exposed by the spoon, which specifies the color that will be used to draw the circle. Colors are defined according to the documentation for the {{{hsapi(hs.drawing.color)}}} module. Several color collections are supported, including the OS X system collections and a few defined by Hammerspoon itself. Color definitions are stored in Lua tables indexed by their name. For example, you can view the {{{hsapi(hs.drawing.color,hammerspoon)}}} table, including the color definitions, by using the convenient {{{hsapi(hs.inspect)}}} method on the console:
> hs.inspect(hs.drawing.color.hammerspoon)
{
black = {
alpha = 1,
blue = 0.0,
green = 0.0,
red = 0.0
},
green = {
alpha = 1,
blue = 0.0,
green = 1.0,
red = 0.0
},
osx_red = {
alpha = 1,
blue = 0.302,
green = 0.329,
red = 0.996
},
osx_green = {
...If we wanted to make the circle green, we can assign the configuration value like this:
spoon.MouseCircle.color = hs.drawing.color.hammerspoon.greenThe next time you invoke the show() method, either directly or through the hotkey, you will see the circle in the new color.
Note that in the second statement, we are calling the method using the dot notation, and explicitly passing the object as the first argument. Normally you would use colon notation, but the alternative can be useful when constructing function pointers. For example, if you wanted to manually bind a second key to show the mouse circle, you might initially try to use the following:
hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
spoon.MouseCircle:show)But this results in an error. The correct way is to wrap the call in an anonymous function:
hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
function() spoon.MouseCircle:show() end)Alternatively, you can use the {{{hsapi(hs.fnutils,partial)}}} function to construct a function pointer that includes the correct first argument:
hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
hs.fnutils.partial(spoon.MouseCircle.show,
spoon.MouseCircle))This is more verbose than the previous example, but the technique can be useful sometimes. Although Lua is not a full functional language, it supports using functions as first-class values, and the {{{hsapi(hs.fnutils)}}} extension includes a number of functions that make it easy to use them.
By now you know enough to use spoons with Hammerspoon’s native capabilities: look for the ones you want, download and install them by hand, and configure them in your init.lua using their configuration variables and API. In the next sections you will learn more about the minimum API of spoons, and how to install and configure spoons in a more automated way.
The advantage of using spoons is that you can count on them to adhere to a defined API, which makes it easier to automate their use. Although each spoon is free to define additional variable and methods, the following are standard:
SPOON:init()is called automatically (if it exists) by {{{hsapi(hs,loadSpoon)}}} after loading the spoon, and can be used to initialize variables or anything else needed by the Spoon.SPOON:start()should exist if the spoon requires any ongoing or background processes such as timers or watchers of any kind.SPOON:stop()should exist ifstart()does, to stop any background processes that were started bystart().SPOON:bindHotkeys(map)is exposed by spoons which allow binding hotkeys to certain actions. Itsmapargument is a Lua table with key/value entries of the following form:ACTION = { MODS, KEY }, where ACTION is a string defined by the spoon (multiple such actions can be defined), MODS is a list of key modifiers (valid values are"cmd","alt","ctrl"and"shift"), and KEY is the key to be bound, as shown in our previous example. All available actions for a spoon should be listed in its documentation.
Once you develop a complex Hammerspoon configuration using spoons, you may start wondering if there is an easy way to manage them. There are no built-in mechanisms for automatically installing spoons, but you can use a spoon called SpoonInstall that implements this functionality. You can download it from http://www.hammerspoon.org/Spoons/SpoonInstall.html. Once installed, you can use it to declaratively install, configure and run spoons. For example, with SpoonInstall you can use the MouseCircle spoon as follows:
hs.loadSpoon("SpoonInstall")
spoon.SpoonInstall:andUse("MouseCircle", {
config = {
color = hs.drawing.color.osx_red,
},
hotkeys = {
show = { { "ctrl", "cmd", "alt"}, "d" }
}})If the MouseCircle spoon is not yet installed, spoon.SpoonInstall:andUse() will automatically download and install it, and set its configuration variables and hotkeys according to the declaration.
If there is nothing to configure in the spoon, spoon.SpoonInstall:andUse("SomeSpoon") does exactly the same as hs.loadSpoon("SomeSpoon"). But if you want to set configuration variables, hotkey bindings or other parameters, the following keys are recognized in the map provided as a second parameter:
configis a Lua table containing keys corresponding to configuration variables in the spoon. In the example above,config = { color = hs.drawing.color.osx_red }has the same effect as settingspoon.MouseCircle.color = hs.drawing.color.osx_redhotkeysis a Lua table with the same structure as the mapping parameter passed to thebindHotkeysmethod of the spoon. In our example above,hotkeys = { show = { { "ctrl", "cmd", "alt"}, "d" } }automatically triggers a call tospoon.MouseCircle:bindHotkeys({ show = { { "ctrl", "cmd", "alt"}, "d" } }).loglevelsets the log level of theloggerattribute within the spoon, if it exists. The valid values for this attribute are ‘nothing’, ‘error’, ‘warning’, ‘info’, ‘debug’, or ‘verbose’.startis a boolean value which indicates whether to call the Spoon’sstart()method (if it has one) after configuring everything else.fnspecifies a function which will be called with the freshly-loaded Spoon object as its first argument. This can be used to execute other startup or configuration actions that are not covered by the other attributes. For example, if you use the {{{spoon(Seal)}}} spoon (a configurable launcher), you need to call itsloadPlugins()method to specify which Seal plugins to use. You can achieve this with something like this:spoon.SpoonInstall:andUse("Seal", { hotkeys = { show = { {"cmd"}, "space" } }, fn = function(s) s:loadPlugins({"apps", "calc", "safari_bookmarks"}) end, start = true, })
repoindicates the repository from where the Spoon should be installed if needed. Defaults to"default", which indicates the official Spoon repository at http://www.hammerspoon.org/Spoons/. I keep a repository of unofficial Spoons at http://zzamboni.org/zzSpoons/, and others may be available by the time you read this.disablecan be set totrueto disable the Spoon (easier than commenting it out when you want to temporarily disable a spoon) in your configuration.
Which allows me to write Install:andUse("MouseCircle", … ), which is shorter and easier to read.
Apart from the andUse() “all-in-one” method, SpoonInstall has methods for specific repository- and spoon-maintenance operations. As of this writing, there are two Spoon repositories: the official one at http://www.hammerspoon.org/Spoons/, and my own at http://zzamboni.org/zzSpoons/, where I host some unofficial and in-progress Spoons.
The configuration variable used to specify repositories is SpoonInstall.repos. Its default value is the following, which configures the official repository identified as “default”:
{
default = {
url = "https://github.com/Hammerspoon/Spoons",
desc = "Main Hammerspoon Spoon repository",
}
}To configure a new repository, you can define an extra entry in this variable. The following code creates an entry named “zzspoons” for my Spoon repository:
spoon.SpoonInstall.repos.zzspoons = {
url = "https://github.com/zzamboni/zzSpoons",
desc = "zzamboni's spoon repository",
}After this, both “zzspoons” and “default” can be used as values to the repo attribute in the andUse() method, and in any of the other methods that take a repository identifier as a parameter. You can find the full API documentation at http://www.hammerspoon.org/Spoons/SpoonInstall.html.
Spoons are a great mechanism for structuring your Hammerspoon configuration. If you want an example of a working configuration based almost exclusively on Spoons, you can view my own Hammerspoon configuration at https://github.com/zzamboni/dot-hammerspoon.
Hammerspoon’s configuration files are written in Lua, so a basic knowledge of the language is very useful to be an effective user of Hammerspoon. In this 2-part article I will show you the basics of Lua so you can read and write Hammerspoon configuration. Along the way you will discover that Lua is a surprisingly powerful language.
Lua is a scripting language created in 1993, and focused from the beginning in being an embedded language for extending other applications. It is easy to learn and use while having pretty powerful features, and is frequently used in games, but also in many other applications including, of course, Hammerspoon.
The purpose of this section is to give you a quick overview of the Lua features and peculiarities you may find most useful for developing Hammerspoon policies. I assume you are a programmer who knows some other C-like language–if you already know C, Java, Ruby, Python, Perl, Javascript or some similar language, picking up Lua should be pretty easy. Instead of detailing every structure, I will focus on the aspects that are most different or that are most likely to trip you up as you learn it.
Lua includes all the common flow-control structures you might expect. Some examples:
local info = "No package selected"
if pkg and pkg ~= "" then
info, st = hs.execute("/usr/local/bin/brew info " .. pkg)
if st == nil then
info = "No information found about formula '" .. pkg .. "'!"
end
endIn this example, in addition to the {{{luadoc(if,3.3.4,)}}} statement, you can see in the line that runs {{{hsapi(hs,execute)}}} that Lua functions can return multiple values (which is not the same as returning an array, which counts as a single value). Within the function, this is implemented simply by separating the values with commas in the return statement, like this: return val1, val2. You can also see in action the following operators:
- ==== for equality;
~=for inequality (in this respect it differs from most C-like languages, which use!=);..for string concatenation;andfor the logical AND operation (by extension, you can deduct thatorandnotare also available).
local doReload = false
for _,file in pairs(files) do
if file:sub(-4) == ".lua" and (not string.match(file, '/[.]#')) then
doReload = true
end
endIn this example we see the {{{luadoc(for,3.3.5)}}} statement in its so-called generic form:
for <var> in <expression> do <block> endThis statement loops the variables over the values returned by the expressions, executing the block with each the consecutive value until it becomes nil.
The for statement also has a numeric form:
for <var> = <first>,<last>,<inc> do <block> endThis form loops the variable from the first to the last value, incrementing it by the given increment (defaults to 1) at each iteration.
Going back to our example, we can also learn the following:
- The {{{luafun(pairs)}}} function, which loops over a table. We will learn more about Lua tables below, but they can be used to represent both regular and associative arrays.
pairs()treats thefilesvariable as an associative array, and returns in each iteration a key/value pair of its contents. - The
_variable, while not special per se, is used by convention in Lua for “throwaway values”. In this case we are not interested in the key in each iteration, just the value, so we assign the key to_, never to be used again. - Our first glimpse into the Lua {{{luadoc(string library,6.4)}}}, and the two ways in which it can be used:
- In
file:sub(-4), the colon indicates the object-oriented notation (see “Lua dot-vs-colon method access” below). This invokes the {{{luafun(string.sub)}}} function, automatically passing thefilevariable as its first argument. This statement is equivalent tostring.sub(file, -4). - In
string.match(file, '/'), we see the function notation used to call {{{luafun(string.match)}}}. Since thefilevariable is being passed as the first argument, you could rewrite this statement asfile:match('/[.]'). In practice, I’ve found myself using both notations somewhat exchangeably - feel free to use whichever you find most comfortable.
- In
You will notice that sometimes, functions contained within a module are called with a dot, and others with a colon. The latter is Lua’s object-method-call notation, and its effect is to pass the object on which the method is being called as an implicit first argument called self. This is simply a syntactic shortcut, i.e. the following two are equivalent:
file:match('/[.]')
string.match(file, '/')Note that in the second statement, we are calling the method using the dot notation, and explicitly passing the object as the first argument. Normally you would use colon notation, but when you need a function pointer, you need to use the dot notation.
Functions are defined using the function keyword.
function leftDoubleClick(modifiers)
local pos=hs.mouse.getAbsolutePosition() -- <1>
hs.eventtap.event.newMouseEvent(
hs.eventtap.event.types.leftMouseDown, pos, modifiers) -- <2>
:setProperty(hs.eventtap.event.properties.mouseEventClickState, 2)
:post() -- <3>
hs.eventtap.event.newMouseEvent( -- <4>
hs.eventtap.event.types.leftMouseUp, pos, modifiers):post()
endIn this example we can also see some examples of the Hammerspoon library in action, in particular two extremely powerful libraries: {{{hsapi(hs.mouse)}}} for interacting with the mouse pointer, and {{{hsapi(hs.eventtap)}}}, which allows you to both intercept and generate arbitrary system events, including key pressed and mouse clicks. This function simulates a double click on the current pointer position:
- We first get the current position of the mouse pointer using {{{hsapi(hs.mouse,getAbsolutePosition)}}}.
- We create a new mouse event of type {{{hsapi(hs.eventtap.event,types,=leftMouseDown=)}}} in the obtained coordinates and with the given modifiers.
- By convention, most Hammerspoon API methods return the same object on which they operate. This allows us to chain the calls as shown:
setProperty()is called on thehs.eventtapobject returned bynewMouseEventto set its type to a double click, andpost()is called on the result to issue the event. - Since we are generating system events directly, we also need to take care of generating a “mouse up” event at the end.
Function parameters are always optional, and those not passed will default to nil, so you need to do proper validation. In this example, the function can be called as leftDoubleClick(), without any parameters, which means the modifiers parameter might have a nil value. Looking at the {{{hsapi(hs.eventtap.event,newMouseEvent,documentation for newMouseEvent())}}}, we see that the parameter is optional, so for this particular function our use is OK.
You should try this function to see that it works. Adding it to you ~/.hammerspoon/init.lua function will make Hammerspoon define it the next time you reload your configuration. You could then try calling it from the console, but the easiest is to bind a hotkey that will generate a double click. For example:
hs.hotkey.bindSpec({ { "cmd", "ctrl", "alt" }, "p" },
leftDoubleClick)Once you reload your config, you can generate a double click by moving the cursor where you want it and pressing {{{keys(Ctrl ⌘ Alt p)}}}. While this is a contrived example, the ability to generate events like this is immensely powerful in automating your system.
Then I simply use hyper or shift_hyper in my key binding declarations:
hs.hotkey.bindSpec({ hyper, "p" }, leftDoubleClick)In the next installment, we will dive into Lua’s types and data structures. In the meantime, feel free to explore and learn on your own. If you need more information, I can recommend the following resources, which I have found useful:
- The Lua 5.3 Reference Manual, available at the official Lua website.
- The Lua Wiki, a community-maintained wiki with many descriptions, tips, examples and tutorials.
In this second article of the “Just Enough Lua” series, we dive into Lua’s types and data structures.
Table are the only compound data type in Lua, and are used to implement arrays, associative arrays (commonly called “maps” or “hashes” in other languages), modules, objects and namespaces. As you can see, it is very important to understand them!
A table in Lua is a collection of values, which can be indexed either by numbers or by arbitrary strings (the two types of indices can coexist within the same table). Let’s go through a few examples that will give you an overview (you can type these in the Hammerspoon console as we go, or at the prompt of the hs command - keep in mind that some of the statements are broken across multiple lines here for formatting, but each statement should be type in a single line in the console).
Table literals are declared using curly braces:
> unicorns = {} -- empty table
> people = { "Chris", "Aaron", "Diego" } -- array
> handles = { Diego = "zzamboni",
Chris = "cmsj",
Aaron = "asmagill" } -- associative arrayIndices are indicated using square brackets. Numeric indices start at 1 (not 0 as in most other languages). For identifier-like string indices, you can use the dot shortcut. Requesting a non-existent index returns nil:
> unicorns[1]
nil
> people[0]
nil
> people[1]
Chris
> handles['Diego']
zzamboni
> handles.Diego
zzamboni
> handles.Michael
nilWithin the curly-brace notation, indices that are not identifier-like (letters, numbers, underscores) need to be enclosed in quotes and square brackets. Values can be tables as well:
colors = { ["U.S."] = { "red", "white", "blue" },
Mexico = { "green", "white", "red" },
Germany = { "black", "red", "yellow" } }With non-identifier indices, you cannot use the dot-notation. Also, to see a table within the Hammerspoon console, use {{{hsapi(hs.inspect)}}}:
> colors["U.S."]
table: 0x618000470400
> hs.inspect(colors.Mexico)
{ "green", "white", "red" }
> hs.inspect(colors)
{
Germany = { "black", "red", "yellow" },
Mexico = { "green", "white", "red" },
["U.S."] = { "red", "white", "blue" }
}Iteration through an array is commonly done using the {{{luafun(ipairs)}}} functions. Note that it will only iterate through contiguous numeric indices starting at 1, so that it does not work well with “sparse” tables.
> for i,v in ipairs(people) do print(i, v) end
1 Chris
2 Aaron
3 Diego
> people[4]='John'
> for i,v in ipairs(people) do print(i, v) end
1 Chris
2 Aaron
3 Diego
4 John
> people[7]='Mike'
> for i,v in ipairs(people) do print(i, v) end
1 Chris
2 Aaron
3 Diego
4 John
> hs.inspect(people)
{ "Chris", "Aaron", "Diego", "John",
[7] = "Mike"
}The {{{luafun(pairs)}}} function, on the other hand, will iterate through all the elements in the table (both numeric and string indices), but does not guarantee their order. Both numeric and string indices can be mixed in a single table (although this gets confusing quickly unless you access everything using {{{luafun(pairs)}}}).
> for i,v in pairs(people) do print(i,v) end
1 Chris
2 Aaron
3 Diego
4 John
7 Mike
> for i,v in ipairs(handles) do print(i,v) end
<no output>
> for i,v in pairs(handles) do print(i,v) end
Aaron asmagill
Diego zzamboni
Chris cmsj
> handles[1]='whoa' -- assign the first numeric index
> hs.inspect(handles)
{ "whoa",
Aaron = "asmagill",
Chris = "cmsj",
Diego = "zzamboni"
}
> for i,v in ipairs(handles) do print(i,v) end
1 whoaThe built-in {{{luadoc(table,6.6)}}} module includes a number of useful table-manipulation functions, including the following:
- {{{luafun(table.concat)}}} for joining the values of a list in a single string (equivalent to
joinin other languages). This only joins the elements that would be returned by {{{luafun(ipairs)}}}.> table.concat(people, ", ") Chris, Aaron, Diego, John
- {{{luafun(table.insert)}}} adds an element to a list, by default adding it to the end.
> hs.inspect(people) { "Chris", "Aaron", "Diego", "John", "Bill", [7] = "Mike" } > table.insert(people, "George") > hs.inspect(people) { "Chris", "Aaron", "Diego", "John", "Bill", "George", "Mike" }
Note how in the last example, the contiguous indices have finally caught up to 7, so the last element is no longer shown separately (and will now be included by {{{luafun(ipairs)}}}, {{{luafun(table.concat)}}}, etc.
- {{{luafun(table.remove)}}} removes an element from a list, by default the last one. It returns the removed element.
> for i=1,4 do print(table.remove(people)) end Mike George Bill John > hs.inspect(people) { "Chris", "Aaron", "Diego" }
Notable omissions from the language and the {{{luadoc(table,6.6)}}} module are “get keys” and “get values” functions, common in other languages. This may be explained by the flexible nature of Lua tables, so that those functions would need to behave differently depending on the contents of the table. If you need them, you can easily build your own. For example, if you want to get a sorted list of the keys in a table, you can use this function:
function sortedkeys(tab)
local keys={}
for k,v in pairs(tab) do table.insert(keys, k) end
table.sort(keys)
return keys
endFunctions in Lua are first-class objects, which means they can be used like any other value. This means that functions can be stored in tables, and this is how namespaces (or “modules”) are implemented in Lua. We can inspect an manipulate them like any other table. Let us look at the {{{luadoc(table,6.6)}}} library itself. First, the module itself is a table:
> table
table: 0x61800046f740Second, we can inspect its contents using the functions we know:
> hs.inspect(table)
{
concat = <function 1>,
insert = <function 2>,
move = <function 3>,
pack = <function 4>,
remove = <function 5>,
sort = <function 6>,
sortedkeys = <function 7>,
unpack = <function 8>
}The function values themselves are opaque (we cannot see their code), but we can easily extend the module. For example, we could add our sortedkeys() function above to the table module for consistency. Lua allows us to specify the namespace of a function in its declaration:
function table.sortedkeys(tab)
local keys={}
for k,v in pairs(tab) do table.insert(keys, k) end
table.sort(keys)
return keys
endAll the Hammerspoon modules are implemented the same way:
> type(hs)
table
> type(hs.mouse)
table
> hs.inspect(hs.mouse)
{
get = <function 1>,
getAbsolutePosition = <function 2>,
getButtons = <function 3>,
getCurrentScreen = <function 4>,
getRelativePosition = <function 5>,
set = <function 6>,
setAbsolutePosition = <function 7>,
setRelativePosition = <function 8>,
trackingSpeed = <function 9>
}The common way of defining a new module in Lua is to create an empty table, and populate it with functions or variables as needed. For example, let’s put our double-click generator in a module. Create the file ~/.hammerspoon/doubleclick.lua with the following contents:
local mod={}
mod.default_modifiers={}
function mod.leftDoubleClick(modifiers)
modifiers = modifiers or mod.default_modifiers
local pos=hs.mouse.getAbsolutePosition()
hs.eventtap.event.newMouseEvent(
hs.eventtap.event.types.leftMouseDown, pos, modifiers)
:setProperty(hs.eventtap.event.properties.mouseEventClickState, 2)
:post()
hs.eventtap.event.newMouseEvent(
hs.eventtap.event.types.leftMouseUp, pos, modifiers):post()
end
function mod.bindto(keyspec)
hs.hotkey.bindSpec(keyspec, mod.leftDoubleClick)
end
return modYou can then, from the console, do the following:
> doubleclick=require('doubleclick')
> doubleclick.bindto({ {"ctrl", "alt", "cmd"}, "p" })
19:53:53 hotkey: Disabled previous hotkey ⌘⌃⌥P
hotkey: Enabled hotkey ⌘⌃⌥PYou have written and loaded your first Lua module. Let’s try it out! Press {{{keys(Ctrl ⌘ Alt p)}}} while your cursor is over a word in your terminal or web browser, to select it as if you had double-clicked it. You can also change the modifiers used with it. For example, did you know that Cmd-double-click can be used to open URLs from the macOS Terminal application?
> doubleclick.default_modifiers={cmd=true}Now try pressing {{{keys(Ctrl ⌘ Alt p)}}} while your pointer is over a URL displayed on your Terminal (you can just type one yourself to test), and it will open in your browser.
Note that the name doubleclick does not have any special meaning—it is a regular variable to which you assigned the value returned by require('doubleclick'), which is the value of the mod variable created within the module file (note that within the module file you use the local variable name to refer to functions and variables within itself). You could assign it to any name you want:
> a=require('doubleclick')
> a.leftDoubleClick()The argument of the {{{luafun(require)}}} function is the name of the file to load, without the .lua extension. Hammerspoon by default adds your ~/.hammerspoon/ directory to its load path, along with any other default directories in your system. You can view the places where Hammerspoon will look for files by examining the package.path variable. On my machine I get the following:
> package.path /Users/zzamboni/.hammerspoon/?.lua;/Users/zzamboni/.hammerspoon/?/ init.lua;/Users/zzamboni/.hammerspoon/Spoons/?.spoon/init.lua;/usr/ local/share/lua/5.3/?.lua;/usr/local/share/lua/5.3/?/init.lua;/usr/ local/lib/lua/5.3/?.lua;/usr/local/lib/lua/5.3/?/init.lua;./?.lua; ./?/init.lua;/Users/zzamboni/Dropbox/Personal/devel/hammerspoon/ hammerspoon/build/Hammerspoon.app/Contents/Resources/extensions/?.lua; /Users/zzamboni/Dropbox/Personal/devel/hammerspoon/hammerspoon/build/ Hammerspoon.app/Contents/Resources/extensions/?/init.lua
If you want to avoid these messages, you need to explicitly load the modules and assign them to variables, as follows:
> app=require('hs.application')
> app.get("Terminal")
hs.application: Terminal (0x610000e49118)This avoids the console message and has the additional benefit of allowing you to use app (you can use whatever variable you want) instead of typing hs.application in your code. This is a matter of taste—I usually prefer to have the full descriptive names (makes the code easier to read), but when dealing with some of the longer module names (e.g. {{{hsapi(hs.distributednotifications)}}}), this technique can be useful.
If you are familiar with regular expressions, you know how powerful they are for examining and manipulating strings in any programming language. Lua has {{{luadoc(patterns,6.4.1)}}}, which fulfill many of the same functions but have a different syntax and some limitations. They are used by many functions in the string library like {{{luafun(string.find)}}} and {{{luafun(string.match)}}}.
The following are some differences and similarities you need to be aware of when using patterns:
- The dot (
.) represents any character, just like in regexes. - The asterisk (
*), plus sign (+) and question mark (?) represent “zero or more”, “one or more” and “one or none” of the previous character, just like in regexes. Unlike regexes, these characters can only be applied to a single character and not to a whole capture group (i.e. the regex(foo)+is not possible). - Alternations, represented by the vertical bar (
|) in regexes, are not supported. - The caret (
^) and dollar sign ($) represent “beginning of string” and “end of string”, just like in regexes. - The dash (
-) represents a non-greedy “zero or more” (i.e. match the shortest possible string instead of the longest one) of the previous character, unlike in regexes, in which it’s commonly indicate by a question mark following the corresponding*or+The regex.*?is equivalent to the Lua pattern.-. - The escape character is the ampersand (
%) instead of the backslash (\). - Most character classes are represented by the same characters, but preceded by ampersand. For example
%dfor digits,%sfor spaces,%wfor alphanumeric characters.
For most common use cases, Lua patterns are enough, you just have to be aware of their differences. If you encounter something that really cannot be done, you can always resort to libraries like Lrexlib, which provide interfaces to real regex libraries. Unfortunately these are not included in Lua, so you would need to install them on your own.
Patterns, just like regular expressions, are commonly used for string manipulation, using primarily functions from the {{{luadoc(string,6.4)}}} library.
Lua includes the {{{luadoc(string,6.4)}}} library to implement common string manipulation functions, including pattern matching. All of these functions can be called either as regular functions, with the string as the first argument, or as method calls on the string itself, using the colon syntax (which, as we saw before, gets converted to the same call). For example, the following two are equivalent:
string.find(a, "^foo")
a:find("^foo")You can find the full documentation in the Lua reference manual and many other examples in the Lua-users wiki String Library Tutorial. The following is a partial list of some of the functions I have found most useful:
- {{{luafun(string.find,=string.find(str\, pat\, pos\, plain)=)}}} finds the pattern within the string. By default the search starts at the beginning of the string, but can be modified with the
posargument (index starts at 1, as with the tables). By defaultpatis intepreted as a Lua pattern, but this can be disabled by passingplainas a true value. If the pattern is not found, returnsnil. If the pattern is found, the function returns the start and end position of the pattern within the string. Furthermore, if the pattern contains parenthesis capture groups, all groups are returned as well. For example:> string.find("bah", "ah") 2 3 > string.find("bah", "foo") nil > string.find("bah", "(ah)") 2 3 ah > p1, p2, g1, g2 = string.find("bah", "(b)(ah)") > p1,p2,g1,g2 1 3 b ah
Note that the return value is not a table, but rather multiple values, as shown in the last example.
{{{luafun(table.unpack)}}} does the opposite, expanding an array into separate values which can be assigned to separate values as needed, or passed as arguments to a function:
> args={"bah", "(b)(ah)"}
> string.find(args)
[string "return string.find(args)"]:1:
bad argument #1 to 'find' (string expected, got table)
> string.find(table.unpack(args))
1 3 b ah- {{{luafun(string.match,=string.match(str\, pat\, pos)=)}}} is similar to
string.find, but it does not return the positions, rather it returns the part of the string matched by the pattern, or if the pattern contains capture groups, returns the captured segments:> string.match("bah", "ah") ah > string.match("bah", "foo") nil > string.match("bah", "(b)(ah)") b ah
- {{{luafun(string.gmatch,=string.gmatch(str\, pat)=)}}} returns a function that returns the next match of
patwithinstrevery time it is called, returningnilwhen there are no more matches. Ifpatcontains capture groups, they are returned on each iteration.> a="Hammerspoon is awesome!" > f=string.gmatch(a, "(%w+)") > f() Hammerspoon > f() is > f() awesome > f()
Most commonly, this is used inside a loop:
> for cap in string.gmatch(a, "%w+") do print(cap) end Hammerspoon is awesome
- {{{luafun(string.format,=string.format(formatstring\, …)=)}}} formats a sequence of values according to the given format string, following the same formatting rules as the ISO C
sprintf()function. It additionally supports a new format character%q, which formats a string value in a way that can be read back by Lua, escaping or quoting characters as needed (for example quotes, newlines, etc.). - {{{luafun(string.len,=string.len(str)=)}}} returns the length of the string.
- {{{luafun(string.lower,=string.lower(str)=)}}} and {{{luafun(string.upper,=string.upper(str)=)}}} convert the string to lower and uppercase, respectively.
- {{{luafun(string.gsub,=string.gsub(str\, pat\, rep\, n)=)}}} is a very powerful string-replacement function which hides considerably more power than its simple syntax would lead you to believe. In general, it replaces all (or the first
n) occurrences ofpatinstrwith the replacementrep. However,repcan take any of the following values:- A string which is used for the replacement. If the string contains %m, where m is a number, the it is replaced by the m-th captured group (or the whole match if m is zero).
- A table which is consulted for the replacement values, using the first capture group as a key (or the whole match if there are no captures). For example:
> a="Event type codes: leftMouseDown=$leftMouseDown, rightMouseDown=$rightMouseDown, mouseMoved=$mouseMoved" > a:gsub("%$(%w+)", hs.eventtap.event.types) Event type codes: leftMouseDown=1, rightMouseDown=3, mouseMoved=5 3
- A function which is executed with the captured groups (or the whole match) as an argument, and whose return value is used as the replacement. For example, using the
os.getenvfunction, we can easily replace environment variables by their values in a string:> a="Hello $USER, your home directory is $HOME" > a:gsub("%$(%w+)", os.getenv) Hello zzamboni, your home directory is /Users/zzamboni 2
Note that
gsubreturns the modified string as its first return value, and the number of replacements it made as the second (2in the example above). If you don’t need the number, you can simply ignore it (you don’t even need to assign it). Also note thatgsubdoes not modify the original string, only returns a copy with the changes:> b = a:gsub("%$(%w+)", os.getenv) > b Hello zzamboni, your home directory is /Users/zzamboni > a Hello $USER, your home directory is $HOME
You know now enough Lua to start being productive with Hammerspoon. You’ll pick up more details as you play with it. If you need more information, I can recommend the following resources, which I have found useful:
- The Lua 5.3 Reference Manual, available at the official Lua website.
- The Lua Wiki, a community-maintained wiki with many descriptions, tips, examples and tutorials.
Have fun!
I am happy to announce the first release of my new book “Learning Hammerspoon”, a book devoted to using Hammerspoon to make using your Mac easier, faster and more fun.
You can find more information, read a free sample, and purchase it at LeanPub.
{{{leanpubbook(learning-hammerspoon,)}}}
This book is based partly on blog posts published in my blog, but with a lot of new material. In this first release, the following chapters are finished:
- Getting started with Hammerspoon
- Using Spoons in Hammerspoon
- Just enough Lua to be productive with Hammerspoon
- Using and extending Seal
It is not complete yet, but already includes a lot of useful information. I will be updating it frequently, and of course you get all updates for free.
I look forward to your feedback! Feel free to message me through the /Email the author/ page at LeanPub.
{{{leanpubbook(learning-hammerspoon,style=”float:right”)}}}
I am happy to announce the third release of my book Learning Hammerspoon, which includes the following changes:
- A brand-new chapter: Writing your own extensions and Spoons. It is not fully finished yet, but you can find already a complete, working example of a new spoon, which you can use as a starting point for creating your own. I will continue adding more details over the next few days, but in the meantime please let me know how you like it: is the example meaningful? Are the instructions easy to follow?
- Two new introductory sections: The Hyper Key, about useful ways of defining a common set of modifiers for your Hammerspoon keybindings; and Keeping private information separate, about loading separate files into your Hammerspoon configuration.
- Multiple overall improvements in terms of wording, figures and formatting.
- One invisible backend change, but which bears mentioning: this book is now generated using Markua sources instead of the default Leanpub Markdown format used before. This has very little impact on the final book as you see it, but it will make it easier in the future to handle more complex formatting as needed.
I hope you enjoy it! As usual, you can find more information, read a free sample, and purchase it at LeanPub.
{{{leanpubbook(learning-hammerspoon,style=”float:right”)}}}
I am happy to announce a new release of my book Learning Hammerspoon!
This release includes a brand new chapter, Hammerspoon Cookbook, Tips and Tricks, which contains practical examples of doing interesting and useful things with Hammerspoon, as well as some tips to improve your configuration.
Also included are multiple other fixes, clarifications and minor improvements.
I hope you enjoy it! As usual, you can find more information, read a free sample, and purchase it at LeanPub.
(Updated on March 19th, 2018 to use the new Elvish Package Manager)
The bash shortcuts (maybe older? I’m not sure in which shell these originated) for “last command” (!!) and “last argument of last command” (!$) are, for me at least, among the most strongly imprinted in my muscle memory, and I use them all the time. Although these shortcuts are not available in Elvish by default, they are easy to implement. I have written a module called bang-bang which you can readily use as follows:
- Use epm to install my elvish-modules package (you can also add this to your
rc.elvfile to have the package installed automatically if needed):use epm epm:install github.com/zzamboni/elvish-modules - In your
rc.elv(see mine as an example), add the following to load thebang-bangmodule and to set up the appropriate keybindings:use github.com/zzamboni/elvish-modules/bang-bang
That’s it! Start a new shell window, and test how command-history mode can be invoked by the ! key. Assuming your last command was ls -l ~/.elvish/rc.elv, when you press ! you will see the following:
bang-lastcmd [A C] _ ! ls -l .elvish/rc.elv 0 ls 1 -l 2/$ .elvish/rc.elv Alt-! !
If you press ! again, the whole last command will be inserted. If you press $ (or 2), only the last argument will be inserted. You can insert any other component of the previous command using its corresponding number. If you want to insert an exclamation sign, you can press Alt-!.
Note that by default, Alt-! will also be bound to trigger this mode, so you can fully replace the default “last command” mode in Elvish.
Have fun with Elvish!
Like other Unix shells, Elvish has advanced command-argument completion capabilities. In this article I will explore the existing completions, and show you how you can create your own (and contribute them back to the community).
There is a growing body of shell completions that you can simply load and use.
Elvish has a still-small but growing collection of completions that have been created by its users. These are a few that I know of (let me know if you know others!):
- My own zzamboni/elvish-completions package, which contains completions for git (providing automatically-generated completions for all commands and their options, plus hand-crafted argument completions for many of them), ssh, vcsh, cd, and a few of Elvish’s built-in functions and modules. It also contains comp, a framework for building completers, which we will explore in more detail below. To use any of these modules, you just need to install the elvish-completions package, and then load the modules you want. For example:
epm:install &silent-if-installed github.com/zzamboni/elvish-completions use github.com/zzamboni/elvish-completions/vcsh use github.com/zzamboni/elvish-completions/cd use github.com/zzamboni/elvish-completions/ssh use github.com/zzamboni/elvish-completions/builtins use github.com/zzamboni/elvish-completions/git
- xiaq’s edit.elv/compl/go.elv, which provides extensive hand-crafted completions for
go. You can also install this one as an Elvish package:epm:install &silent-if-installed github.com/xiaq/edit.elv use github.com/xiaq/edit.elv/compl/go go:apply
- occivink’s completers.elv file, which contains completers for
kak,ssh,systemctl,ffmpegand a few other commands. - Tw’s completer/ files, which contains completions for
adb,gitandssh. - SolitudeSF’s completers.elv file, which contains completers for
cd,kak,kitty,git,man,pkilland quite a few other commands.
As of this writing, there is no “official” collection of Elvish completions, so feel free to look at the existing ones and choose/use the ones that work best for you.
Since the collection is not yet very big, it’s likely you will want to build your own completions. This is what the next section is about.
Elvish has a simple but powerful argument-completion mechanism. You can find the full documentation in the Elvish reference, but let’s take a quick look here.
Command argument completion in Elvish is implemented by functions stored inside $edit:completion:arg-completer. This variable is a map in which the indices are command names, and the values are functions which must receive a variable number of arguments. When the user types cat Space Tab, the function stored in $edit:completion:arg-completer[cat] (if any) is called, as follows:
$edit:completion:arg-completer[cat] cat ''The function receives the full current command line as arguments, including the current argument, which might be empty as in the example above, or be a partially typed string. For example, if the user types cat f Tab, the completer function will be called like this:
$edit:completion:arg-completer[cat] cat 'f'The completion function must use its arguments to determine the appropriate completions at that point, and return them by one of the following methods (which can be combined):
- Output the completions to stdout, one per line;
- Output the completions to the data stream (using
put); - Output the completions using the
edit:complex-candidatecommand, which can additionally specify a suffix to append to the completion in the completion menu or in the returned value, and a style to use (as accepted byedit:styled). The full syntax ofedit:complex-candidateis as follows:edit:complex-candidate &code-suffix='' &display-suffix='' &style='' $string
$stringis the option to display;&code-suffixindicates a suffix to be appended to the completion string when the user selects it;&display-suffixindicates a suffix to be shown in the completion menu (but which is not returned as part of the completion); and&styleindicates a text style to use in the completion menu.
Keep in mind that the options returned by the completion function are additionally filtered by what the user has typed so far. This means that the last argument can usually be ignored, since Elvish will automatically do the filtering. An exception to this is if you want to return different types of things depending on what the user has typed already. For example, if the last argument start with -, you may want to return the possible command-line options, and return regular argument completions otherwise.
Example #1: A very simple completer for the brew command:
edit:completion:arg-completer[brew] = [@cmd]{
len = (count $cmd)
if (eq $len 2) {
if (has-prefix $cmd[-1] -) {
put '--version' '--verbose'
} else {
put install uninstall
}
} elif (eq $len 3) {
brew search | eawk [l @f]{ put $@f }
}
}If the function receives two arguments, we check to see if the last argument begins with a dash. If so, we return the possible command-line options, otherwise we return the two commands install and uninstall. If we receive three arguments (i.e. we are past the initial command), we return the list of possible packages to install or uninstall.
You may noticed that there are many cases that this simple function does not handle correctly. For example, if you type brew --verbose Space Tab, you get the list of packages as completion, which does not make sense at that point. We will look at more complex and complete completion functions next.
The first step to more complex completions is the edit:complete-getopt command, which allows us to specify a sequence of positional completion functions. The general syntax of the command is:
edit:complete-getopt $args $opts $handlersPlease see its documentation for a full description of the arguments.
Example #2: The completer for brew shown before can be specified like this:
edit:completion:arg-completer[brew] = [@cmd]{
edit:complete-getopt $cmd[1:] \
[[&long=version] [&long=verbose]] \
[
[_]{ put install uninstall }
[_]{ brew search | eawk [_ @f]{ put $@f } }
...
]
}This new completer overcomes a few of the limitations in our previous attempt. For one, the install and uninstall commands are now properly completed even if you specify options before. Furthermore, the ... at the end of the handler list indicates that the previous one (the package names) will be repeated for all further arguments - this makes it possible to complete multiple package names to install or uninstall. However, it still has some limitations! For example, it will give you all existing packages as possible arguments to uninstall, which only accepts already installed packages.
In addition to complete-getopt, Elvish includes a few other functions to help build completers:
edit:complete-filenameproduces a listing of all the files and directories in the directory of its argument, and is the default completion function when no other completer is specified. See its documentation for full details.edit:complete-sudoprovides completions for commands likesudowhich take a command as their first argument. It is the default completer for thesudocommand, so that if you typesudoSpaceTab, you get a list of all the commands on your execution path. It can be reused for other commands, for exampletime:edit:completion:arg-completer[time] = $edit:complete-sudo~
Finally, note that if $edit:completion:arg-completer[''] exists, it will be called as a fall-back completer if no command-specific argument completer exists. You can see that the default completer is edit:complete-filename, as mentioned before:
~> put $edit:completion:arg-completer['']
▶ $edit:complete-filename~With the tools you know so far, you can already create fairly complex completers. In the next section, we will explore comp, an external library I wrote to make it easier to specify complex completion trees.
The built-in completion functions make it possible to build any completer you want. However, you might realize that for more complex cases, the specifications can be quite complex. For this reason, I wrote the comp library as a framework to more easily specify completion functions. The basic Elvish mechanisms and functions are still used in the backend, so you can rest assured about their compatibility with the basic mechanisms.
As a first step, if you haven’t done so already, you should install the elvish-completions package using epm:
use epm
epm:install github.com/zzamboni/elvish-completionsFrom the file where you will define your completions (or from your interactive session if you just want to play with it), load the comp module:
use github.com/zzamboni/elvish-completions/compThe main entry points for this module are comp:item, comp:sequence and comp:subcommands. Each one receives a single argument containing a “completion definition”, which indicates how the completions will be produced. Each one receives a different kind of completion structure, and returns a ready-to-use completion function, which can be assigned directly to an element of $edit:completion:arg-completer. A simple example:
edit:completion:arg-completer[foo] = (comp:item [ bar baz ])If you type this in your terminal, and then type foo<space> and press Tab, you will see the appropriate completions:
> foo <Tab> COMPLETING argument _ bar baz
To create completions for new commands, your main task is to define the corresponding completion definition. The different types of definitions and functions are explained below, with examples of the different available structures and features.
Note: the main entry points return a ready-to-use argument handler function. If you ever need to expand a completion definition directly (maybe for some advanced usage), you can call comp:-expand-item, comp:-expand-sequence and comp:-expand-subcommands, respectively. These functions all take the definition structure and the current command line, and return the appropriate completions at that point.
We now look at the different types of completion definitions understood by comp.
The base building block is the “item”, can be one of the following:
- An array containing all the potential completions (it can be empty, in which case no completions are provided). This is useful for providing a static list of completions.
- A function which returns the potential completions (it can return nothing, in which case no completions are provided). The function should have one of the following arities, which affect which arguments will be passed to it (other arities are not valid, and in that case the item will not be executed):
- If it takes no arguments, no arguments are passed to it.
- If it takes a single argument, it gets the current (last) component of the command line
@cmd; this is just like the handler functions understood by theedit:complete-getoptcommand. - If it takes a rest argument, it gets the full current command line (the contents of
@cmd); this is just like the functions assigned to$edit:completion:arg-completer.
Example #3: a simple completer for cd
In this case, we define a function which receives the current “stem” (the part of the filename the user has typed so far) and offers all the relevant files, then filters those which are directories, and returns them as completion possibilities. We pass the function directly as a completion item to comp:-expand.
fn complete-dirs [arg]{ put {$arg}* | each [x]{ if (-is-dir $x) { put $x } } }
edit:completion:arg-completer[cd] = (comp:item $complete-dirs~)For file and directory completion, you can use the utility function comp:files instead of defining your own function (see Utility functions). comp:files uses edit:complete-filename in the backend but offers a few additional filtering options:
edit:completion:arg-completer[cd] = (comp:item [arg]{ comp:files $arg &dirs-only })Completion items can be aggregated in a sequence of items and used with the comp:sequence function when you need to provide different completions for different positional arguments of a command, including support for command-line options at the beginning of the command (comp:sequence uses edit:complete-getopt in the backend, but provides a few additional convenient features). The definition structure in this case has to be an array of items, which will be applied depending on their position within the command parameter sequence. If the the last element of the list is the string ... (three periods), the next-to-last element of the list is repeated for all later arguments. If no completions should be provided past the last argument, simply omit the periods. If a sequence should produce no completions at all, you can use an empty list []. If any specific elements of the sequence should have no completions, you can specify { comp:empty } or [] as its value.
If the &opts option is passed to the comp:sequence function, it must contain a single definition item which produces a list of command-line options that are allowed at the beginning of the command, when no other arguments have been provided. Options can be specified in either of the following formats:
- As a string which gets converted to a long-style option; e.g.
allto specify the--alloption. The string must not contain the dashes at the beginning. - As a map in the style of
complete-getopt, which may contain the following keys:shortfor the short one-letter option;longfor the long-option string;descfor a descriptive string which gets shown in the completion menu;arg-mandatoryorarg-optional: either one but not both can be set to$trueto indicate whether the option takes a mandatory or optional argument;arg-completercan be specified to produce completions for the option argument. If specified, it must contain completion item as described in Items, and which will be expanded to provide completions for that argument’s values.
Simple example of a completion data structure for option -t (long form --type), which has a mandatory argument which can be elv, org or txt:
[ &short=t &long=type &desc="Type of file to show" &arg-mandatory=$true &arg-completer= [ elv org txt ] ]
Note: options are only offered as completions when the use has typed a dash as the first character. Otherwise the argument completers are used.
Example #4: we can improve on the previous completer for cd by preventing more than one argument from being completed (only the first argument will be completed using complete-dirs, since the list does not end with ...):
edit:completion:arg-completer[cd] = (comp:sequence [ [arg]{ comp:files $arg &dirs-only }])Example #5: a simple completer for ls with a subset of its options. Note that -l and -R are only provided as completions when you have not typed any filenames yet. Also note that we are using comp:files to provide the file completions, and the ... at the end of the sequence to use the same completer for all further elements.
ls-opts = [
[ &short=l &desc='use a long listing format' ]
[ &short=R &long=recursive &desc='list subdirectories recursively' ]
]
edit:completion:arg-completer[ls] = (comp:sequence &opts=$ls-opts [ $comp:files~ ... ])Example #6: See the ssh completer for a real-world example of using sequences.
Finally, completion sequences can be aggregated into subcommand structures using the comp:subcommands function, to provide completion for commands such as git, which accept multiple subcommands, each with their own options and completions. In this case, the definition is a map indexed by subcommand names. The value of each element can be a comp:item, a comp:sequence or another comp:subcommands (to provide completion for sub-sub-commands, see the example below for vagrant). The comp:subcommands function can also receive the &opts option to generate any available top-level options.
Example #7: let us reimplement our completer for the brew package manager, but now with support for the install, uninstall and cat commands. install and cat gets as completions all available packages (the output of the brew search command), while uninstall only completes installed packages (the output of brew list). Note that for install and uninstall we automatically extract command-line options from their help messages using the comp:extract-opts function (wrapped into the -brew-opts function), and pass them as the &opts option in the corresponding sequence functions. Also note that all &opts elements get initialized at definition time (they are arrays), whereas the sequence completions get evaluated at runtime (they are lambdas), to automatically update according to the current packages. The cat command sequence allows only one option. The load-time initialization of the options incurs a small delay, and you could replace these with lambdas as well so that the options are computed at runtime. Note also the usage of the comp:decorate function to colorize the package names in different colors for each command.
fn -brew-opts [cmd]{
brew $cmd -h | take 1 | \
comp:extract-opts ®ex='--(\w[\w-]*)' ®ex-map=[&long= 1]
}
brew-completions = [
&install= (comp:sequence &opts= [ (-brew-opts install) ] \
[ { brew search | comp:decorate &style=green } ... ]
)
&uninstall= (comp:sequence &opts= [ (-brew-opts uninstall) ] \
[ { brew list | comp:decorate &style=red } ... ]
)
&cat= (comp:sequence [{ brew search | comp:decorate &style=blue }])
]
edit:completion:arg-completer[brew] = (comp:subcommands \
&opts= [ version verbose ] $brew-completions
)Note that in contrast to our previous brew completer, this definition is much more expressive, accurate, and much easier to extend.
Example #8: a simple completer for a subset of vagrant, which receives commands which may have subcommands and options of their own. Note that the value of &up is a comp:sequence, but the value of &box is another comp:subcommands which includes the completions for box add and box remove. Also note the use of the comp:extract-opts function to extract the command-line arguments automatically from the help messages. The output of the vagrant help messages matches the default format expected by comp:extract-opts, so we don’t even have to specify a regular expression like for brew.
Tip: note that the values of &opts are functions (e.g. { vagrant-up -h | comp:extract-opts }) instead of arrays (e.g. ( vagrant up -h | comp:extract-opts )). As mentioned in the previous example, both are valid, but in the latter case they are all initialized at load time (when the data structure is defined), which might introduce a delay, particularly with more command definitions. By using functions the options are only extracted at runtime when the completion is requested. For further optimization, vagrant-opts could be made to memoize the values so that the delay only occurs the first time.
vagrant-completions = [
&up= (comp:sequence [] \
&opts= { vagrant up -h | comp:extract-opts }
)
&box= (comp:subcommands [
&add= (comp:sequence [] \
&opts= { vagrant box add -h | comp:extract-opts }
)
&remove= (comp:sequence [ { \
vagrant box list | eawk [_ @f]{ put $f[0] } \
} ... ] \
&opts= { vagrant box remove -h | comp:extract-opts }
)
])]
edit:completion:arg-completer[vagrant] = (comp:subcommands \
&opts= [ version help ] $vagrant-completions
)Example #9: See the git completer for a real-world subcommand completion example, which also shows how extensively auto-population of subcommands and options can be done by extracting information from help messages.
The comp module includes a few utility functions, some of which you have seen already in the examples.
comp:decorate maps its input through edit:complex-candidate with the given options. Can be passed the same options as edit:complex-candidate. In addition, if &suffix is specified, it is used to set both &display-suffix and &code-suffix. Input can be given either as arguments or through the pipeline:
> comp:decorate &suffix=":" foo bar
▶ (edit:complex-candidate foo &code-suffix=: &display-suffix=: &style='')
▶ (edit:complex-candidate bar &code-suffix=: &display-suffix=: &style='')
> put foo bar | comp:decorate &style="red"
▶ (edit:complex-candidate foo &code-suffix='' &display-suffix='' &style=31)
▶ (edit:complex-candidate bar &code-suffix='' &display-suffix='' &style=31)comp:extract-opts takes input from the pipeline and extracts command-line option data structures from its output. By default it understand the following common formats:
-o, --option Option description
-p, --print[=<what>] Option with an optional argument
--select <type> Option with a mandatory argument
Typical use would be to populate an &opts element with something like this:
comp:sequence &opts= { vagrant -h | comp:extract-opts } [ ... ]The regular expression used to extract the options can be specified with the ®ex option. Its default value (which parses the common formats shown above) is:
®ex='^\s*(?:-(\w),?\s*)?(?:--?([\w-]+))?(?:\[=(\S+)\]|[ =](\S+))?\s*?\s\s(\w.*)$'The mapping of capture groups from the regex to option components is defined by the ®ex-map option. Its default value (which also shows the available fields) is:
®ex-map=[&short=1 &long=2 &arg-optional=3 &arg-mandatory=4 &desc=5]At least one of short or long must be present in regex-map. The arg-optional and arg-mandatory groups, if present, are handled specially: if any of them is not empty, then its contents is stored as arg-desc in the output, and the corresponding arg-mandatory / arg-optional is set to $true.
If &fold is $true, then the input is preprocessed to join option descriptions which span more than one line (the heuristic is not perfect and may not work in all cases, also for now it only joins one line after the option).
So you have created a brand-new completion function and would like to share it with the Elvish community. Nothing could be easier! You have two main options:
- Publish them on your own. For example, if you put your
.elvfiles into their own repository in GitHub or Gitlab, they are ready to be installed and used using epm. - Contribute it to an existing repository (for example elvish-completions). Just add your files, submit a pull request, and you are done.
I hope you have found this tutorial useful. Please let me know in the comments if you have any questions, feedback or if you find something that is incorrect.
Now, go have fun with Elvish!
2019/09/12: Updated with some new links and information based on my later usage of Elvish. See https://zzamboni.org/tags/elvish for other things I have written about it.
I’m always on the lookout for new toys, particularly if they make my work more productive or enjoyable. For a couple of months now I have been using a new Unix shell called Elvish as my default login shell on macOS, and I love it. It’s a young project but very usable and with some very nice features.
Here are a few of the things I like about it:
- The Elvish language is clean and powerful, with clear syntax. It supports name spaces, exception handling and proper data structures, including lists and maps.
- It has a rich built-in library of functions, including string manipulation, regex matching, JSON parsing, etc.
- Commands and functions can output two types of streams: bytes (what we usually see as standard output/error) and data (data values, potentially containing structured data), which makes it very flexible.
- It has very nice interactive features, including as-you-type syntax checking and highlighting, asynchronous prompt themes, support for custom completions, a built-in command-line file browser, prompt hooks, configurable keybindings, and many others.
- Exceptions! In Elvish, if a command exits with a non-zero code, it generates an exception that stops the execution of the current script or function. This breaks completely with the Unix-shell tradition of silently setting return codes, and it takes a while to get used to it, but once you do it’s a very powerful idea, as it forces you to think about failures and to plan for them in your code.
- It is extensively documented. Take a look at https://elv.sh and you will see.
Elvish is very powerful, but it’s different enough from other shells that it’s worth your time to read through some of the documentation when you start. I would recommend the Some Unique Semantics page, which assumes you know other shells already. From there you can move to some of the other tutorials and reference documentation.
Of course, Elvish is not without its quirks and drawbacks. None of these has been a deal-breaker for me, but just for completeness:
- It’s very young. Occasionally I still encounter crashes, but they are few and far between, and the developer is always very responsive. Also, the language is still subject to change and there are still backwards-incompatible changes with relative frequency. If you are looking for absolute stability, it’s not yet ready. (2019/06/12 update: it has been months (probably at least a year) without encountering a single crash)
- Its language syntax is still a bit quirky. Spacing is sometimes important. For example, in the
if=/=elseconstruct, the ”} else {” has to be just like that—with spaces, and in a single line, for it to be recognized by the parser. (2019/06/12 update: the reason for this is explained in Effective Elvish) - The data/byte separation is not fully clear (at least to me) yet. Sometimes the data stream can be interpreted as bytes as well, sometimes it does not. The language is still evolving, so I am sure this will become clearer in the future. (2019/06/12 update: this is just a matter of getting used to it)
- There is not yet a large body of code/scripts you can use. This is very noticeable in completions—while Elvish supports completions, there are very few implementations. Coming from Fish, which has an impressive library of custom completions out-of-the-box, this lack is very noticeable, particularly with complex commands like
git. (2019/06/12 update: there is still not as much as for Fish or other shells, but steadily increasing. See Awesome Elvish for a list)
I am very happy with Elvish, and if you are interested in this sort of thing, I encourage you to take a look. If you need a starting point, you can use my configuration files at as an example of the kind of things you can do with it.
I will post more Elvish tips and tricks over time.
{{{leanpubbook(courses/leanpub/cissp-training,style=”float:right” height=”400”)}}}
I am happy to finally reveal a project I’ve been working on for some time: my new online course /CISSP Training/, now available on Leanpub.
There are multitude of CISSP courses, books and materials out there, but this one is special: it is a collection of topics I found useful when preparing for my own certification, complemented with examples, exercises and additional information based on my own experience and knowledge. I have been improving and fine tuning it for months now, and I’m happy to finally release it. The course is content-complete for all 8 (ISC)2 Common Body of Knowledge domains, although I continue improving formatting, structure and adding exercises (quizzes and exercises for the CBK domains 1-3 are there, domains 4-8 are coming).
To celebrate its release, you can get the course, until the end of January with a 25% discount off its regular minimum price. Just click here to get the discount!
If you are thinking of, or already preparing for, taking the CISSP exam, I am sure you will find this course useful. Get it now!
My blogging has seen multiple iterations over the years, and with it, the tools I use have changed. At the moment I use a set of tools and workflows which make it very easy to keep my blog updated, and I will describe them in this post. In short, they are:
- Writing: Emacs, org-mode
- Exporting: ox-hugo
- Publishing: Hugo and Netlify
Let’s take a closer look at each of the stages.
I have been using Emacs for almost 30 years, so its use for me is second nature. For some time I’ve been using org-mode for writing, blogging, coding, presentations and more. I am duly impressed. I have been a fan of the idea of literate programming for many years, and I have tried other tools before (most notably noweb, which I used during grad school for many of my homeworks and projects), but org-mode is the first tool I have encountered which seems to make it practical. Here are some of the resources I have found useful in learning it:
- Howard Abrams’ Introduction to Literate Programming, which got me jumpstarted into writing code documented with org-mode.
- Nick Anderson’s Level up your notes with Org, which contains many useful tips and configuration tricks.
- Sacha Chua’s Some tips for learning Org Mode for Emacs, her Emacs configuration and many of her other articles.
- Rainer König’s OrgMode Tutorial video series.
You can see some examples in my “literate config files” series, and all recent posts in this blog are written using org-mode (you can find the source file in GitLab).
Over time I have tweaked my Emacs configuration to make writing with org-mode more pleasant. You can see my Emacs configuration for reference.
So, I write posts using Emacs, in org-mode markup. What’s next?
When I first started writing my blog posts in org-mode, I relied on Hugo’s built-in support for it, which allows you to simply create posts in .org files instead of .md and have them parse in org-mode format. Unfortunately, the support is not perfect. Hugo relies on the go-org library which, while quite powerful, does not support the full org-mode markup capabilities, so many elements are not rendered or processed properly.
Happily, I discovered ox-hugo, an org-mode exporter which produces Hugo-ready Markdown files from the org-mode source, from which Hugo can produce the final HTML output. This is a much better arrangement, because each component handles only its native format: ox-hugo processes the org-mode source with the full support of org-mode and Emacs, and Hugo processes Markdown files, which are its native input format. You can use the full range of org-mode markup in your posts, and they will be correctly converted to their equivalents in Markdown. Furthermore, your source files remain output-agnostic, as you can still use all other org-mode exporters if you need to produce other formats.
Ox-hugo supports two ways of organizing your posts: one post per org file, and one post per org subtree. In the first one, you write a separate org file for each post. In the second, you keep all your posts in a single org file, and specify (through org-mode properties) which subtrees should be exported as posts. The latter is the recommended way to organize posts. At first I was skeptical - who wants to keep everything in a single file? However, as I have worked more with it, I have come to realize its advantages. For one, it makes it easier to specify post metadata - for example, I have defined sections in my org-mode source file for certain frequent topics, and those are tagged accordingly in the org source. When I create posts as subtrees of those sections, they inherit the top-level tags automatically, as well as any other properties, which I use, for example, to define the header images used in the posts. Having all posts in a single file also makes it easier to share other content, such as org macro definitions, ox-hugo configuration options, etc.
Note that ox-hugo is not limited to exporting blog posts, but any content processed by Hugo. For example, my org source file also includes all the static pages in my web site - they are differentiated from blog posts simply by the Hugo section to which they belong, which is defined using the HUGO_SECTION property in my Org file.
Since the full power of org markup is available when using ox-hugo, you can do very interesting things. For example, all the posts in my Literate Config Files category are automatically updated every time I export them with the actual, real content of the corresponding config file, which I also keep in org format. There is a lot of hidden power in org-mode and ox-hugo. My recommendation is to go through the source files for some of the websites listed in ox-hugo’s Real World Examples section. I have learned a lot by reading through the source files for the ox-hugo website itself.
Once you have some contents in your Org file, you can export them into Markdown files. For this, use the standard Org export dispatcher (bound to C-c C-e by default) and choose [H] Export to Hugo-compatible Markdown / [A] All subtrees (or File) to Md file(s) options (you may choose other options for course, but this one exports the whole file). Ox-hugo knows the default structure expected by Hugo (a top-level content/ directory in which you have directories for each section), so there’s usually not much to do other than point ox-hugo to where your top-level Hugo directory is, using the HUGO_BASE_DIR property.
Hugo has extensive capabilities and it is beyond the scope of this article to show you how to use it, but it has very good documentation which I would urge you to peruse to learn more about it. Feel free to peruse my setup for ideas.
Normally I run hugo locally to make sure the export is OK, particularly when I’m tweaking with my sites’ theme or settings. To do this, you can simply run:
hugo serverAnd browse to http://localhost:1313.
Finally! Once you are happy with the results, we have come to the point of publishing the website. I used GitHub Pages for a long time, but nowadays I use Netlify, which does a great job of hosting websites. After connecting Netlify to my website’s GitLab repository, all I have to do is push the files, and Netlify takes care of running Hugo on them and publishing the results. Netlify even handles the DNS records and SSL certificates for my domain!
Netlify has impressive capabilities, but for a basic website like mine, a mostly default setup works well. This is what the Build Settings look like:
Note that I change Hugo’s publishDir parameter from its default value of public to docs, and configure Netlify to match (note that this could also be configured in netlify.toml, below). This is done by specifying the parameter in Hugo’s =config.toml= file:
publishDir = "docs"My repository contains a netlify.toml file which is used to configure some Hugo environment variables, and to specify the version of Hugo to use:
I also keep an Elvish script for automatically updating this file to the version of Hugo currently installed on my laptop. Whenever I update Hugo locally, I test my website using hugo server, and then run this to instruct Netlify to upgrade to the latest version as well:
Finally, Hugo aliases can be handled via Netlify redirects by following the instructions from this blog post to automatically populate the redirects configuration from the Hugo source files.
That’s it! With this setup, I can write all the contents for my website in Org-mode, and the rest is handled automatically by the tools. This makes it very easy to keep my website updated. And all these tools are available for free!
I hope you find this useful. Let me know in the comments if you have any questions.
{{{leanpubbook(emacs-org-leanpub,style=”float:right” height=”380”)}}}
A new release of Publishing with Emacs, Org-mode and Leanpub is out!
This second release includes many improvements and new content, including:
- A whole new chapter, The workflow, which covers detailed techniques for writing, exporting, previewing and publishing your book.
- A new section, Code block execution and output processing, which shows how you can have code within your document which is executed on the fly, and which you can use to generate content within your book.
- Two new appendices, the first one containing the source code for the overall workflow diagram, and the second one containing samples of all the different block types supported by
ox-leanpub. - Many other changes, including new instructions on configuring Emacs Doom, innumerable wording, structure and clarity improvements, and a lot more.
Hope you enjoy it!
To celebrate its release, click the following link to get the book with a 50% discount from its suggested price, for a limited time (until December 11, 2020): Get it now!
{{{leanpubbook(emacs-org-leanpub,style=”float:right” height=”380”)}}}
Publishing your words has never been easier than it is today. Blogging means you can have your words read by thousands of people within minutes of writing them. Even publishing a book has become considerably easier through self publishing. There are many tools and publishers that allow you to get started for little or no money. Still, getting started can be confusing, and that is what this book is about.
In this book, I will show you the workflow and tools I use to publish my books. The three main tools involved are:
- The GNU Emacs editor together with Org-mode for writing, editing and exporting your text;
- GitHub or Bitbucket to store your book files.
- Leanpub for typesetting, previewing, publishing and selling your work.
To illustrate the process and provide you with a starting point, the source repository for this book is available at https://github.com/zzamboni/emacs-org-leanpub. I am populating the repository live as I write this book.
I hope you enjoy it! Your feedback, as usual, is welcome.
{{{leanpubbook(lit-config,style=”float:right”)}}}
I am happy to announce the new release of my new book “Literate Config”, devoted to the use of Literate Programming for writing and documenting configuration files with Emacs and org-mode.
This is the first in my planned “Geek Booklets” series, which will include short texts about specific topics. You can get at LeanPub, like all my other books. You can get it for free or for a price of your choosing. You can also read it online.
I hope you enjoy it! Your feedback, as usual, is welcome.
{{{leanpubbook(utilerias-unix,style=”float:right”)}}}
(I usually write in English, but this new book is in Spanish, so the announcement is also in Spanish. An English version of this book is in the works)
Me complace anunciar la primera edición de mi nuevo libro, y mi primer libro en Español, “Utilerías de Unix”, en el que podrás aprender a utilizar algunas de las herramientas más útiles que se encuentran en un sistema Unix/Linux para automatizar y hacer más eficientemente todo tipo de tareas.
Puedes obtener más información, leer una muestra gratuita y comprarlo en LeanPub.
¡Espero lo disfrutes! Favor de enviarme tus comentarios a través de la página /Escribe al Autor/ en LeanPub.
I am the author of two books: /Learning CFEngine/ and /Learning Hammerspoon/, both self-published using Leanpub. The source of my books is kept in GitHub repositories. In this post I will show you how I use the Leanpub API together with Hammerspoon and CircleCI as part of my workflow, to automate and monitor the building, previewing and publishing of my books.
file:images/hammerspoon-github-circleci-leanpub-transp.jpg
First, some basic concepts about the Leanpub API. See the documentation for the full details, I’m only mentioning clear some things you need to know to understand the rest of this post.
- Book slug: Each Leanpub book is identified by a slug, which is basically an unique author-chosen identifier for the book. The book slug is included in the book’s Leanpub URL. For example, the URL for Learning Hammerspoon is https://leanpub.com/learning-hammerspoon, therefore its slug is
learning-hammerspoon. The slug can be changed by the author as part of the book configuration, and is used in all the API calls to identify the book for which an operation should be performed.
- API key: Every Leanpub author gets an /API key/, which is a randomly-generated string of characters which is used as an authentication token. The API key needs to be provided on most Leanpub API calls (some query operations are allowed without a key).
- Build types: The Leanpub API allows you to trigger several types of build operations on a book:
- Preview builds all the formats supported by Leanpub (PDF, ePub, Mobi), using the whole book as defined in the
Book.txtfile. - Subset preview builds only the PDF version of a book, from a subset of files defined in the
Subset.txtfile in your repository. - File preview builds only a segment of text you need to provide as part of the API call. I do not use this operation in my workflows.
- Publish builds and publishes a new version of the book. Publishing means that it becomes the version available for purchase. Optionally, when publishing a new release you can send out an email with release notes to people who have already purchased the book (in any case, the new version of the book also becomes available for them to download).
- Preview builds all the formats supported by Leanpub (PDF, ePub, Mobi), using the whole book as defined in the
- Book writing mode refers to the source from which Leanpub gets the text for your book. I use “Git and Github”, but these techniques should work equally well with BitBucket or any other platforms that can trigger a webhook when your text is updated.
As an initial step, I wrote some shell scripts to trigger and watch the progress of Leanpub builds by hand. I use the Elvish shell, and my scripts are published as the leanpub Elvish module. These allow you to trigger book builds (only preview and subset builds, no publishing) by hand, and also to watch the progress of any operation. If you use Elvish, you can install the module like this:
use epm
epm:install github.com/zzamboni/elvish-modules
use github.com/zzamboni/elvish-modules/leanpubThen, whenever you commit changes to your text, you can trigger a build and watch its progress like this:
leanpub:preview # or leanpub:subset
leanpub:watchYou can combine build-and-watch in a single command:
leanpub:preview-and-watch
leanpub:subset-and-watchAfter a while using these scripts, I thought I would put some work on improving both the aesthetics and the functionality of my automation. The next sections are what I came up with.
The first step is to get rid of the need to run those “watch” scripts, which produce raw JSON output from the Leanpub API, and use nice macOS notifications to track the activity, like these:
These are produced by a Spoon I wrote called Leanpub. You can install, load and configure it using the SpoonInstall spoon.
For example, in my configuration I have the following code to configure the spoon to watch for both of my books:
Install:andUse("Leanpub",
{
config = {
-- api_key = "my-api-key",
watch_books = {
{ slug = "learning-hammerspoon" },
{ slug = "learning-cfengine" }
}
},
start = true
})Note that you also need to specify your Leanpub API key, which you can get and manage in the Author / Your API Key in Leanpub:
file:images/leanpub-api-key.png
This file in turn gets loaded into my main config file as follows:
local localfile = hs.configdir .. "/init-local.lua"
if hs.fs.attributes(localfile) then
dofile(localfile)
endReload your Hammerspoon configuration. Now when you trigger a preview or publish (for example, using the scripts above), you will after a few seconds start seeing the corresponding notifications.
The most basic way of automatically triggering builds is by using a webhook to trigger the Leanpub API directly. This is described in your book’s “Getting Started” page, which you can access at https://leanpub.com/YOUR_BOOK/getting_started (replacing YOUR_BOOK with your book’s slug). This works well, but the downside is that the webhook is “hardcoded” so you can only trigger a fixed type of build per webhook (e.g. subset or regular preview). This means that if you want to trigger a different type of build, you need to keep multiple webhooks defined, and activate the one you want by hand:
file:images/github-webhooks.png
Looking for ways to further automate the preview and publish workflow of my books, I came across CircleCI, a popular CI/CD platform which is easy to use and allows creation of libraries called “orbs” to encapsulate more complex behaviors. I wrote an orb called zzamboni/leanpub for automating interactions with the Leanpub API. With it, you can set up a build/preview/publish workflow which you can trigger directly from git.
Here’s my preferred workflow (you can build others as well using the leanpub orb, see below for ideas):
- A subset preview is triggered for every commit to the book’s repository. I keep
Subset.txtwith the same content asBook.txt, so a subset preview gives me a PDF-only build of my whole book (you could also modifySubset.txtbefore each commit depending on the part of the book you want to preview, but this is outside the scope of this article). This allows me to have a continuous PDF preview of any changes I make to my book. - A regular preview is triggered for commits that are tagged with a tag starting with
preview. This builds the book in all the formats supported by Leanpub (PDF, epub, mobi). This allows me to check the output in all formats when I’m doing finishing touches before publishing a new version of the book, or when I make major changes. - A silent publish is triggered for commits tagged with a tag starting with
silent-publish. This builds and publishes the book, but without sending out release notes. I use this for “minor” updates to the published book which I don’t think need to be widely announced (e.g. fixing typos and formatting, etc.) - Finally, a publish is triggered for commits tagged with a tag starting with
publish. This builds and publishes the book, but also sends out release notes to its readers. The release notes are taken from the description of the tag (if it is an annotated tag) or from the commit message of the tagged commit (if it’s a regular tag).
To implement this workflow, all you have to do is add a file .circleci/config.yml to your repository, containing the following:
version: 2.1
orbs:
leanpub: zzamboni/leanpub@0.1.1
# This tag-based book building workflow dispatches to the correct job
# depending on tagging
workflows:
version: 2
build-book:
jobs:
- leanpub/subset-preview:
filters:
tags:
ignore:
- /^preview.*/
- /^publish.*/
- /^silent-publish.*/
- leanpub/full-preview:
filters:
tags:
only: /^preview.*/
branches:
ignore: /.*/
- leanpub/auto-publish:
name: leanpub/silent-publish
auto-release-notes: false
filters:
tags:
only: /^silent-publish.*/
branches:
ignore: /.*/
- leanpub/auto-publish:
auto-release-notes: true
filters:
tags:
only: /^publish.*/
branches:
ignore: /.*/Once you have committed this file, you can enable CircleCI on it as follows:
- If you have defined static webhooks in your repository as described before, make sure to disable them.
- Login at https://circleci.com/ using your GitHub account.
- In the “Add projects” screen, choose your repository and click “Set Up Project”. Since you have already added the
config.ymlfile, you can skip that part and click on “Start building”. - The first build will fail because you have not provided your Leanpub API key yet: file:images/circleci-first-job-fail.png
- To fix this, you need to define an environment variable called
LEANPUB_API_KEYwithin your CircleCI project. Click on the project name, and then on the settings button at the top-left of the screen. Once there, select the “Environment Variables” section and enter the environment variable: file:images/circleci-add-leanpub-api-key.png - Now you can go to the “Workflows” screen and click on “Rerun” for your book’s workflow (alternatively, make a new commit on your git repository). Assuming you have the Leanpub Spoon installed as described before, you should see the notifications for your book’s build within a few seconds. file:images/circleci-successful-job.png
Using the techniques described above has made my book building and publishing much easier. I have been using them for a few weeks, and the latest release of /Learning Hammerspoon/ was published using this workflow already. I hope you find it useful as well! Please let me know in the comments if you have any questions or feedback.
When I was planning the reboot of my website, I seriously considered using Ghost. It has a very nice UI, beautiful and usable theme out of the box, and a very active community. Eventually I decided to use Hugo, but in the process discovered that it is possible to host a statically-generated Ghost website using GitHub Pages.
The general approach, described in multiple articles I found, is the following:
- Install and run Ghost locally
- Edit/create your content on your local install
- Create a static copy of your Ghost site by scraping it off the local install.
- Push the static website to GitHub Pages
So far, so good. It makes sense. But all those articles share one thing: they suggest using a tool called buster which, as far as I can tell, it’s a web-scraping tool, specialized for Ghost. However, it has a number limitations–for example, it does not slurp Ghost static pages, and it hasn’t been updated in a very long time (there’s a fork with somewhat more recent activity).
I found the use of buster puzzling, since there is a perfectly mature, functional and complete tool for scraping off a copy of a website: good old trusty wget. It is included (or easily available) in most Unix/Linux distributions, it is extremely powerful, and has features that make it really easy to create a local, working copy of a website (including proper translation of URLs). I used it to create the static archive of my old blog, BrT, when I decided to retire its WordPress backend years ago.
Another thing I found is that most instructions suggest storing only the generated website in your GitHub repository. I prefer keeping the source files and the generated website together. GitHub pages allows serving the website from different sources, including the repo’s gh-pages branch, its master branch, or the /docs directory in the master branch. Personally, I prefer using the /docs directory since it allows me to keep both the source and the generated website in the same place, without any branch fiddling.
So, without further ado, here are the detailed instructions. I ran these on my Mac, but most of them should work equally well on Linux or any other Unix-like system.
- Download Ghost (version 1.7.1 as of this writing):
cd ~/tmp # or some other suitable place wget https://github.com/TryGhost/Ghost/releases/download/1.7.1/Ghost-1.7.1.zip
- Unpack it in a suitable directory, initialize it as a GitHub repository and commit the Ghost plain install (to have a baseline with the fresh install):
mkdir test-ghost-blog cd test-ghost-blog unzip ../Ghost-1.7.1.zip git init . git add . git commit -m 'Initial commit'
- Install the necessary Node modules, update the git repository:
npm install git add . git commit -m 'Installed Node dependencies'
- Install
knex-migrator, needed for the DB initialization:npm install -g knex-migrator
- Initialize the database and start Ghost (
knex-migratormay give a “Module version mismatch” message, but it seems to work OK anyway):knex-migrator npm start
- Your blog is running! You can visit it at http://localhost:2368/: file:images/ghost-initial-screen.png
- Go to http://localhost:2368/ghost, create your user and set up your blog info: file:images/ghost-setup-blog.png
- You can now start creating content and configuring the local Ghost instance. file:images/ghost-admin-screen.png
- When you have things the way you like them, you can commit the changes to the git repository:
git add . git commit -m 'Finished local Ghost setup'
Now that you have your blog set up locally, we need to generate a static copy that can be published to GitHub. For this we will use the wget command. I gathered the correct options from this blog post by Ilya a few years ago, although it’s not too hard to deduct them from the wget man page.
- We will publish the blog from the
docsdirectory of our repository, so that’s where we need to store the static copy:wget -r -nH -P docs -E -T 2 -np -k http://localhost:2368/
This command will crawl the entire site and create a static copy of it under the
docsdirectory. You can open the filedocs/index.htmlin your web browser to verify. - Add the generated pages to the git repository:
git add docs git commit -m 'Initial commit of static web site'
We can finally create our GitHub repo and push the contents to it.
- Create the repository. I’m using here the =hub= command, but of course you can also do it by hand in the GitHub website (in this case you need to add the git remote by hand as well):
hub create
- Push the local repository to GitHub (this includes both the Ghost source and the generated website under
docs):git push -u origin master
Now all we need to do is enable GitHub Pages on our repository, so that the contents under docs gets published.
- Go to your repository’s “Settings” screen: file:images/ghost-repo-settings-screen.png
- Scroll down to the “GitHub Pages” section, choose the “master branch /docs folder” option and click the “Save” button: file:images/ghost-repo-github-pages-setting.png
We are done! After a few minutes (usually takes 2-5 minutes for the contents to be published the first time, afterwards updates are nearly instantaneous), you will find your new website’s content under http://<github-username>.github.io/<github-repo-name>. In our example, the URL is https://zzamboni.github.io/test-ghost-blog/:
file:images/ghost-published-blog.png
After the initial setup, you need to follow these steps when you want to update your website:
- Start Ghost inside your GitHub repository:
npm start
- Connect to http://localhost:2368/ and update your contents. You can also change the blog settings, themes, etc.
- Re-crawl the site to generate the local copy:
wget -r -nH -P docs -E -T 2 -np -k http://localhost:2368/
- Update and push the whole git repository:
git add . git commit -m 'Website update'
Steps 3 and 4 can be easily automated. I keep the following =update_website.sh= script in the repository:
#!/bin/bash
OUTDIR=docs
LOCAL_GHOST="http://localhost:2368/"
wget -r -nH -P $OUTDIR -E -T 2 -np -k $LOCAL_GHOST && \
git add . && \
git ci -m 'Update website' && \
git pushThen you can just run this script from within your repository after making any changes:
./update_website.shThe method described above is my favorite because it allows me to keep the source data and generated pages in the same repository. However, there are other variations that you might want to use:
- If your repository is named
<username>.github.io, you cannot configure GitHub Pages to serve content from the/docsdirectory, it is automatically served from the root directory of themasterbranch. In this case you need to store only the generated pages in the repository (you could also reverse the setup: have the generated website in the root directory, and the local Ghost install under the/sourcedirectory). - You can choose to serve contents from the
gh-pagesbranch instead of the/docsdirectory. This allows you to keep the source and output still in the same repository. You will need to switch from one branch to the other between updating the contents and generating the static web site (you may need to keep both branches in different directories, so that your local Ghost install can still access its database in themasterbranch while you fetch it to generate the static website).
You can read more about the different ways to serve GitHub Pages content in the GitHub documentation.
You can use the same method to host your static content somewhere other than GitHub pages.
One of the most-touted benefits of a static website (also known, I discovered recently, as the JAMstack) is security – without server-side active components, it’s much harder for a website to be compromised.
Sharp-eyed readers may have noticed that with the setup described above, Ghost’s entire database file gets checked into your repository. This file contains not only your published blog posts, but also your user definitions (including hashed versions of the user passwords) and draft posts.
The local Ghost install uses a SQLite database, stored at content/data/ghost-dev.db, which you can query using the sqlite3 command. For example, to see the user definitions:
sqlite3 content/data/ghost-dev.db 'select * from users'While this seems scandalous, keep in mind that the active Ghost installation is only running locally on your machine, and is not accessible to anyone from the outside, even when it is running (the server is only bound to localhost). Still, you may want to keep in mind:
- Your name and email address are accessible. Your name is visible in any posts you write, but you may want to set your email address to one you don’t mind being public.
- Don’t use a password that you also use in any other place (this is a good security recommendation in general). The password is hashed, but this prevents the password from being useful even if someone manages to figure it out.
- If you share your machine with others, keep in mind that any other local users will be able to access your local Ghost install as long as it’s running. If you don’t trust those users, make sure you set a good password and shut it down when you are not using it.
- To prevent the problem altogether, add
content/data/ghost-dev.dbto your.gitignorefile so it does not get checked into the repository. Make sure you make a backup of it separately so you can recover your blog in case of local data loss.
I followed the steps described above to create a test Ghost install, which you can access at https://zzamboni.github.io/test-ghost-blog/, and its corresponding GitHub repository at https://github.com/zzamboni/test-ghost-blog/. You can also find this article published there.
I hope you’ve found it useful. I’d love to hear your comments!
Welcome to the new zzamboni.org.
Over the years, my website has seen its fair share of transformations, change and breakage. For a few years now, it had stagnated - me with not having much time to update, and its accumulated technological cruft piling high enough to keep me from even touching it, lest I break something.
So, the time has come to do a reboot. With this post, I’m launching the new and reinvented zzamboni.org. Through previous incarnations I had always tried to preserve backwards compatibility in my URLs so people could still find my old stuff, but this has become too big a burden. I may add some things under their old location, and I may add over time some tools to make it easier to find the old stuff, but in general this won’t be a concern and I will not let it stop me from adding new content.
Over the years, the underlying technologies have changed (Wordpress, Posterous, Jekyll, Octopress, Enwrite). As a tech guy, I always enjoy playing with new toys. For now I have settled on using Hugo, a great static-website generator, using its Ananke theme. Hugo and Ananke are powerful and flexible to satisfy my needs, both current and future, as far as I can foresee them. The website continues to be hosted through the fantastic Github Pages. My website was hosted on Github Pages for many years, but it is now served by Netlify.
So this is it. For now the site is mostly empty, but new content will be appearing shortly, both new and ported from my old website. I have started adding posts from my previous blogs. Thanks to Hugo’s aliases feature, most of them should be accessible still through their old URLs.
Please take a look around, and let me know if you find anything broken.
:export_hugo_custom_front_matter+: :series ‘(“Linux on Mac”)’ :series_order 1I’ve been a Mac user and fan for many years. However, my current work machine is a Windows laptop (to which, reluctantly, I’ve gotten used by now), and my personal machine is a 13” 2015 Macbook Pro which was pretty beefed up at the time (16GB, 500GB SSD), but which has felt slower as macOS has progressed. Officially it can only run up to Monterey (macOS 12), and even then it felt quite slow. Some time ago I installed OpenCore Legacy Patcher so I could use it with newer versions of macOS. Most recently I had updated it to Sequoia (macOS 15), and by this time the machine felt almost unusable. So I decided to bring it back to life by installing Linux on it. In this and following posts I will write about this process, including some tips and problems I’ve encountered along the way.
Let me start by saying I’m extremely happy with the result! My old Mac feels like a new machine again - it’s fast and responsive. It’s also been great to catch up with the Linux desktop universe. The last time I had regularly used a Linux machine as my desktop was back in 2007 or so, when I had Gentoo Linux installed on my laptop at work. I’ve been pleasantly surprised by the evolution of Linux desktop environments since then!
The road hasn’t been a straight line. I started with Ubuntu 25.04, which I thought was the safest way to go given its popularity. I used it with Omakub, which provides a great starting point. However I decided to keep experimenting. I briefly tried Pop!_OS, which is very nice, but eventually landed on elementary OS, which feels highly polished, works great, and reminds me a lot of macOS. It’s also based on Ubuntu, so most packages work. I kept my machine as dual-boot with macOS in case I ever need it, but so far I’ve remained firmly on the Linux side and don’t regret it.
Here are some screenshots to wet your appetite: my current dual-boot OpenCore boot screen and my custom mac-like boot theme for Elementary.
Anyhow, how did we get here?
After this, the first step was to free enough space on macOS so I could resize its disk and leave a chunk of disk empty for my Linux partition. I used the excellent DaisyDisk for this, but there are a number of other utilities that can help you identify where the space is going. Fire at will. In my 500GB disk, I managed to reduce my usage to below 300GB on macOS, which allowed me to create a 200GB partition for Linux.
- Delete them:
sudo tmutil deletelocalsnapshots <snapshot-date>
- Repeat until they are all gone.
Once you’ve freed up the space, you can resize the macOS partition to leave unused space in the disk. This can be done graphically using Disk Utility or from the command line with diskutil. For example:
diskutil apfs resizeContainer disk0s2 300gMake sure you are using the right partition, you can view them with diskutil list.
Make sure you can still reboot into macOS and that everything works OK.
If you had disabled the boot picker in OCLP, make sure to reenable it again, it will make it easier to see what’s happening when you dual-boot the machine through OpenCore.
Finally! We can install Linux. Download the installer for your chosen distro and store it in a USB stick (make sure you keep it around, it will be useful if the machine cannot boot after installation). I first installed Ubuntu and fine-tuned my OpenCore boot process with it, but you can also try going directly with something else. Let me give you a few general tips:
- In most Linux installers, you will need to identify the EFI partition where the boot components will be stored. Ubuntu allows you to use the existing macOS/OpenCore boot partition, which is 200MB in size. Elementary OS however, requires its EFI partition to be at least 1GB in size, and will not let you choose a smaller partition. Fortunately, OpenCore automatically knows how to handle multiple EFI partitions, so you can just create an extra 1GB EFI partition in your free space, use it for Elementary, and leave the rest for the main Linux partition. It looks like this in GParted (
dev/sda1was the preexisting EFI partition,dev/sda3is the one I created for installing Elementary):
- After the Linux installation finishes, try rebooting. In principle, you should see the OpenCore boot screen. However, in my case OpenCore refused to identify the Linux partitions. After a lot of head banging I figured out the solution: The trick was to manually install the latest OpenCore release into the EFI partition, overwriting the existing BOOT and OC directories (make sure you back them up first!). You can do this either from macOS if it boots, or from the Linux USB installer. Then OpenCore automatically detected and correctly booted both Linux and macOS. Once everything works, you should see something like this:
This is good! You can select macOS or Linux. Give them a try. You can set your default boot entry by choosing it with the arrow keys and pressing Ctrl-Enter.
In the next post in this series we will look into making the boot screen prettier.
:export_hugo_custom_front_matter+: :series ‘(“Linux on Mac”)’ :series_order 2In the previous post, we saw the steps to install Linux in a dual-boot setup on an old Mac, using OpenCore as the boot manager.
However those icons are fairly generic, and there are a couple of spurious entries there introduced by default by OpenCore. Let’s make it prettier.
The OpenCore boot loader “is a complex piece of software” (in its own words). It’s extensively documented but works reasonably well with defaults. You may need to refer to its Configuration guide for most advanced use cases. For what we want here, see OpenCore beauty treatment, but I’ve extracted the main steps below.
First, we can configure the OpenCore theme. You can find links to many nice themes at OpenCanopy Gallery. The Acidanthera themes are included with OpenCore, but I personally like the Blackosx themes. The one I’m currently using is BsxDarkFenceLight1. Here’s how to install and configure a theme:
- Make sure the OpenCore EFI partition is mounted. It should normally be
/dev/sda1. Depending on how you installed Linux (e.g. if you installed Ubuntu), it may already be mounted asboot/efi. If you installed Elementary OS and created a second EFI partition, you’ll need to explicitly mount it:sudo mkdir /mnt/OC_EFI; sudo mount /dev/sda1 /mnt/OC_EFI - Download the theme you want to use - either clone the git repository or download and unpack its zip file.
git clone https://github.com/blackosx/BsxDarkFenceLight1.git - Copy the theme under
EFI/OC/Resources/Image/in the EFI partition, creating both the top- and sub-directory. For example:sudo rsync -v BsxDarkFenceLight1/Blackosx /mnt/OC_EFI/EFI/OC/Resources/Image/ - Edit
EFI/OC/config.plistto point at the theme you want to try. You need to modify the Misc/PickerVariant attribute. You can use your favorite plist editor, but for such a simple change you can just edit it as a text file, findPickerVariantand edit directly. You need to specify the top/sub directories of the theme you want to use. For example:<key>PickerVariant</key> <string>Blackosx\BsxDarkFenceLight1</string>
- Reboot! You should now see the new theme in action, but still with the extra icons, and with the default Penguin icon for Linux:
Removing and customizing those extra icons took a bit of experimentation. The key is in using .contentVisibility files to hide the undesired icons, and .contentFlavour files to define the icon to use (these files are described in the full OpenCore Configuration guide mentioned above):
- Find all the
.contentFlavourfiles under all your EFI partitions (/mnt/OC_EFIand/boot/efiin our example so far) that contain “OpenCore”. In those same directories, create a.contentVisibilityfile containing the stringDisabled. You may also need to add them to other directories where.efifiles are located. In my case I had these:echo Disabled | sudo tee /mnt/OC_EFI/System/Library/CoreServices/.contentVisibility echo Disabled | sudo tee /boot/efi/EFI/BOOT/.contentVisibility
- Reboot and adjust as needed until you only have the icons you want.
- The Apple icon seems to be hardcoded, but to customize the icon for the Linux partition you can create a
.contentFlavourfile in the directory from which Linux is booted (in Elementary OS this is/boot/, in Ubuntu it was/boot/efi/EFI/ubuntu). The contents of the file should match the basename of one of the icons in the OpenCore theme you are using. The Blackosx icon sets include a vast number of icons for most common Linux distributions, including Elementary OS. So in my case I had to do the following:echo ElementaryOS | sudo tee /boot/.contentFlavour
- Reboot and enjoy your new clean boot screen!
- Bonus tip: to configure the default boot entry, just select it and press
Ctrl-Enter- the choice will be remembers in future boots.
Most Ubuntu-based distributions (and maybe others) these days use Plymouth as their boot-screen manager. Plymouth is a bit of a dark art - it’s somewhat documented but a lot of trial and error is still needed. In elementary OS, by default you will see the following boot screen:
It’s not bad, but I prefer something a bit more Mac-like, with the logo not so big, and a progress bar at the bottom. Here’s what my boot screen looks like:
And here’s how to achieve it:
- Clone the mac-like repository:
git clone https://github.com/zzamboni/mac-like - Copy the
mac-likedirectory to/usr/share/plymouth/themes:sudo cp -a mac-like /usr/share/plymouth/themes/ - The theme comes with the elementary OS logo by default. If you want to change it, you need to replace the
header-image.pngfile with the logo you want to use. You can use whatever logos you want. What I did was to reuse the logos from the OpenCore theme (for example, my eOS logo comes from BsxDarkFenceLight1, before I was using the Ubuntu logo from the same theme). The OpenCore icons include the logos in Apple’s ICNS format, but Plymouth expect the images in PNG, so you need to convert it. You can do it locally with something like Gimp, or online. For example, I have used https://www.coolutils.com/online/ICNS-to-PNG and it works well. Once you have your logo converted, copy it to/usr/share/plymouth/themes/header-image.png. - You need to register the new theme with Ubuntu’s alternatives system:
sudo update-alternatives --install /usr/share/plymouth/themes/default.plymouth default.plymouth /usr/share/plymouth/themes/mac-like/mac-like.plymouth 100 - Choose the new theme as default (choose the appropriate number for
mac-likein the list that appears):sudo update-alternatives --config default.plymouth - Regenerate initramfs:
sudo update-initramfs -u - Reboot and enjoy!
In the next posts I will log some of the further configuration I have done in my laptop.
Eight years ago I announced The Big Website Reboot, when I switched my website to Hugo. I’m happy to announce the first major update since then.
Since then, I’ve been using the Hugo Ananke theme, which is very nice and powerful. Over time, I made adjustments not only to the configuration but also to the theme itself. Due to my lack of experience with Hugo, I failed to separate my changes from the upstream theme, which has made it difficult to upgrade to new versions. While my website was still functional, it felt a bit kludged together under the hood, and it was hard to make any changes, so I decided a major overhaul was needed.
I considered many possibilities, from upgrading Ananke itself or various other Hugo themes, including Congo, Papermod and Tufte. In the end I decided on Blowfish. I liked that it’s extensively documented, very flexible and with many nice built-in features, including search, code highlighting, support for article series, and much more.
With this post I’m inaugurating the website with its new look. All the contents are structure remain mostly the same, though I’ve used the opportunity to do some cleanup and improvements as well. You’ll find many new features thanks to Blowfish, such as the ability to switch dark/light mode, built-in search, “zen mode” for articles, nicer tables of contents, and more.
Please take a look! Please let me know if you find anything broken.
:export_hugo_custom_front_matter+: :series ‘(“Linux on Mac”)’ :series_order 3I noticed that after being asleep, my 2015 MBP running Linux would not wake up if it had an Ethernet cable connected when it went to sleep. With some help from ChatGPT I learned that this seems to be a known problem on some Macs in which the Thunderbolt controller doesn’t resume cleanly, leaving the system stuck in a low-level power state — which effectively hangs the resume process.
The fix is to disable the offending drivers before going to sleep, and reenabling them after. Fortunately, this was easy to automate with systemd. The Thunderbolt Ethernet device on my machine is at /sys/bus/thunderbolt/devices/0-3/ on my machine, and the /sys/bus/thunderbolt/devices/0-3/authorized file can be used to enable/disable the device (by writing a 1 or 0 to it). This can be automated by using a system-hook script. Here’s the script, which should be stored at /lib/systemd/system-sleep/thunderbolt-auto-rebind.sh and made executable. The script automatically detects all the Thunderbolt devices that can be disabled (works for me, but you may have to change it):
#!/bin/sh
# thunderbolt-auto-rebind.sh
# Auto-detect Thunderbolt endpoint devices and deauthorize/reauthorize them
# around suspend/resume to avoid resume-hangs when TB devices (eg TB->Ethernet)
# are attached.
LOGTAG="tb-auto-rebind"
# Find TB devices under /sys/bus/thunderbolt/devices/
find_tb_devices() {
for devpath in /sys/bus/thunderbolt/devices/*; do
[ -d "$devpath" ] || continue
devname="$(basename "$devpath")"
# Skip the host controller or special entries if desired (commonly 0-0)
# Only include endpoints that have an "authorized" control file
if [ -w "$devpath/authorized" ]; then
echo "$devname"
fi
done
}
case "$1" in
pre)
# Before suspend: deauthorize all TB endpoint devices found
for dev in $(find_tb_devices); do
devpath="/sys/bus/thunderbolt/devices/$dev"
echo 0 > "$devpath/authorized" 2>/dev/null && \
systemd-cat -t "$LOGTAG" echo "pre-suspend: deauthorized $dev"
done
;;
post)
# After resume: small wait then re-authorize
sleep 2
for dev in $(find_tb_devices); do
devpath="/sys/bus/thunderbolt/devices/$dev"
echo 1 > "$devpath/authorized" 2>/dev/null && \
systemd-cat -t "$LOGTAG" echo "post-resume: reauthorized $dev"
done
;;
esac(by the way, this script was also written by ChatGPT. I have found it’s quite good at debugging issues like this. It doesn’t normally get there at the first try, but with a bit of back and forth and giving it additional information its very useful)
That’s it! Now my machine sleeps and wakes up correctly regardless of how it’s connected to the network!
I’ve had a Twitter account (sorry, I refuse to call it X) since 2007 (though my first tweet was in January 2008). I was never super prolific, but I enjoyed using it even as it started going downhill with noise, trolls and advertising. Its sale in 2022 was the beginning of the end. In October 2024 I posted my last tweet (#5,051!). I have mostly dropped out of social media, but now I have a Bluesky account. Although Bluesky has also seem some growing pains, it still reminds me a lot of Twitter in the early days, in which community and interaction was more important than marketing and provocation.
(image courtesy of @davis.social)
My Twitter account is now protected, but I decided to keep a local archive of my tweets, to which the Twitter icon in the header/footer in my website now points. I used the excellent tweetback to create this archive. It’s a great tool and very easy to use, and I love the look of the archive. Check it out?
Why protect my account instead of deleting it? Glad you asked:
- Protecting my account makes my tweets visible only to approved followers. I can still control who sees them, without erasing my history.
- Deleting my account would surrender control entirely — once it’s gone, I wouldn’t be able to reclaim the same account name. Worse - someone could reuse it! Leaving my account in place prevents it.
- Who knows - Twitter may recover. It may become again a platform worth using. Not placing a lot of hope on this, but if it ever happens, I could easily reactivate my account.
I have wished for a long time to have separation between my CV contents and layout. After maintaining it directly in LaTeX for a while, I switched years ago to org-mode with LaTeX as the main export target. It worked and it enforced a certain degree of separation (most of the “visual” decisions were left up to the LaTeX exporter, with the org-mode file focusing on the contents), but it was fragile: too many custom LaTeX bits, too much hand-editing, and a pipeline that was hard to reuse elsewhere. It was also impossible to produce a nice HTML version from it. In this post I’ll document how I switched the source of truth to JSON Resume and built a custom pipeline that outputs both my CV and my publications in HTML and PDF. I’m very happy with the result, which you can find in this same website: my full CV.
This post is a quick walk-through of the steps, the trade-offs, and the pieces I ended up keeping. This was a months-long side project with many detours, so this post is necessarily lacking in detail, but I hope it provides you with a good overview. The end result is in my vita repo in GitLab, so feel free to use it as a starting point.
I did a quick survey of the “semantic resume” ecosystem before committing. There are several options that are all pretty good and popular: JSON Resume, YAML Resume, RenderCV, and a few smaller variants. I picked JSON Resume mainly because the schema feels mature and stable, it is flexible enough for the kinds of entries I keep, and it has a large ecosystem of themes, validators, and exporters. The openness angle also mattered to me. YAML Resume and RenderCV are open, but both are tied to commercial products and workflows, which made me worry about drift or lock-in over time. JSON Resume felt like the most neutral, lowest‑friction baseline: I can keep the data portable, and still build my own pipeline around it without depending on any one vendor’s tooling.
The first task was simply translating the structure. Org-mode let me be loose; JSON Resume forces me to be explicit, which is both a blessing and a constraint. I built a first cut of zamboni-vita.json by using ChatGPT to map headings to JSON Resume sections and then iterating until the HTML output looked reasonable. Early on I chose the even JSONresume theme. I like its’ cleanliness and nice structure. However, I added many custom features to it, as you’ll see below.
Key lessons:
- Keep the JSON flat and normalized. I removed most of the presentation-specific markup and kept only semantic data. The only concession is to include Markdown in many of the JSONresume fields, to allow for some formatting and links.
- Convert lists and sublists into fields instead of raw text. This makes them easier to render in multiple formats later.
- Store links, dates, and IDs consistently (I ended up standardizing on ISO dates and stable IDs for publications).
The CV is one thing. My publications are another: they were already in BibTeX, and I wanted a separate publications page and a PDF. I kept the BibTeX source, and built a small pipeline around it:
- =scripts/build_publications.py= reads my BibTeX sources from =pubs-src=.
- It generates publications HTML with =templates/publications.html.j2=.
- It also writes an aggregated =zamboni-pubs.bib= for download.
This kept the bibliographic source clean while giving me a consistent, reproducible output.
I used =jsonresume-theme-even= as a base, but I wanted it to look like my CV. I install the theme locally in my development environment so I could iterate without publishing anything upstream. I have contributed my changes to the upstream theme, you can find them separately or in an aggregated pull request.
These were the main changes I made:
- support automated grouping of ”Projects” entries in JSONresume according to their type. This is how I create the “Research”, “Teaching”, “Software” and other sections in my CV while maintaining compatibility with the JSONresume schema.
- reorder sections to match my preferred layout
- add icons and lightweight UI polish so the HTML feels like a “document” and not just a web page. For example, I added support for a floating table of contents, for displaying certification badges, for FontAwesome icons (the upstream theme uses Feather icons, but they don’t have as many icons as FontAwesome), and for floating icons that can link to arbitrary URLs.
This is where JSON Resume started paying off: I could re-render quickly and keep the content stable while iterating on layout.
For generating the CV PDF I considered converting to RenderCV, but found its schema too limited, existing conversion tools and available themes not to my liking. From the visual point of view I really liked Awesome CV, the LaTeX template I was already using for my PDF CV. I decided to implement a custom JSONresume exporter to Typst. I had heard of Typst and this gave me an excuse to start learning it. Furthermore, I found Brilliant CV, a Typst template which replicates Awesome CV. In the end, my new PDF CV looks very similar to the previous one, which is good because I really like the format!
The main component here is the =scripts/render_typst_cv.py= script, which converts JSON Resume into a self-contained Typst document which uses the brilliant-cv template.
For publications I tried to use Typst as well, since it has built-in support for importing BibTeX files. This worked well with a single bibliography, but I wanted to keep my publications list structured by type. In the end I decided to stay with LaTeX and used a small wrapper so the output matches the HTML:
- =pubs-src/zamboni-pubs.tex= is the source for the publications PDF. Note that this still uses the old AwesomeCV class, so my publications list still has the same style as my new CV.
- A =tectonic= build step produces the PDF version of my publications list, with the same structure (and even order of entries) of the online version.
The output of both pipelines lands under build/zamboni-jsonresume/{dev,prod}/vita/, which mirrors the final website structure.
The last bit was making the whole thing repeatable. I added a few mise tasks so I can rebuild everything quickly:
mise run build-devfor local HTML + PDFs (the key difference is that the dev HTML outputs include some code for auto-reloading, and embedded debug information, which is omitted in the prod version)mise run build-prodfor the production outputsmise run deploy-prodto sync the final output to my website
This was the ultimate payoff: JSON/BibTeX are the source of truth, and the rest is mechanical.
To further automate my development environment, I used the opportunity to learn about Dev Containers, which make it easy to edit files locally, while running compilation/debug steps inside a container with all the necessary tools:
- devcontainer.json defines the tools and environment that will be installed in the container. This way I can have all the necessary tools in a consistent and repeatable way, without polluting or depending on the setup of my local machine.
- A few additional mise tasks help me interact with the devcontainer:
mise dev-upto create the container and run the development loop inside it.mise dev-rebuild-upto force creation of a new container before startup.mise dev-bashto open a shell session in the running container.
- Pick a semantic source format early. JSON Resume makes it easy to reuse the data.
- Separate “content” from “layout” aggressively. Theme tweaks are now safe and quick.
- Keep publications as a distinct pipeline. They have different constraints and I want the flexibility.
If you are thinking of a similar conversion, start with the data model and only then worry about the rendering. That order made the whole project go faster than I expected.
I was also impressed by how much AI tools helped me during the process. I used ChatGPT Codex and Claude Code for much of the process, and they helped me quickly create and fine tune the scripts, and also to understand and modify the JSONresume theme.
A few weeks ago I wrote about porting my CV from Org+LaTeX to JSON Resume. At the time, I had reached a point where my data lived in a much saner format, I could produce HTML and PDF output, and I had a publications pipeline that more or less did what I needed. It was already a big improvement over the old setup.
But there was still an obvious problem: it was my pipeline.
It lived inside my CV repository, it knew too much about my files and my directory structure, and although I had tried to keep things generic, the whole thing still felt like a personal construction project that happened to work. Reusing it would have required copying half the repository and then carefully undoing all the places where I had hardcoded my own name, information and assumptions.
That bothered me more than I expected.
If I was going to invest this much effort in moving to a semantic CV format, then I wanted the result to be genuinely reusable. Not “reusable if you don’t mind editing five scripts and reading my mind,” but actually reusable by someone else, or by future me after I had forgotten how it all fit together.
So the next phase of the project was to separate the tool from the data, and to turn the whole thing into something I could invoke on any JSON Resume file and get sensible output.
The first big step was to pull the build pipeline out of my CV repository and make it stand on its own. That sounds obvious in retrospect, but in practice it meant a lot of clean up.
I had to make paths configurable. I had to stop assuming that my publications always lived in a specific directory. I had to make the scripts work no matter where they were invoked from. I had to teach them how to find local assets, profile pictures, logos, and BibTeX files in a way that would make sense for arbitrary input files, not just for my own layout.
For this I relied heavily on ChatGPT Codex, which acted as a competent programmer, implementing my specifications. I was able to give instructions of what I wanted to do, and guide it step by step through making the necessary changes in the code. I am very impressed by its abilities.
The result is what is now =resume-toolkit=: a standalone project that takes a JSON Resume file, optional BibTeX sources, and produces:
- an HTML CV
- a PDF CV
- if BibTeX files are given:
- a standalone HTML publications page
- a standalone PDF publications list
- an aggregated BibTeX file for download
The important change was not that the scripts moved to another repository. The important change was that the boundary became cleaner: the toolkit is now the build system, while my CV repository is just content plus a little configuration.
That sounds mundane, but it changed the whole feel of the project.
Once the pipeline lived on its own, the next challenge was making it robust enough that I could point it at other people’s JSON Resume files and not have it immediately fall apart. I tested it by grabbing random JSON Resume files from the JSON Resume Registry and testing the toolkit with them. This helped me identify several bugs and hidden assumptions that got fixed and resulted in several new useful features.
This turned out to be a very good forcing function.
As soon as I started trying random JSON Resume files “from the wild”, all the edge cases came out:
- entries using
companyinstead ofnamein work history - missing locations
- missing profile images
- image URLs with no filename extension
- relative links that worked in HTML but not in downloaded PDFs
- optional sections that were present in some resumes and absent in others
- data that was valid enough for one renderer but broke another
Fixing those cases made the toolkit much better. In many ways, this was the point where it stopped being “my CV exporter” and started becoming a general-purpose tool.
I also wrapped the whole thing in a Docker image and a small build-resume.sh helper so the user-facing entry point became dead simple:
build-resume.sh resume.jsonThat script grew a few useful features over time: watch mode, serve mode, optional pulling of a newer image, automatic port selection for multiple local preview sessions, and so on. None of those are particularly glamorous, but they make the difference between “interesting code” and “something I actually enjoy using.”
One thing I wanted from the start was to keep the source of truth in the data, not in the scripts.
The initial version still had too many decisions hidden in code: link definitions, publications behavior, section titles, PDF layout knobs, and little bits of rendering policy. It worked, but it was not very transparent.
So I kept moving configuration into the JSON file, mostly under the meta section. I created three new subsections in addition to the existing meta.themeOptions supported by the Even theme:
meta.sitefor defining the URL of the site where the resume will be published (to expand relative to full links in PDF rendering);meta.pdfthemeOptionsfor configuring the PDF rendering;meta.publicationsOptionsfor configuring the behavior of the publications section and the standalone publications page.
This turned out to be one of the nicest improvements in the whole project. Instead of editing Python or shell code, I can now control a lot of behavior directly from the resume data:
- floating links in the HTML CV and publications page
- publications grouping and section titles
- whether publications are inlined in the PDF CV
- whether inline publications use the full list or a filtered subset
- bibliography filtering by explicit BibTeX entries or by keywords
- PDF layout options for the
brilliant-cvTypst theme - visible printable URLs in the PDF
- footer URLs for both the CV and the publications PDF
- base site URL handling for converting relative links into absolute PDF links
The more I did this, the more the system started feeling like a toolkit rather than a collection of scripts.
It also made the behavior easier to reason about. If something is controlled by the JSON, it is much easier to inspect, version, and reuse than if it lives in a Python function somewhere in the middle of the pipeline.
The publications side of the project kept evolving too.
I already had my publications in BibTeX, and I wanted to preserve that as the source of truth. At first the pipeline simply aggregated the BibTeX and rendered everything. But once I started using it more seriously, I wanted more control.
In particular, I wanted to distinguish between:
- the full standalone publications list
- a selected subset of publications shown inline in the CV
That led to support for filtering by BibTeX entry key and by BibTeX keywords, and to better separation between the standalone publications outputs and the inline bibliography used in the PDF CV.
I also added a command to write the selected publications back into the JSON Resume publications section, so the HTML CV can show the same curated subset without needing special support from the HTML theme.
This part was more fiddly than I expected, mostly because BibTeX is both wonderfully flexible and slightly chaotic. But in the end it gave me a setup I really like: BibTeX remains the canonical source, while the JSON Resume gets just enough derived data to render nicely everywhere.
In my previous post I mentioned that I had started from =jsonresume-theme-even=. That is still true. It was a great base and it gave me a clean starting point.
I started making some tweaks, but over time, the number of changes kept growing. Some were small visual touches. Some were structural changes. Some were features I wanted for my own CV that turned out to be generally useful:
- grouped project sections
- floating links
- Font Awesome icon support
- certification badges
- better note-style entries
- more configurable colors and layout behavior
- support for my publications page conventions
- lots of little fixes and polish
At some point I had to admit the obvious: this was no longer a small customization layer on top of the original theme.
I had been maintaining my changes as a large pull request against the upstream theme, but realistically they are too extensive, too opinionated, and too tightly coupled to the direction I want for this project. I don’t think that is a bad thing. It just means the project has its own identity now.
So I finally split it out as its own fork: jsonresume-theme-eventide.
I wanted a name that still nodded to the original even theme without implying “version 2” or “better” or anything like that. eventide seemed right: related, but clearly its own thing.
What I like most now is not any individual feature, but the shape of the system.
- My CV repository is only content.
- The toolkit repository is mostly machinery.
- The theme repository is mostly HTML presentation.
That separation is not perfect, but it is much better than where I started.
It means I can evolve the pipeline without touching my CV data. I can evolve the theme without digging through the build scripts. I can test the toolkit with other resumes and make it more robust. And I can publish the whole thing as something that other people might actually reuse, instead of as a pile of personal glue code.
That was the real goal all along.
A few things I learned from this second phase:
- extracting a personal tool into a reusable one is mostly about removing assumptions
- testing with other people’s data is one of the fastest ways to harden your own code
- pushing configuration into the data model makes the system easier to understand
- once a heavily customized theme starts developing its own direction, it is better to acknowledge that honestly and fork it
- small workflow improvements matter a lot when you run the same build loop dozens of times
I’m pretty happy with where resume-toolkit is now. It started as a side effect of moving my CV away from Org and LaTeX, but it has turned into a nice standalone project in its own right.
There is still plenty I could improve, of course. But it now feels like I’m building on top of a tool I trust, instead of constantly patching a one-off pipeline that only I understand.
If you use (or are interested in ) JSON Resume, please give resume-toolkit and Eventide a try! I would be happy to hear your comments or feedback.