Skip to content
Permalink
master
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time
2563 lines (1868 sloc) 164 KB

zzamboni.org source file

This file is the source for all new and updated content on my website since March 2018. Content here may be in progress or incomplete. This file gets converted to Hugo files by the excellent ox-hugo.

You really should not read it here but at zzamboni.org.

Table of Contents

Pages

This section contains all the static pages.

About

Who:
My name is Diego Zamboni. I am a computer scientist, security expert, consultant, project and team leader, programmer, sysadmin, author and overall geek. I am from Mexico but live in Switzerland with my beautiful wife and our two awesome daughters.
What:
I work as Enterprise Security Architect in the IT Clouds Division of Swisscom. I am also the author of “Learning CFEngine” and “Learning Hammerspoon”.
Where:
I was born in Argentina, but have moved around all my life. When I was very young I moved to Mexico, where I lived in four different cities before moving to the U.S. to pursue my Ph.D. at Purdue University under the direction of Gene Spafford. Upon finishing my studies, my wife and I decided to go to Switzerland, where I worked at the IBM Zurich Research Lab. Eight years and two kids later, we moved to Mexico in late 2009. In 2015, we moved back to Switzerland.
Long version:
If you are interested, here’s my Curriculum Vitae. For other useless trivia about me, see here.

My online past

2009-2016:
During this time my blog went through several iterations, hosted/powered by Posterous, Jekyll, Octopress, Postach.io and Enwrite. Most blog entries from this period have been merged into my current blog;
2005-2009:
My blog titled BrT, powered mainly by a self-hosted Wordpress installation, with some intermixed use of Posterous. What you find at the link is a static archive copy;
1997-2001:
My hand-maintained web page at Purdue University (the original has disappeared, here’s a mirror).

About this site

This entire site is generated by Hugo and served by Netlify.

The source is stored in my zzamboni/zzamboni.org GitLab repository. I use ox-hugo to generate the content from a single source file in org-mode format, although all the older articles and pages are still stored in their original source Markdown files (I gradually convert them whenever I update them). Some of my project pages are stored in the gh-pages branch of their own GitHub repositories.

I think it’s incredible that all of this infrastructure is so easy to use and available for free.

Image attributions:

  • C128 Code (code header background) is from the source code listing from my Commodore 128 program Supercataloguer 128.
  • Scrabble letters (blog header background) from Pexels, licensed under CC0.
  • All other header background photos were taken either by my wife or me.

If you have any concerns or questions about the images used in this site, please let me know.

Contact

If you have any questions, comments or feedback about this site, please use the form below to send me a message.

—-

@@html:{{< form-contact-netlify >}}@@

Books

I have a few self-published books, all of them available at Leanpub. Here you can find short descriptions and links to their respective pages.

Book pages

The individual pages for each book.

Learning CFEngine

{{< bookimglink style=”float:right” >}}

I am the author of “Learning CFEngine”, the best book for learning CFEngine.

The book has its own webpage at http://cf-learn.info, please visit it for more information, code samples, etc.

You can buy the book at https://leanpub.com/learning-cfengine or by clicking the link on the right.

 

Learning Hammerspoon

{{< hsbookimglink style=”float:right” >}}

Automate all the things! From window manipulation to automated system settings depending on your current location, Hammerspoon makes it possible. In this book you will learn how to get started with Hammerspoon, how to use pre-made modules, and how to write your own, to achieve an unprecedented level of control over your Mac.

Learn more, read a free sample and get the book (you choose how much you pay!) at https://leanpub.com/learning-hammerspoon/, or click on the banner on the right.

 

Utilerías de Unix

{{< utileriasbookimglink style=”float:right” >}}

(this book is in Spanish)

¡Automatiza tus tareas e incrementa tu eficiencia! Este libro proporciona una introducción a algunos de los comandos más útiles en un sistema Unix/Linux, que te permiten realizar fácilmente tareas de todo tipo, desde la búsqueda de texto en un archivo hasta complejas operaciones de procesamiento, cálculo y extracción de datos.

Puedes obtener una muestra gratis y comprar el libro (¡tu eliges cuánto pagas!) en https://leanpub.com/utileras-unix, o haz click en la imágen de la derecha.

 

Literate Config

{{< leanpubbook book=”lit-config” style=”float:right” >}}

This booklet will teach you about Literate Configuration, which is the application of Literate Programming to configuration files. Literate Programming can be especially applicable to configuration files for the following reasons:

  • Configuration files are inherently focused, since they correspond to a single application, program or set of programs, all related. This makes it easier to draw a narrative for them;
  • Most configuration files are self-contained but their structure and syntax may not be immediately evident, so they benefit from a human-readable explanation of their contents;
  • Configuration files are often shared and read by others, as we all like to learn by reading the config files of other people. Applying Literate Programming to config files makes them much easier to share, since their explanation is naturally woven into the code.

Org-mode is a powerful and simple markup language for general writing, but with unique features that make it easy to include code within the text, and even further, to easily extract that code into stand-alone source files which can be interpreted by their corresponding programs.

Whether you already use Emacs and org-mode or not, you will find value in this book by seeing how uniquely Literate Programming can help you better write, maintain, understand and share your config files.

 

Ideas

Ideas for things to write about.

Using and extending Seal

Sending things to OmniFocus

ox-hugo

CryFS

Hosting a Hugo blog in Netlify

Short guide to setting up Hugo, using /docs to host to avoid branches, and setting up your custom domain.

My blogging workflow with Emacs, org-mode+ox-hugo, Hugo and GitHub

My blogging has seen multiple iterations over the years, and with it, the tools I use have changed. At the moment I use a set of tools and workflows which make it very easy to keep my blog updated, and I will describe them in this post. In short, they are:

  • Writing: Emacs, org-mode
  • Exporting: ox-hugo
  • Publishing: Hugo
  • Hosting: GitHub Pages

Let’s take a closer look at each of the stages.

Writing using Emacs and org-mode

I have been using Emacs for almost 30 years, so its use for me is second nature. However, I only recently started using org-mode for writing, blogging, coding, presentations and more, thanks to the hearty recommendations and information from Nick and many others. I am duly impressed. I have been a fan of the idea of literate programming for many years, and I have tried other tools before (most notably noweb, which I used during grad school for many of my homeworks and projects), but org-mode is the first tool I have encountered which seems to make it practical. Here are some of the resources I have found useful in learning it:

You can see some examples in my “literate config files” series, and all recent posts in this blog are written using org-mode (you can find the source file in GitHub).

Over time I have tweaked my Emacs configuration to make writing with org-mode more pleasant. You can see my tutorial on Beautifying org-mode in Emacs, and also see the full org-mode section of my Emacs config for reference.

So, I write posts using Emacs, in org-mode markup. What’s next?

Exporting

When I first started writing my blog posts in org-mode, I relied on Hugo’s built-in support for it, which allows you to simply create posts in .org files instead of .md and have them parse in org-mode format. Unfortunately, the support is not perfect. Hugo relies on the goorgeous library which, while quite powerful, does not support the full org-mode markup capabilities, so many elements are not rendered or processed properly.

Happily, I discovered ox-hugo, an org-mode exporter which produces Hugo-ready Markdown files from the org-mode source, from which Hugo can produce the final HTML output. This is a much better arrangement, because each component handles only its native format: ox-hugo processes the org-mode source with the full support of org-mode and Emacs, and Hugo processes Markdown files, which are its native input format. You can use the full range of org-mode markup in your posts, and they will be correctly converted to their equivalents in Markdown. Furthermore, your source files remain output-agnostic, as you can still use all other org-mode exporters if you need to produce other formats.

Ox-hugo supports two ways of organizing your posts: one post per org file, and one post per org subtree. In the first one, you write a separate org file for each post. In the second, you keep all your posts in a single org file, and specify (through org-mode properties) which subtrees should be exported as posts. The latter is the recommended way to organize posts. At first I was skeptical - who wants to keep everything in a single file? However, as I have worked more with it, I have come to realize its advantages. For one, it makes it easier to specify post metadata - for example, I have defined sections in my org-mode source file for certain frequent topics, and those are tagged accordingly in the org source. When I create posts as subtrees of those sections, they inherit the top-level tags automatically, as well as any other properties, which I used to define (for example) the header images used in the posts. Having all posts in a single file also makes it easier to share other content, such as org macro definitions, ox-hugo configuration options, etc.

Note that ox-hugo is not limited to exporting blog posts, but any content processed by Hugo. For example, my org source file also includes all the static pages in my web site - they are differentiated from blog posts simply by the Hugo section to which they belong, which is defined using the HUGO_SECTION property, which is interpreted accordingly by ox-hugo.

Since the full power of org markup is available when using ox-hugo, you can do very interesting things. For example, all posts in my Literate Config Files category are automatically updated every time I export them with the actual, real content of the corresponding config files, which I also keep in org format. There is a lot of hidden power in org-mode and ox-hugo. My recommendation is to go through the source files for some of the websites listed in ox-hugo’s Real World Examples section. For example, I have learned a lot by reading through the source files for the ox-hugo website itself.

Publishing

Using Netlify. Caveats:

Once ox-hugo has generated the Markdown files for my posts, it is Hugo’s turn to convert them into HTML files that can be published as a website. Ox-hugo knows the default structure expected by Hugo (a top-level content/ directory in which you have directories for each section), so there’s usually not much to do other than point ox-hugo to where your top-level Hugo directory is, using the HUGO_BASE_DIR property. Of course, this presumes you already have a Hugo site. If you have never used Hugo before, I would suggest you go through the Quick Start guide, which shows how to set up a basic website using the Ananke theme. This is the same theme I use for my website.

One particular piece of configuration I use to make publishing with GitHub pages easier: I change Hugo’s publishDir parameter from its default value of public to docs, to make it easier to publish my final website from within the same repository (more below in the Hosting section). This is done by specifying the parameter in Hugo’s =config.toml= file:

publishDir = "docs"

Hugo has extensive capabilities and it is beyond the scope of this article to show you how to use it, but it has very good documentation which I would urge you to peruse to learn more about it. Feel free to peruse my setup for ideas.

Hosting

  • Publishing from docs/
  • Setting up SSL certificate

What’s next?

  • Automatic publishing using GitHub CI
  • Maybe migrate to GitLab pages?

Posts

Blog posts.

Literate config files

I group here the posts about my documented config files, which include the live files from my current configuration.

My Emacs Configuration, With Commentary

{{< leanpubbook book=”lit-config” style=”float:right” >}}

Last update: {{{updatetime}}}

I have enjoyed slowly converting my configuration files to literate programming style style using org-mode in Emacs. I previously posted my Elvish configuration, and now it’s the turn of my Emacs configuration file. The text below is included directly from my init.org file. Please note that the text below is a snapshot as the file stands as of the date shown above, but it is always evolving. See the init.org file in GitHub for my current, live configuration, and the generated file at init.el.

If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!

My Hammerspoon Configuration, With Commentary

{{< leanpubbook book=”lit-config” style=”float:right” >}} {{< leanpubbook book=”learning-hammerspoon” style=”float:right” >}}

Last update: {{{updatetime}}}

In my ongoing series of literate config files, I present to you my Hammerspoon configuration file. You can see the generated file at https://github.com/zzamboni/dot-hammerspoon/blob/master/init.lua. As usual, this is just a snapshot at the time shown above, you can see the current version of my configuration in GitHub.

If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!

My Elvish Configuration With Commentary

{{< leanpubbook book=”lit-config” style=”float:right” >}}

Last update: {{{updatetime}}}

In this blog post I will walk you through my current Elvish configuration file, with running commentary about the different sections.

This is also my first blog post written using org-mode, which I have started using for writing and documenting my code, using literate programming. The content below is included unmodified from my rc.org file (as of the date shown above), from which the rc.elv file is directly generated.

If you are interested in writing your own Literate Config files, check out my new book Literate Config on Leanpub!

Without further ado…

Emacs

Beautifying Org Mode in Emacs

Over the last few months, I have used org-mode more and more for writing and programming in Emacs. I love its flexibility and power, and it is the first literate programming tool which “feels right”, and I have been able to stick with it for a longer period of time than in my previous attempts.

Recently I started thinking about how I could make my editing environment more visually appealing. I am in general very happy with my Emacs’ appearance. I use the Gruvbox theme, and org-mode has very decent syntax highlighting. But as I write more and more prose in Emacs these days, I started thinking it might be nice to edit text in more visually-appealing fonts, including using a proportional font, which makes regular prose much more readable. I would like to share with you what I learned and my current Emacs configuration.

In the end, you can have an Emacs setup for editing org documents which looks very nice, with proportional fonts for text and monospaced fonts for code blocks, examples and other elements. To wet your appetite, here is what a fragment of my init.org file looks like:

images/emacs-init-propfonts.png

Step 1: Configure faces for Org headlines and lists

My first step was to make org-mode much more readable by using different fonts for headings, hiding some of the markup, and improving list bullets. I took these settings originally from Howard Abrams’ excellent Org as a Word Processor article, although I have tweaked them a bit.

First, we ask org-mode to hide the emphasis markup (e.g. /.../ for italics, *...* for bold, etc.):

(setq org-hide-emphasis-markers t)

Then, we set up a font-lock substitution for list markers (I always use ”-” for lists, but you can change this if you want) by replacing them with a centered-dot character:

(font-lock-add-keywords 'org-mode
                        '(("^ *\\([-]\\) "
                           (0 (prog1 () (compose-region (match-beginning 1) (match-end 1) ""))))))

The org-bullets package replaces all headline markers with different Unicode bullets:

(use-package org-bullets
  :config
  (add-hook 'org-mode-hook (lambda () (org-bullets-mode 1))))

Finally, we set up a nice proportional font, in different sizes, for the headlines. The fonts listed will be tried in sequence, and the first one found will be used. Feel free to add your own favorite font:

(let* ((variable-tuple
        (cond ((x-list-fonts "Source Sans Pro") '(:font "Source Sans Pro"))
              ((x-list-fonts "Lucida Grande")   '(:font "Lucida Grande"))
              ((x-list-fonts "Verdana")         '(:font "Verdana"))
              ((x-family-fonts "Sans Serif")    '(:family "Sans Serif"))
              (nil (warn "Cannot find a Sans Serif Font.  Install Source Sans Pro."))))
       (base-font-color     (face-foreground 'default nil 'default))
       (headline           `(:inherit default :weight bold :foreground ,base-font-color)))

  (custom-theme-set-faces
   'user
   `(org-level-8 ((t (,@headline ,@variable-tuple))))
   `(org-level-7 ((t (,@headline ,@variable-tuple))))
   `(org-level-6 ((t (,@headline ,@variable-tuple))))
   `(org-level-5 ((t (,@headline ,@variable-tuple))))
   `(org-level-4 ((t (,@headline ,@variable-tuple :height 1.1))))
   `(org-level-3 ((t (,@headline ,@variable-tuple :height 1.25))))
   `(org-level-2 ((t (,@headline ,@variable-tuple :height 1.5))))
   `(org-level-1 ((t (,@headline ,@variable-tuple :height 1.75))))
   `(org-document-title ((t (,@headline ,@variable-tuple :height 2.0 :underline nil))))))

Step 2: Setting up variable-pitch and fixed-pitch faces

My next realization was that Emacs already includes support for displaying proportional fonts with the variable-pitch-mode command. You can try it right now: type M-x variable-pitch-mode and your current buffer will be shown in a proportional font (you can disable it by running variable-pitch-mode again). On my Mac the default variable-pitch font is Helvetica. You can change the font used by configuring the variable-pitch face. You can do this interactively through the customize interface by typing M-x customize-face variable-pitch. At the moment I like Source Sans Pro.

As a counterpart to variable-pitch, you need to configure the fixed-pitch face for the text that needs to be shown in a monospaced font. My first instinct was to inherit this from my default face (I use Inconsolata), but it seems that this gets remapped when variable-pitch-mode is active, so I had to configure it by hand with the same font as my default face.

What I would suggest is that you customize the fonts interactively, as you can see live how it looks on your text. You can make the configuration permanent from the customize screen as well. If you want to explicitly set them in your configuration file, you can do it with the custom-theme-set-faces function, like this:

(custom-theme-set-faces
 'user
 '(variable-pitch ((t (:family "Source Sans Pro" :height 180 :weight light))))
 '(fixed-pitch ((t ( :family "Inconsolata" :slant normal :weight normal :height 1.0 :width normal)))))

Tip: you can get the LISP expression for your chosen font (the part that looks like ((t (:family ... ))) from the customize-face screen - open the “State” button and choose the “Show Lisp Expression” menu item.

You can enable variable-pitch-mode automatically for org buffers by setting up a hook like this:

(add-hook 'org-mode-hook 'variable-pitch-mode)

Step 3: Use long lines and visual-line-mode

One thing you will notice right away with proportional fonts is that filling paragraphs no longer makes sense. This is because fill-paragraph works based on the number of characters in a line, but with a proportional font, characters have different widths, so a filled paragraph looks strange:

images/emacs-filled-paragraph.png

Of course, you can still do it, but there’s a better way. With visual-line-mode enabled, long lines will flow and adjust to the width of the window. This is great for writing prose, because you can choose how wide your lines are by just resizing your window.

images/emacs-narrow-window.png

images/emacs-wide-window.png

There is one habit you have to change for this to work: the instinct (at least for me) of pressing M-q every once in a while to readjust the current paragraph. I personally think it’s worth it.

You can enable visual-line-mode automatically for org buffers by setting up another hook:

(add-hook 'org-mode-hook 'visual-line-mode)

Step 4: Configure faces for specific Org elements

After all the changes above, you will have nice, proportional fonts in your Org buffers. However, there are some things for which you still want monospace fonts! Things like source blocks, examples, tags and some other markup elements still look better in a fixed-spacing font, in my opinion. Fortunately, org-mode has an extremely granular face selection, so you can easily customize them to have different elements shown in the correct font, color, and size.

Tip: you can use C-u C-x = (which runs the command what-cursor-position with a prefix argument) to show information about the character under the cursor, including the face which is being used for it. If you find a markup element which is not correctly configured, you can use this to know which face you have to customize.

You can configure specific faces any way you want, but if you simply want them to be rendered in monospace font, you can set them to inherit from the fixed-pitch face we configured before. You can also inherit from multiple faces to combine their attributes.

Here are the faces I have configured so far (there are probably many more to do, but I don’t use org-mode to its full capacity yet). I’m showing here the LISP expressions, but you can just as well configure them using customize-face.

(custom-theme-set-faces
 'user
 '(org-block ((t (:inherit fixed-pitch))))
 '(org-code ((t (:inherit (shadow fixed-pitch)))))
 '(org-document-info ((t (:foreground "dark orange"))))
 '(org-document-info-keyword ((t (:inherit (shadow fixed-pitch)))))
 '(org-indent ((t (:inherit (org-hide fixed-pitch)))))
 '(org-link ((t (:foreground "royal blue" :underline t))))
 '(org-meta-line ((t (:inherit (font-lock-comment-face fixed-pitch)))))
 '(org-property-value ((t (:inherit fixed-pitch))) t)
 '(org-special-keyword ((t (:inherit (font-lock-comment-face fixed-pitch)))))
 '(org-table ((t (:inherit fixed-pitch :foreground "#83a598"))))
 '(org-tag ((t (:inherit (shadow fixed-pitch) :weight bold :height 0.8))))
 '(org-verbatim ((t (:inherit (shadow fixed-pitch))))))

Update (2019/10/24): updated the settings above based on my latest config. Update (2019/02/24): thanks to Ben for figuring out the fix to the vertical spacing issue noted below. The trick is to set the org-indent face (see above) to inherit from fixed-pitch as well.

One minor issue I have noticed is that, in =variable-pitch-mode=, the fixed-pitch blocks have a slight increase in inter-line spacing. This is not a deal breaker for me, but it is a noticeable difference. This can be observed in the following screenshot, which shows the block of code above embedded in the org-mode buffer and in the block-editing buffer, which uses the fixed-width font. If you know a way in which this could be fixed, please let me know!

Conclusion

The setup described above has considerably improved my enjoyment of writing in Emacs. I hope you find it useful. If you have any feedback, suggestions or questions, please let me know in the comments.

Hammerspoon

Getting Started With Hammerspoon

This is the first installment of a series of posts about Hammerspoon, a staggeringly powerful automation utility which gives you an amazing degree of control over your Mac, allowing you to automate and control almost anything. In the word of Hammerspoon’s motto: Making the runtime, funtime.

Why Hammerspoon?

Hammerspoon is a Mac application that allows you to achieve an unprecedented level of control over your Mac. Hammerspoon enables interaction with the system at multiple layers–from low-level file system or network access, mouse or keyboard event capture and generation, all the way to manipulating applications or windows, processing URLs and drawing on the screen. It also allows interfacing with AppleScript, Unix commands and scripts, and other applications. Hammerspoon configuration is written in Lua, a popular embedded programming language.

Using Hammerspoon, you can replace many stand-alone Mac utilities for controlling or customizing specific aspects of your Mac (the kind that tends to overcrowd the menubar). For example, the following are doable using Hammerspoon (these are all things I do with it on my machine - each paragraph links to the corresponding sections in my config file):

Hammerspoon is the most powerful Mac automation utility I have ever used. If you are a programmer, it can make using your Mac vastly more fun and productive.

How does Hammerspoon work?

Hammerspoon acts as a thin layer between the operating system and a Lua-based configuration language. It includes extensions for querying and controlling many aspects of the system. Some of the lower-level extensions are written in Objective-C, but all of them expose a Lua API, and it is trivial to write your own extensions or modules to extend its functionality.

From the Hammerspoon configuration you can also execute external commands, run AppleScript or JavaScript code using the OSA scripting framework, establish network connections and even run network servers; you can capture and generate keyboard events, detect network changes, USB or audio devices being plugged in or out, changes in screen or keyboard language configuration; you can draw directly on the screen to display whatever you want; and many other things. Take a quick look at the Hammerspoon API index page to get a feeling of its extensive capabilities. And that is only the libraries that are built into Hammerspoon. There is an extensive and growing collection of Spoons, modules written in pure Lua that provide additional functionality and integration. And of course, the configuration is simply Lua code, so you can write your own code to do whatever you want.

Interested? Let’s get started!

Installing Hammerspoon

Hammerspoon is a regular Mac application. To install it by hand, you just need to download it from https://github.com/Hammerspoon/hammerspoon/releases/latest, unzip the downloaded file and drag it to your /Applications folder (or anywhere else you want).

If you are automation-minded like me, you probably use Homebrew and its plugin Cask to manage your applications. In this case, you can use Cask to install Hammerspoon:

brew cask install hammerspoon

When you run Hammerspoon for the first time, you will see its icon appear in the menubar, and a notification telling you that it couldn’t find a configuration file. Let’s fix that!

images/hammerspoon-startup.png

{{% tip %}} If you click on the initial notification, your web browser will open to the excellent Getting Started with Hammerspoon page, which I highly recommend you read for more examples. {{% /tip %}}

Your first Hammerspoon configuration

Let us start with a few simple examples. As tradition mandates, we will start with a “Hello World” example. Open $HOME/.hammerspoon/init.lua (Hammerspoon will create the directory upon first startup, but you need to create the file) in your favorite editor, and type the following:

hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "h" },
  function()
    hs.notify.show("Hello World!", "Welcome to Hammerspoon", "")
  end
)

Save the file, and from the Hammerspoon icon in the menubar, select “Reload config”. Apparently nothing will happen, but if you then press {{{keys(Ctrl ⌘ Alt h)}}} on your keyboard, you will see a notification on your screen welcoming you to the world of Hammerspoon.

images/hammerspoon-hello-world.png

Although it should be fairly self-explanatory, let us dissect this example to give you a clearer understanding of its components:

  • All Hammerspoon built-in extensions start with hs. In this case, {{{hsapi(hs.hotkey)}}} is the extension that handles keyboard bindings. It allows us to easily define which functions will be called in response to different keyboard combinations. You can even differentiate between the keys being pressed, released or held down if you need to. The other extension used in this example is {{{hsapi(hs.notify)}}}, which allows us to interact with the macOS Notification Center to display, react and interact with notifications.
  • Within hs.hotkey, the {{{hsapi(hs.hotkey,bindSpec)}}} function allows you to bind a function to a pressed key. Its first argument is a key specification which consists of a list (Lua lists and table literals are represented using curly braces) with two elements: a list of the key modifiers, and the key itself. In this example, { { "ctrl", "cmd", "alt" }, "h" } represents pressing {{{keys(Ctrl ⌘ Alt h)}}}.
  • The second argument to bindSpec is the function to call when the key is pressed. Here we are defining an inline anonymous function using function() ... end.
  • The callback function uses {{{hsapi(hs.notify,show)}}} to display the message. Take a quick look at the {{{hsapi(hs.notify)}}} documentation to get an idea of its extensive capabilities, including configuration of all aspects of a notification’s appearance and buttons, and the functions to call upon different user actions.

Try changing the configuration to display a different message or use a different key. After every change, you need to instruct Hammerspoon to reload its configuration, which you can do through its menubar item.

Debugging tools and the Hammerspoon console

As you start modifying your configuration, errors will happen, as they always do when coding. To help in development and debugging, Hammerspoon offers a console window where you can see any errors and messages printed by your Lua code as it executes, and also type code to be evaluated. It is a very useful tool while developing your Hammerspoon configuration.

To invoke the console, you normally choose “Console…” from the Hammerspoon menubar item. However, this is such a common operation, that you might find it useful to also set a key combination for showing the console. Most of Hammerspoon’s internal functionality is also accessible through its API. In this case, looking at the {{{hsapi(hs,,documentation for the main hs module)}}} reveals that there is an {{{hsapi(hs,toggleConsole)}}} function. Using the knowledge you have acquired so far, you can easily configure a hotkey for opening and hiding the console:

hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "y" }, hs.toggleConsole)

Once you reload your configuration, you should be able to use {{{keys(Ctrl ⌘ Alt y)}}} to open and close the console. Any Lua code you type in the Console will be evaluated in the main Hammerspoon context, so you can add to your configuration directly from there. This is a good way to incrementally develop your code before committing it to the init.lua file.

You may have noticed by now another common operation while developing Hammerspoon code: reloading the configuration, which you normally have to do from the Hammerspoon menu. So why not set up a hotkey to do that as well? Again, the {{{hsapi(hs)}}} module comes to our help with the {{{hsapi(hs,reload)}}} method:

hs.hotkey.bindSpec({ { "ctrl", "cmd", "alt" }, "r" }, hs.reload)

Another useful development tool is the hs command, which you can run from your terminal to get a Hammerspoon console. To install it, you can use the {{{hsapi(hs.ipc”,cliInstall)}}} function, which you can just add to your init.lua file to check and install the command every time Hammerspoon runs.

{{% warning %}} {{{hsapi(hs.ipc,cliInstall)}}} creates symlinks under /usr/local/ to the hs command and its manual page file, located inside the Hammerspoon application bundle. Under some circumstances (particularly if you build Hammerspoon from source, or if you install different versions of it), you may end up with broken symlinks. If the hs command stops working and hs.ipc.cliInstall() doesn’t fix it, look for broken symlinks left behind from old versions of Hammerspoon. Remove them and things should work again. {{% /warning %}}

Now you have all the tools for developing your Hammerspoon configuration. In the next installment we will look at how you can save yourself a lot of coding by using pre-made modules. In the meantime, feel free to look through my Hammerspoon configuration file for ideas, and please let me know your thoughts in the comments!

Using Spoons in Hammerspoon

In this second article about Hammerspoon, we look into Spoons, modules written in Lua which can be easily installed and loaded into Hammerspoon to provide ready-to-use functionality. Spoons provide a predefined API to configure and use them. They are also a good way to share your own work with other users.

See also the first article in this series.

Using a Spoon to locate your mouse

As a first example, we will use the MouseCircle spoon, which allows us to set up a hotkey that displays a color circle around the current location of the mouse pointer for a few seconds, to help you locate it.

To install the spoon, download its zip file from https://github.com/Hammerspoon/Spoons/raw/master/Spoons/MouseCircle.spoon.zip, unpack it, and double-click on the resulting MouseCircle.spoon file. Hammerspoon will install the Spoon under ~/.hammerspoon/Spoons/.

images/mousecircle.png

Once a Spoon is installed, you need to use the hs.loadSpoon() function to load it. Type the following in the Hammerspoon console, or add it to your init.lua file and reload the configuration:

hs.loadSpoon("MouseCircle")

After a spoon is loaded, and depending on what it does, you may need to configure it, assign hotkeys, and start it. A spoon’s API is available through the spoon.<SpoonName> namespace. To learn the API you need to look at the spoon documentation page. In the case of MouseCircle, a look at http://www.hammerspoon.org/Spoons/MouseCircle.html reveals that it has two methods (bindHotkeys() and show()) and one configuration variable (color) available under spoon.MouseCircle.

The first API call is spoon.MouseCircle:bindHotkeys(), which allows us to set up a hotkey that shows the mouse locator circle around the location of the mouse pointer. Let’s say we wanted to bind the mouse circle to {{{keys(Ctrl ⌘ Alt d)}}}. According to the MouseCircle documentation, the name for this action is show, so we can do the following:

spoon.MouseCircle:bindHotkeys({
  show = { { "ctrl", "cmd", "alt" }, "d" }
})

Once you do this, press the hotkey and you should see a red circle appear around the mouse cursor, and fade away after 3 seconds.

{{% tip %}} All spoons which offer the possibility of binding hotkeys have to expose it through the same API:

spoon.SpoonName:bindHotkeys({ action1 = keySpec1,
                              action2 = keySpec2, ... })

Each actionX is a name defined by the spoon, which refers to something that can be bound to a hotkey, and each keySpecX is a table with two elements: a list of modifiers and the key itself, such as { { "ctrl", "cmd", "alt" }, "d" }. {{% /tip %}}

The second API call in the MouseCircle spoon is show(), which triggers the functionality of showing the locator circle directly. Let’s try it – type the following in the console:

spoon.MouseCircle:show()

Most spoons are structured like this: you can set up hotkeys to trigger the main functionality, but you can also trigger it via method calls. Normally you won’t use these methods, but their availability makes it possible for you to use spoon functionality from our own configuration, or from other spoons, to create further automation.

spoon.MouseCircle.color is a public configuration variable exposed by the spoon, which specifies the color that will be used to draw the circle. Colors are defined according to the documentation for the {{{hsapi(hs.drawing.color)}}} module. Several color collections are supported, including the OS X system collections and a few defined by Hammerspoon itself. Color definitions are stored in Lua tables indexed by their name. For example, you can view the {{{hsapi(hs.drawing.color,hammerspoon)}}} table, including the color definitions, by using the convenient {{{hsapi(hs.inspect)}}} method on the console:

> hs.inspect(hs.drawing.color.hammerspoon)
{
  black = {
    alpha = 1,
    blue = 0.0,
    green = 0.0,
    red = 0.0
  },
  green = {
    alpha = 1,
    blue = 0.0,
    green = 1.0,
    red = 0.0
  },
  osx_red = {
    alpha = 1,
    blue = 0.302,
    green = 0.329,
    red = 0.996
  },
  osx_green = {
...

{{% tip %}}

Lua does not include a function to easily get the keys of a table so you have to use the {{{luafun(pairs)}}} function to loop over the key/value pairs of the table. The {{{hsapi(hs.inspect)}}} function is convenient, but to get just the list of tables and the color names, without the color definitions themselves, you can use the following code (if you type this in the console you have to type it all in a single line – and beware, the output is a long list):

for listname,colors in pairs(hs.drawing.color.lists()) do
  print(listname)
  for color,def in pairs(colors) do
    print("  " .. color)
  end
end

{{% /tip %}}

If we wanted to make the circle green, we can assign the configuration value like this:

spoon.MouseCircle.color = hs.drawing.color.hammerspoon.green

The next time you invoke the show() method, either directly or through the hotkey, you will see the circle in the new color.

{{% tip %}}

(We will look at this in more detail in a future installment about Lua, but in case you were wondering…)

You may have noticed that the configuration variable was accessed with a dot (spoon.MouseCircle.color), and we also used it for some function calls earlier (e.g. {{{hsapi(hs.notify,show)}}}, whereas for the show() method we used a colon (spoon.MouseCircle:show()). The latter is Lua’s object-method-call notation, and its effect is to pass the object on which the method is being called as an implicit first argument called self. This is simply a syntactic shortcut, i.e. the following two are equivalent:

spoon.MouseCircle:show()
spoon.MouseCircle.show(spoon.MouseCircle)

Note that in the second statement, we are calling the method using the dot notation, and explicitly passing the object as the first argument. Normally you would use colon notation, but the alternative can be useful when constructing function pointers. For example, if you wanted to manually bind a second key to show the mouse circle, you might initially try to use the following:

hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
                   spoon.MouseCircle:show)

But this results in an error. The correct way is to wrap the call in an anonymous function:

hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
                   function() spoon.MouseCircle:show() end)

Alternatively, you can use the {{{hsapi(hs.fnutils,partial)}}} function to construct a function pointer that includes the correct first argument:

hs.hotkey.bindSpec({ {"ctrl", "alt", "cmd" }, "p" },
                   hs.fnutils.partial(spoon.MouseCircle.show,
                                      spoon.MouseCircle))

This is more verbose than the previous example, but the technique can be useful sometimes. Although Lua is not a full functional language, it supports using functions as first-class values, and the {{{hsapi(hs.fnutils)}}} extension includes a number of functions that make it easy to use them.

{{% /tip %}}

By now you know enough to use spoons with Hammerspoon’s native capabilities: look for the ones you want, download and install them by hand, and configure them in your init.lua using their configuration variables and API. In the next sections you will learn more about the minimum API of spoons, and how to install and configure spoons in a more automated way.

The Spoon API

The advantage of using spoons is that you can count on them to adhere to a defined API, which makes it easier to automate their use. Although each spoon is free to define additional variable and methods, the following are standard:

  • SPOON:init() is called automatically (if it exists) by {{{hsapi(hs,loadSpoon)}}} after loading the spoon, and can be used to initialize variables or anything else needed by the Spoon.
  • SPOON:start() should exist if the spoon requires any ongoing or background processes such as timers or watchers of any kind.
  • SPOON:stop() should exist if start() does, to stop any background processes that were started by start().
  • SPOON:bindHotkeys(map) is exposed by spoons which allow binding hotkeys to certain actions. Its map argument is a Lua table with key/value entries of the following form: ACTION = { MODS, KEY }, where ACTION is a string defined by the spoon (multiple such actions can be defined), MODS is a list of key modifiers (valid values are =”cmd”=, =”alt”=, =”ctrl”= and =”shift”=), and KEY is the key to be bound, as shown in our previous example. All available actions for a spoon should be listed in its documentation.

Automated Spoon installation and configuration

Once you develop a complex Hammerspoon configuration using spoons, you may start wondering if there is an easy way to manage them. There are no built-in mechanisms for automatically installing spoons, but you can use a spoon called SpoonInstall that implements this functionality. You can download it from http://www.hammerspoon.org/Spoons/SpoonInstall.html. Once installed, you can use it to declaratively install, configure and run spoons. For example, with SpoonInstall you can use the MouseCircle spoon as follows:

hs.loadSpoon("SpoonInstall")
spoon.SpoonInstall:andUse("MouseCircle", {
                  config = {
                     color = hs.drawing.color.osx_red,
                  },
                  hotkeys = {
                     show = { { "ctrl", "cmd", "alt"}, "d" }
                  }})

If the MouseCircle spoon is not yet installed, spoon.SpoonInstall:andUse() will automatically download and install it, and set its configuration variables and hotkeys according to the declaration.

If there is nothing to configure in the spoon, spoon.SpoonInstall:andUse("SomeSpoon") does exactly the same as hs.loadSpoon("SomeSpoon"). But if you want to set configuration variables, hotkey bindings or other parameters, the following keys are recognized in the map provided as a second parameter:

  • config is a Lua table containing keys corresponding to configuration variables in the spoon. In the example above, config = { color = hs.drawing.color.osx_red } has the same effect as setting spoon.MouseCircle.color = hs.drawing.color.osx_red
  • hotkeys is a Lua table with the same structure as the mapping parameter passed to the bindHotkeys method of the spoon. In our example above, hotkeys = { show = { { "ctrl", "cmd", "alt"}, "d" } } automatically triggers a call to spoon.MouseCircle:bindHotkeys({ show = { { "ctrl", "cmd", "alt"}, "d" } }).
  • loglevel sets the log level of the logger attribute within the spoon, if it exists. The valid values for this attribute are ‘nothing’, ‘error’, ‘warning’, ‘info’, ‘debug’, or ‘verbose’.
  • start is a boolean value which indicates whether to call the Spoon’s start() method (if it has one) after configuring everything else.
  • fn specifies a function which will be called with the freshly-loaded Spoon object as its first argument. This can be used to execute other startup or configuration actions that are not covered by the other attributes. For example, if you use the {{{spoon(Seal)}}} spoon (a configurable launcher), you need to call its loadPlugins() method to specify which Seal plugins to use. You can achieve this with something like this:
    spoon.SpoonInstall:andUse("Seal",
      { hotkeys = { show = { {"cmd"}, "space" } },
        fn = function(s)
                 s:loadPlugins({"apps", "calc", "safari_bookmarks"})
             end,
             start = true,
      })
        
  • repo indicates the repository from where the Spoon should be installed if needed. Defaults to =”default”=, which indicates the official Spoon repository at http://www.hammerspoon.org/Spoons/. I keep a repository of unofficial Spoons at http://zzamboni.org/zzSpoons/, and others may be available by the time you read this.
  • disable can be set to true to disable the Spoon (easier than commenting it out when you want to temporarily disable a spoon) in your configuration.

{{% tip %}}

You can assign functions and modules to variables to improve readability of your code. For example, in my init.lua file I make the following assignment:

Install=spoon.SpoonInstall

Which allows me to write Install:andUse("MouseCircle", …​ ), which is shorter and easier to read.

{{% /tip %}}

Managing repositories and spoons using SpoonInstall

Apart from the andUse() “all-in-one” method, SpoonInstall has methods for specific repository- and spoon-maintenance operations. As of this writing, there are two Spoon repositories: the official one at http://www.hammerspoon.org/Spoons/, and my own at http://zzamboni.org/zzSpoons/, where I host some unofficial and in-progress Spoons.

The configuration variable used to specify repositories is SpoonInstall.repos. Its default value is the following, which configures the official repository identified as “default”:

{
  default = {
      url = "https://github.com/Hammerspoon/Spoons",
      desc = "Main Hammerspoon Spoon repository",
  }
}

To configure a new repository, you can define an extra entry in this variable. The following code creates an entry named “zzspoons” for my Spoon repository:

spoon.SpoonInstall.repos.zzspoons = {
   url = "https://github.com/zzamboni/zzSpoons",
   desc = "zzamboni's spoon repository",
}

After this, both “zzspoons” and “default” can be used as values to the repo attribute in the andUse() method, and in any of the other methods that take a repository identifier as a parameter. You can find the full API documentation at http://www.hammerspoon.org/Spoons/SpoonInstall.html.

Conclusion

Spoons are a great mechanism for structuring your Hammerspoon configuration. If you want an example of a working configuration based almost exclusively on Spoons, you can view my own Hammerspoon configuration at https://github.com/zzamboni/dot-hammerspoon.

Just Enough Lua to Be Productive in Hammerspoon, Part 1

Hammerspoon’s configuration files are written in Lua, so a basic knowledge of the language is very useful to be an effective user of Hammerspoon. In this 2-part article I will show you the basics of Lua so you can read and write Hammerspoon configuration. Along the way you will discover that Lua is a surprisingly powerful language.

Lua is a scripting language created in 1993, and focused from the beginning in being an embedded language for extending other applications. It is easy to learn and use while having pretty powerful features, and is frequently used in games, but also in many other applications including, of course, Hammerspoon.

The purpose of this section is to give you a quick overview of the Lua features and peculiarities you may find most useful for developing Hammerspoon policies. I assume you are a programmer who knows some other C-like language–if you already know C, Java, Ruby, Python, Perl, Javascript or some similar language, picking up Lua should be pretty easy. Instead of detailing every structure, I will focus on the aspects that are most different or that are most likely to trip you up as you learn it.

Flow control

Lua includes all the common flow-control structures you might expect. Some examples:

local info = "No package selected"
if pkg and pkg ~= "" then
   info, st = hs.execute("/usr/local/bin/brew info " .. pkg)
   if st == nil then
      info = "No information found about formula '" .. pkg .. "'!"
   end
end

In this example, in addition to the {{{luadoc(if,3.3.4,)}}} statement, you can see in the line that runs {{{hsapi(hs,execute)}}} that Lua functions can return multiple values (which is not the same as returning an array, which counts as a single value). Within the function, this is implemented simply by separating the values with commas in the return statement, like this: return val1, val2. You can also see in action the following operators:

  • ==== for equality;
  • ~= for inequality (in this respect it differs from most C-like languages, which use !=);
  • .. for string concatenation;
  • and for the logical AND operation (by extension, you can deduct that or and not are also available).
local doReload = false
for _,file in pairs(files) do
   if file:sub(-4) == ".lua" and (not string.match(file, '/[.]#')) then
      doReload = true
   end
end

In this example we see the {{{luadoc(for,3.3.5)}}} statement in its so-called generic form:

for <var> in <expression> do <block> end

This statement loops the variables over the values returned by the expressions, executing the block with each the consecutive value until it becomes nil.

{{% tip %}} Strictly speaking, expression is executed once and its value must be an iterator function, which returns one new value from the sequence every time it is called, returning nil at the end. {{% /tip %}}

The for statement also has a numeric form:

for <var> = <first>,<last>,<inc> do <block> end

This form loops the variable from the first to the last value, incrementing it by the given increment (defaults to 1) at each iteration.

Going back to our example, we can also learn the following:

  • The {{{luafun(pairs)}}} function, which loops over a table. We will learn more about Lua tables below, but they can be used to represent both regular and associative arrays. pairs() treats the files variable as an associative array, and returns in each iteration a key/value pair of its contents.
  • The _ variable, while not special per se, is used by convention in Lua for “throwaway values”. In this case we are not interested in the key in each iteration, just the value, so we assign the key to _, never to be used again.
  • Our first glimpse into the Lua {{{luadoc(string library,6.4)}}}, and the two ways in which it can be used:
    • In file:sub(-4), the colon indicates the object-oriented notation (see “Lua dot-vs-colon method access” below). This invokes the {{{luafun(string.sub)}}} function, automatically passing the file variable as its first argument. This statement is equivalent to string.sub(file, -4).
    • In string.match(file, '/'), we see the function notation used to call {{{luafun(string.match)}}}. Since the file variable is being passed as the first argument, you could rewrite this statement as file:match('/[.]'). In practice, I’ve found myself using both notations somewhat exchangeably - feel free to use whichever you find most comfortable.

Dot-vs-colon method access in Lua

You will notice that sometimes, functions contained within a module are called with a dot, and others with a colon. The latter is Lua’s object-method-call notation, and its effect is to pass the object on which the method is being called as an implicit first argument called self. This is simply a syntactic shortcut, i.e. the following two are equivalent:

file:match('/[.]')
string.match(file, '/')

Note that in the second statement, we are calling the method using the dot notation, and explicitly passing the object as the first argument. Normally you would use colon notation, but when you need a function pointer, you need to use the dot notation.

Functions

Functions are defined using the function keyword.

function leftDoubleClick(modifiers)
   local pos=hs.mouse.getAbsolutePosition() -- <1>
   hs.eventtap.event.newMouseEvent(
     hs.eventtap.event.types.leftMouseDown, pos, modifiers) -- <2>
      :setProperty(hs.eventtap.event.properties.mouseEventClickState, 2)
      :post() -- <3>
   hs.eventtap.event.newMouseEvent( -- <4>
     hs.eventtap.event.types.leftMouseUp, pos, modifiers):post()
end

In this example we can also see some examples of the Hammerspoon library in action, in particular two extremely powerful libraries: {{{hsapi(hs.mouse)}}} for interacting with the mouse pointer, and {{{hsapi(hs.eventtap)}}}, which allows you to both intercept and generate arbitrary system events, including key pressed and mouse clicks. This function simulates a double click on the current pointer position:

  1. We first get the current position of the mouse pointer using {{{hsapi(hs.mouse,getAbsolutePosition)}}}.
  2. We create a new mouse event of type {{{hsapi(hs.eventtap.event,types,=leftMouseDown=)}}} in the obtained coordinates and with the given modifiers.
  3. By convention, most Hammerspoon API methods return the same object on which they operate. This allows us to chain the calls as shown: setProperty() is called on the hs.eventtap object returned by newMouseEvent to set its type to a double click, and post() is called on the result to issue the event.
  4. Since we are generating system events directly, we also need to take care of generating a “mouse up” event at the end.

Function parameters are always optional, and those not passed will default to nil, so you need to do proper validation. In this example, the function can be called as leftDoubleClick(), without any parameters, which means the modifiers parameter might have a nil value. Looking at the {{{hsapi(hs.eventtap.event,newMouseEvent,documentation for newMouseEvent())}}}, we see that the parameter is optional, so for this particular function our use is OK.

You should try this function to see that it works. Adding it to you ~/.hammerspoon/init.lua function will make Hammerspoon define it the next time you reload your configuration. You could then try calling it from the console, but the easiest is to bind a hotkey that will generate a double click. For example:

hs.hotkey.bindSpec({ { "cmd", "ctrl", "alt" }, "p" },
                   leftDoubleClick)

Once you reload your config, you can generate a double click by moving the cursor where you want it and pressing {{{keys(Ctrl ⌘ Alt p)}}}. While this is a contrived example, the ability to generate events like this is immensely powerful in automating your system.

{{% tip %}}

By now you have seen that we are using {{{keys(Ctrl ⌘ Alt)}}} very frequently in our keybindings. To avoid having to type this every time, and since the modifiers are defined as an array, you can define them as variable. For example, I have the following at the top of my init.lua:

hyper = {"cmd","alt","ctrl"}
shift_hyper = {"cmd","alt","ctrl","shift"}

Then I simply use hyper or shift_hyper in my key binding declarations:

hs.hotkey.bindSpec({ hyper, "p" }, leftDoubleClick)

{{% /tip %}}

Until next time!

In the next installment, we will dive into Lua’s types and data structures. In the meantime, feel free to explore and learn on your own. If you need more information, I can recommend the following resources, which I have found useful:

Just Enough Lua to Be Productive in Hammerspoon, Part 2

In this second article of the “Just Enough Lua” series, we dive into Lua’s types and data structures.

{{% tip %}} If you haven’t already, be sure to read the first installment of this series to learn about the basic Lua concepts. {{% /tip %}}

Tables

Table are the only compound data type in Lua, and are used to implement arrays, associative arrays (commonly called “maps” or “hashes” in other languages), modules, objects and namespaces. As you can see, it is very important to understand them!

A table in Lua is a collection of values, which can be indexed either by numbers or by arbitrary strings (the two types of indices can coexist within the same table). Let’s go through a few examples that will give you an overview (you can type these in the Hammerspoon console as we go, or at the prompt of the hs command - keep in mind that some of the statements are broken across multiple lines here for formatting, but each statement should be type in a single line in the console).

Table literals are declared using curly braces:

> unicorns = {}  -- empty table
> people = { "Chris", "Aaron", "Diego" }  -- array
> handles = { Diego = "zzamboni",
              Chris = "cmsj",
              Aaron = "asmagill" } -- associative array

Indices are indicated using square brackets. Numeric indices start at 1 (not 0 as in most other languages). For identifier-like string indices, you can use the dot shortcut. Requesting a non-existent index returns nil:

> unicorns[1]
nil
> people[0]
nil
> people[1]
Chris
> handles['Diego']
zzamboni
> handles.Diego
zzamboni
> handles.Michael
nil

Within the curly-brace notation, indices that are not identifier-like (letters, numbers, underscores) need to be enclosed in quotes and square brackets. Values can be tables as well:

colors = { ["U.S."] = { "red", "white", "blue" },
           Mexico = { "green", "white", "red" },
           Germany = { "black", "red", "yellow" } }

With non-identifier indices, you cannot use the dot-notation. Also, to see a table within the Hammerspoon console, use {{{hsapi(hs.inspect)}}}:

> colors["U.S."]
table: 0x618000470400
> hs.inspect(colors.Mexico)
{ "green", "white", "red" }
> hs.inspect(colors)
{
  Germany = { "black", "red", "yellow" },
  Mexico = { "green", "white", "red" },
  ["U.S."] = { "red", "white", "blue" }
}

Iteration through an array is commonly done using the {{{luafun(ipairs)}}} functions. Note that it will only iterate through contiguous numeric indices starting at 1, so that it does not work well with “sparse” tables.

> for i,v in ipairs(people) do print(i, v) end
1   Chris
2   Aaron
3   Diego
> people[4]='John'
> for i,v in ipairs(people) do print(i, v) end
1   Chris
2   Aaron
3   Diego
4   John
> people[7]='Mike'
> for i,v in ipairs(people) do print(i, v) end
1   Chris
2   Aaron
3   Diego
4   John
> hs.inspect(people)
{ "Chris", "Aaron", "Diego", "John",
  [7] = "Mike"
}

The {{{luafun(pairs)}}} function, on the other hand, will iterate through all the elements in the table (both numeric and string indices), but does not guarantee their order. Both numeric and string indices can be mixed in a single table (although this gets confusing quickly unless you access everything using {{{luafun(pairs)}}}).

> for i,v in pairs(people) do print(i,v) end
1   Chris
2   Aaron
3   Diego
4   John
7   Mike
> for i,v in ipairs(handles) do print(i,v) end
<no output>
> for i,v in pairs(handles) do print(i,v) end
Aaron   asmagill
Diego   zzamboni
Chris   cmsj
> handles[1]='whoa'  -- assign the first numeric index
> hs.inspect(handles)
{ "whoa",
  Aaron = "asmagill",
  Chris = "cmsj",
  Diego = "zzamboni"
}
> for i,v in ipairs(handles) do print(i,v) end
1   whoa

The built-in {{{luadoc(table,6.6)}}} module includes a number of useful table-manipulation functions, including the following:

  • {{{luafun(table.concat)}}} for joining the values of a list in a single string (equivalent to join in other languages). This only joins the elements that would be returned by {{{luafun(ipairs)}}}.
    > table.concat(people, ", ")
    Chris, Aaron, Diego, John
        
  • {{{luafun(table.insert)}}} adds an element to a list, by default adding it to the end.
    > hs.inspect(people)
    { "Chris", "Aaron", "Diego", "John", "Bill",
      [7] = "Mike"
    }
    > table.insert(people, "George")
    > hs.inspect(people)
    { "Chris", "Aaron", "Diego", "John", "Bill", "George", "Mike" }
        

    Note how in the last example, the contiguous indices have finally caught up to 7, so the last element is no longer shown separately (and will now be included by {{{luafun(ipairs)}}}, {{{luafun(table.concat)}}}, etc.

  • {{{luafun(table.remove)}}} removes an element from a list, by default the last one. It returns the removed element.
    > for i=1,4 do print(table.remove(people)) end
    Mike
    George
    Bill
    John
    > hs.inspect(people)
    { "Chris", "Aaron", "Diego" }
        

Notable omissions from the language and the {{{luadoc(table,6.6)}}} module are “get keys” and “get values” functions, common in other languages. This may be explained by the flexible nature of Lua tables, so that those functions would need to behave differently depending on the contents of the table. If you need them, you can easily build your own. For example, if you want to get a sorted list of the keys in a table, you can use this function:

function sortedkeys(tab)
  local keys={}
  for k,v in pairs(tab) do table.insert(keys, k) end
  table.sort(keys)
  return keys
end

Tables as namespaces

Functions in Lua are first-class objects, which means they can be used like any other value. This means that functions can be stored in tables, and this is how namespaces (or “modules”) are implemented in Lua. We can inspect an manipulate them like any other table. Let us look at the {{{luadoc(table,6.6)}}} library itself. First, the module itself is a table:

> table
table: 0x61800046f740

Second, we can inspect its contents using the functions we know:

> hs.inspect(table)
{
  concat = <function 1>,
  insert = <function 2>,
  move = <function 3>,
  pack = <function 4>,
  remove = <function 5>,
  sort = <function 6>,
  sortedkeys = <function 7>,
  unpack = <function 8>
}

The function values themselves are opaque (we cannot see their code), but we can easily extend the module. For example, we could add our sortedkeys() function above to the table module for consistency. Lua allows us to specify the namespace of a function in its declaration:

function table.sortedkeys(tab)
  local keys={}
  for k,v in pairs(tab) do table.insert(keys, k) end
  table.sort(keys)
  return keys
end

All the Hammerspoon modules are implemented the same way:

> type(hs)
table
> type(hs.mouse)
table
> hs.inspect(hs.mouse)
{
  get = <function 1>,
  getAbsolutePosition = <function 2>,
  getButtons = <function 3>,
  getCurrentScreen = <function 4>,
  getRelativePosition = <function 5>,
  set = <function 6>,
  setAbsolutePosition = <function 7>,
  setRelativePosition = <function 8>,
  trackingSpeed = <function 9>
}

The common way of defining a new module in Lua is to create an empty table, and populate it with functions or variables as needed. For example, let’s put our double-click generator in a module. Create the file ~/.hammerspoon/doubleclick.lua with the following contents:

local mod={}

mod.default_modifiers={}

function mod.leftDoubleClick(modifiers)
  modifiers = modifiers or mod.default_modifiers
  local pos=hs.mouse.getAbsolutePosition()
  hs.eventtap.event.newMouseEvent(
    hs.eventtap.event.types.leftMouseDown, pos, modifiers)
    :setProperty(hs.eventtap.event.properties.mouseEventClickState, 2)
    :post()
  hs.eventtap.event.newMouseEvent(
    hs.eventtap.event.types.leftMouseUp, pos, modifiers):post()
end

function mod.bindto(keyspec)
  hs.hotkey.bindSpec(keyspec, mod.leftDoubleClick)
end

return mod

You can then, from the console, do the following:

> doubleclick=require('doubleclick')
> doubleclick.bindto({ {"ctrl", "alt", "cmd"}, "p" })
19:53:53     hotkey: Disabled previous hotkey ⌘⌃⌥P
             hotkey: Enabled hotkey ⌘⌃⌥P

You have written and loaded your first Lua module. Let’s try it out! Press {{{keys(Ctrl ⌘ Alt p)}}} while your cursor is over a word in your terminal or web browser, to select it as if you had double-clicked it. You can also change the modifiers used with it. For example, did you know that Cmd-double-click can be used to open URLs from the macOS Terminal application?

> doubleclick.default_modifiers={cmd=true}

Now try pressing {{{keys(Ctrl ⌘ Alt p)}}} while your pointer is over a URL displayed on your Terminal (you can just type one yourself to test), and it will open in your browser.

Note that the name doubleclick does not have any special meaning—it is a regular variable to which you assigned the value returned by require('doubleclick'), which is the value of the mod variable created within the module file (note that within the module file you use the local variable name to refer to functions and variables within itself). You could assign it to any name you want:

> a=require('doubleclick')
> a.leftDoubleClick()

The argument of the {{{luafun(require)}}} function is the name of the file to load, without the .lua extension. Hammerspoon by default adds your ~/.hammerspoon/ directory to its load path, along with any other default directories in your system. You can view the places where Hammerspoon will look for files by examining the package.path variable. On my machine I get the following:

> package.path
/Users/zzamboni/.hammerspoon/?.lua;/Users/zzamboni/.hammerspoon/?/
init.lua;/Users/zzamboni/.hammerspoon/Spoons/?.spoon/init.lua;/usr/
local/share/lua/5.3/?.lua;/usr/local/share/lua/5.3/?/init.lua;/usr/
local/lib/lua/5.3/?.lua;/usr/local/lib/lua/5.3/?/init.lua;./?.lua;
./?/init.lua;/Users/zzamboni/Dropbox/Personal/devel/hammerspoon/
hammerspoon/build/Hammerspoon.app/Contents/Resources/extensions/?.lua;
/Users/zzamboni/Dropbox/Personal/devel/hammerspoon/hammerspoon/build/
Hammerspoon.app/Contents/Resources/extensions/?/init.lua

{{% tip %}}

Hammerspoon automatically loads any modules under the hs namespace the first time you use them. For example, when you use {{{hsapi(hs.application)}}} for the first time, you will see a message in the console:

> hs.application.get("Terminal")
2017-10-31 06:47:15: -- Loading extension: application
hs.application: Terminal (0x61000044dfd8)

If you want to avoid these messages, you need to explicitly load the modules and assign them to variables, as follows:

> app=require('hs.application')
> app.get("Terminal")
hs.application: Terminal (0x610000e49118)

This avoids the console message and has the additional benefit of allowing you to use app (you can use whatever variable you want) instead of typing hs.application in your code. This is a matter of taste—I usually prefer to have the full descriptive names (makes the code easier to read), but when dealing with some of the longer module names (e.g. {{{hsapi(hs.distributednotifications)}}}), this technique can be useful.

{{% /tip %}}

Patterns

If you are familiar with regular expressions, you know how powerful they are for examining and manipulating strings in any programming language. Lua has {{{luadoc(patterns,6.4.1)}}}, which fulfill many of the same functions but have a different syntax and some limitations. They are used by many functions in the string library like {{{luafun(string.find)}}} and {{{luafun(string.match)}}}.

The following are some differences and similarities you need to be aware of when using patterns:

  • The dot (.) represents any character, just like in regexes.
  • The asterisk (*), plus sign (+) and question mark (?) represent “zero or more”, “one or more” and “one or none” of the previous character, just like in regexes. Unlike regexes, these characters can only be applied to a single character and not to a whole capture group (i.e. the regex (foo)+ is not possible).
  • Alternations, represented by the vertical bar (|) in regexes, are not supported.
  • The caret (^) and dollar sign ($) represent “beginning of string” and “end of string”, just like in regexes.
  • The dash (-) represents a non-greedy “zero or more” (i.e. match the shortest possible string instead of the longest one) of the previous character, unlike in regexes, in which it’s commonly indicate by a question mark following the corresponding * or + The regex .*? is equivalent to the Lua pattern .-.
  • The escape character is the ampersand (%) instead of the backslash (\).
  • Most character classes are represented by the same characters, but preceded by ampersand. For example %d for digits, %s for spaces, %w for alphanumeric characters.

For most common use cases, Lua patterns are enough, you just have to be aware of their differences. If you encounter something that really cannot be done, you can always resort to libraries like Lrexlib, which provide interfaces to real regex libraries. Unfortunately these are not included in Lua, so you would need to install them on your own.

Patterns, just like regular expressions, are commonly used for string manipulation, using primarily functions from the {{{luadoc(string,6.4)}}} library.

String manipulation

Lua includes the {{{luadoc(string,6.4)}}} library to implement common string manipulation functions, including pattern matching. All of these functions can be called either as regular functions, with the string as the first argument, or as method calls on the string itself, using the colon syntax (which, as we saw before, gets converted to the same call). For example, the following two are equivalent:

string.find(a, "^foo")
a:find("^foo")

You can find the full documentation in the Lua reference manual and many other examples in the Lua-users wiki String Library Tutorial. The following is a partial list of some of the functions I have found most useful:

  • {{{luafun(string.find,=string.find(str\, pat\, pos\, plain)=)}}} finds the pattern within the string. By default the search starts at the beginning of the string, but can be modified with the pos argument (index starts at 1, as with the tables). By default pat is intepreted as a Lua pattern, but this can be disabled by passing plain as a true value. If the pattern is not found, returns nil. If the pattern is found, the function returns the start and end position of the pattern within the string. Furthermore, if the pattern contains parenthesis capture groups, all groups are returned as well. For example:
    > string.find("bah", "ah")
    2   3
    > string.find("bah", "foo")
    nil
    > string.find("bah", "(ah)")
    2   3   ah
    > p1, p2, g1, g2 = string.find("bah", "(b)(ah)")
    > p1,p2,g1,g2
    1   3   b   ah
        

    Note that the return value is not a table, but rather multiple values, as shown in the last example.

    {{% tip %}}

It can sometimes be convenient to handle multiple values as a table or as separate entities, depending on the circumstances. For example, you may have a programmatically-constructed pattern with a variable number of capture groups, so you cannot know to how many variables you need to assign the result. In this case, the {{{luafun(table.pack)}}} and {{{luafun(table.unpack)}}} functions can be useful.

{{{luafun(table.pack)}}} takes a variable number of arguments and returns them in a table which contains an array component containing the values, plus an index n containing the total number of elements:

> res = table.pack(string.find("bah", "(b)(ah)"))
> res
table: 0x608000c76e80
> hs.inspect(res)
{ 1, 3, "b", "ah",
  n = 4
}

{{{luafun(table.unpack)}}} does the opposite, expanding an array into separate values which can be assigned to separate values as needed, or passed as arguments to a function:

> args={"bah", "(b)(ah)"}
> string.find(args)
[string "return string.find(args)"]:1:
  bad argument #1 to 'find' (string expected, got table)
> string.find(table.unpack(args))
1   3   b   ah

{{% /tip %}}

  • {{{luafun(string.match,=string.match(str\, pat\, pos)=)}}} is similar to string.find, but it does not return the positions, rather it returns the part of the string matched by the pattern, or if the pattern contains capture groups, returns the captured segments:
    > string.match("bah", "ah")
    ah
    > string.match("bah", "foo")
    nil
    > string.match("bah", "(b)(ah)")
    b   ah
        
  • {{{luafun(string.gmatch,=string.gmatch(str\, pat)=)}}} returns a function that returns the next match of pat within str every time it is called, returning nil when there are no more matches. If pat contains capture groups, they are returned on each iteration.
    > a="Hammerspoon is awesome!"
    > f=string.gmatch(a, "(%w+)")
    > f()
    Hammerspoon
    > f()
    is
    > f()
    awesome
    > f()
        

    Most commonly, this is used inside a loop:

    > for cap in string.gmatch(a, "%w+") do print(cap) end
    Hammerspoon
    is
    awesome
        
  • {{{luafun(string.format,=string.format(formatstring\, …​)=)}}} formats a sequence of values according to the given format string, following the same formatting rules as the ISO C sprintf() function. It additionally supports a new format character %q, which formats a string value in a way that can be read back by Lua, escaping or quoting characters as needed (for example quotes, newlines, etc.).
  • {{{luafun(string.len,=string.len(str)=)}}} returns the length of the string.
  • {{{luafun(string.lower,=string.lower(str)=)}}} and {{{luafun(string.upper,=string.upper(str)=)}}} convert the string to lower and uppercase, respectively.
  • {{{luafun(string.gsub,=string.gsub(str\, pat\, rep\, n)=)}}} is a very powerful string-replacement function which hides considerably more power than its simple syntax would lead you to believe. In general, it replaces all (or the first n) occurrences of pat in str with the replacement rep. However, rep can take any of the following values:
    • A string which is used for the replacement. If the string contains %m, where m is a number, the it is replaced by the m-th captured group (or the whole match if m is zero).
    • A table which is consulted for the replacement values, using the first capture group as a key (or the whole match if there are no captures). For example:
      > a="Event type codes: leftMouseDown=$leftMouseDown, rightMouseDown=$rightMouseDown, mouseMoved=$mouseMoved"
      > a:gsub("%$(%w+)", hs.eventtap.event.types)
      Event type codes: leftMouseDown=1, rightMouseDown=3, mouseMoved=5   3
              
    • A function which is executed with the captured groups (or the whole match) as an argument, and whose return value is used as the replacement. For example, using the os.getenv function, we can easily replace environment variables by their values in a string:
      > a="Hello $USER, your home directory is $HOME"
      > a:gsub("%$(%w+)", os.getenv)
      Hello zzamboni, your home directory is /Users/zzamboni  2
              

    Note that gsub returns the modified string as its first return value, and the number of replacements it made as the second (2 in the example above). If you don’t need the number, you can simply ignore it (you don’t even need to assign it). Also note that gsub does not modify the original string, only returns a copy with the changes:

    > b = a:gsub("%$(%w+)", os.getenv)
    > b
    Hello zzamboni, your home directory is /Users/zzamboni
    > a
    Hello $USER, your home directory is $HOME
        

Keep learning!

You know now enough Lua to start being productive with Hammerspoon. You’ll pick up more details as you play with it. If you need more information, I can recommend the following resources, which I have found useful:

Have fun!

First release of “Learning Hammerspoon”

I am happy to announce the first release of my new book “Learning Hammerspoon”, a book devoted to using Hammerspoon to make using your Mac easier, faster and more fun.

You can find more information, read a free sample, and purchase it at LeanPub.

{{< hsbookimglink >}}

This book is based partly on blog posts published in my blog, but with a lot of new material. In this first release, the following chapters are finished:

  • Getting started with Hammerspoon
  • Using Spoons in Hammerspoon
  • Just enough Lua to be productive with Hammerspoon
  • Using and extending Seal

It is not complete yet, but already includes a lot of useful information. I will be updating it frequently, and of course you get all updates for free.

I look forward to your feedback! Feel free to message me through the /Email the author/ page at LeanPub.

New release of “Learning Hammerspoon” is out!

{{< hsbookimglink style=”float:right” >}}

I am happy to announce the third release of my book Learning Hammerspoon, which includes the following changes:

  • A brand-new chapter: Writing your own extensions and Spoons. It is not fully finished yet, but you can find already a complete, working example of a new spoon, which you can use as a starting point for creating your own. I will continue adding more details over the next few days, but in the meantime please let me know how you like it: is the example meaningful? Are the instructions easy to follow?
  • Two new introductory sections: The Hyper Key, about useful ways of defining a common set of modifiers for your Hammerspoon keybindings; and Keeping private information separate, about loading separate files into your Hammerspoon configuration.
  • Multiple overall improvements in terms of wording, figures and formatting.
  • One invisible backend change, but which bears mentioning: this book is now generated using Markua sources instead of the default Leanpub Markdown format used before. This has very little impact on the final book as you see it, but it will make it easier in the future to handle more complex formatting as needed.

I hope you enjoy it! As usual, you can find more information, read a free sample, and purchase it at LeanPub.

Elvish

Bang-Bang (!!, !$) Shell Shortcuts in Elvish

(Updated on March 19th, 2018 to use the new Elvish Package Manager)

The bash shortcuts (maybe older? I’m not sure in which shell these originated) for “last command” (!!) and “last argument of last command” (!$) are, for me at least, among the most strongly imprinted in my muscle memory, and I use them all the time. Although these shortcuts are not available in Elvish by default, they are easy to implement. I have written a module called bang-bang which you can readily use as follows:

  • Use epm to install my elvish-modules package (you can also add this to your rc.elv file to have the package installed automatically if needed):
    use epm
    epm:install github.com/zzamboni/elvish-modules
        
  • In your rc.elv (see mine as an example), add the following to load the bang-bang module and to set up the appropriate keybindings:
    use github.com/zzamboni/elvish-modules/bang-bang
        

That’s it! Start a new shell window, and test how command-history mode can be invoked by the ! key. Assuming your last command was ls -l ~/.elvish/rc.elv, when you press ! you will see the following:

bang-lastcmd [A C] _
! ls -l .elvish/rc.elv
0 ls
1 -l
2/$ .elvish/rc.elv
Alt-! !

If you press ! again, the whole last command will be inserted. If you press $ (or 2), only the last argument will be inserted. You can insert any other component of the previous command using its corresponding number. If you want to insert an exclamation sign, you can press Alt-!.

Note that by default, Alt-! will also be bound to trigger this mode, so you can fully replace the default “last command” mode in Elvish.

Have fun with Elvish!

Using and writing completions in Elvish

Like other Unix shells, Elvish has advanced command-argument completion capabilities. In this article I will explore the existing completions, and show you how you can create your own (and contribute them back to the community).

Using existing completions

There is a growing body of shell completions that you can simply load and use.

Elvish has a still-small but growing collection of completions that have been created by its users. These are a few that I know of (let me know if you know others!):

  • My own zzamboni/elvish-completions package, which contains completions for git (providing automatically-generated completions for all commands and their options, plus hand-crafted argument completions for many of them), ssh, vcsh, cd, and a few of Elvish’s built-in functions and modules. It also contains comp, a framework for building completers, which we will explore in more detail below. To use any of these modules, you just need to install the elvish-completions package, and then load the modules you want. For example:
    epm:install &silent-if-installed github.com/zzamboni/elvish-completions
    use github.com/zzamboni/elvish-completions/vcsh
    use github.com/zzamboni/elvish-completions/cd
    use github.com/zzamboni/elvish-completions/ssh
    use github.com/zzamboni/elvish-completions/builtins
    use github.com/zzamboni/elvish-completions/git
        
  • xiaq’s edit.elv/compl/go.elv, which provides extensive hand-crafted completions for go. You can also install this one as an Elvish package:
    epm:install &silent-if-installed github.com/xiaq/edit.elv
    use github.com/xiaq/edit.elv/compl/go
    go:apply
        
  • occivink’s completers.elv file, which contains completers for kak, ssh, systemctl, ffmpeg and a few other commands.
  • Tw’s completer/ files, which contains completions for adb, git and ssh.
  • SolitudeSF’s completers.elv file, which contains completers for cd, kak, kitty, git, man, pkill and quite a few other commands.

As of this writing, there is no “official” collection of Elvish completions, so feel free to look at the existing ones and choose/use the ones that work best for you.

Since the collection is not yet very big, it’s likely you will want to build your own completions. This is what the next section is about.

Creating your own completions

Elvish has a simple but powerful argument-completion mechanism. You can find the full documentation in the Elvish reference, but let’s take a quick look here.

Basic (built-in) argument completion mechanisms

Command argument completion in Elvish is implemented by functions stored inside $edit:completion:arg-completer. This variable is a map in which the indices are command names, and the values are functions which must receive a variable number of arguments. When the user types cat Space Tab, the function stored in $edit:completion:arg-completer[cat] (if any) is called, as follows:

$edit:completion:arg-completer[cat] cat ''

The function receives the full current command line as arguments, including the current argument, which might be empty as in the example above, or be a partially typed string. For example, if the user types cat f Tab, the completer function will be called like this:

$edit:completion:arg-completer[cat] cat 'f'

The completion function must use its arguments to determine the appropriate completions at that point, and return them by one of the following methods (which can be combined):

  • Output the completions to stdout, one per line;
  • Output the completions to the data stream (using put);
  • Output the completions using the edit:complex-candidate command, which can additionally specify a suffix to append to the completion in the completion menu or in the returned value, and a style to use (as accepted by edit:styled). The full syntax of edit:complex-candidate is as follows:
    edit:complex-candidate &code-suffix='' &display-suffix='' &style='' $string
        

    $string is the option to display; &code-suffix indicates a suffix to be appended to the completion string when the user selects it; &display-suffix indicates a suffix to be shown in the completion menu (but which is not returned as part of the completion); and &style indicates a text style to use in the completion menu.

Keep in mind that the options returned by the completion function are additionally filtered by what the user has typed so far. This means that the last argument can usually be ignored, since Elvish will automatically do the filtering. An exception to this is if you want to return different types of things depending on what the user has typed already. For example, if the last argument start with -, you may want to return the possible command-line options, and return regular argument completions otherwise.

Example #1: A very simple completer for the brew command:

edit:completion:arg-completer[brew] = [@cmd]{
  len = (count $cmd)
  if (eq $len 2) {
    if (has-prefix $cmd[-1] -) {
      put '--version' '--verbose'
    } else {
      put install uninstall
    }
  } elif (eq $len 3) {
    brew search | eawk [l @f]{ put $@f }
  }
}

If the function receives two arguments, we check to see if the last argument begins with a dash. If so, we return the possible command-line options, otherwise we return the two commands install and uninstall. If we receive three arguments (i.e. we are past the initial command), we return the list of possible packages to install or uninstall.

You may noticed that there are many cases that this simple function does not handle correctly. For example, if you type brew --verbose Space Tab, you get the list of packages as completion, which does not make sense at that point. We will look at more complex and complete completion functions next.

The first step to more complex completions is the edit:complete-getopt command, which allows us to specify a sequence of positional completion functions. The general syntax of the command is:

edit:complete-getopt $args $opts $handlers

Please see its documentation for a full description of the arguments.

Example #2: The completer for brew shown before can be specified like this:

edit:completion:arg-completer[brew] = [@cmd]{
  edit:complete-getopt $cmd[1:] \
  [[&long=version] [&long=verbose]] \
  [
    [_]{ put install uninstall }
    [_]{ brew search | eawk [_ @f]{ put $@f } }
    ...
  ]
}

This new completer overcomes a few of the limitations in our previous attempt. For one, the install and uninstall commands are now properly completed even if you specify options before. Furthermore, the ... at the end of the handler list indicates that the previous one (the package names) will be repeated for all further arguments - this makes it possible to complete multiple package names to install or uninstall. However, it still has some limitations! For example, it will give you all existing packages as possible arguments to uninstall, which only accepts already installed packages.

In addition to complete-getopt, Elvish includes a few other functions to help build completers:

  • edit:complete-filename produces a listing of all the files and directories in the directory of its argument, and is the default completion function when no other completer is specified. See its documentation for full details.
  • edit:complete-sudo provides completions for commands like sudo which take a command as their first argument. It is the default completer for the sudo command, so that if you type sudo Space Tab, you get a list of all the commands on your execution path. It can be reused for other commands, for example time:
    edit:completion:arg-completer[time] = $edit:complete-sudo~
        

Finally, note that if $edit:completion:arg-completer[''] exists, it will be called as a fall-back completer if no command-specific argument completer exists. You can see that the default completer is edit:complete-filename, as mentioned before:

~> put $edit:completion:arg-completer['']
▶ $edit:complete-filename~

With the tools you know so far, you can already create fairly complex completers. In the next section, we will explore comp, an external library I wrote to make it easier to specify complex completion trees.

Complex completions using the comp framework

The built-in completion functions make it possible to build any completer you want. However, you might realize that for more complex cases, the specifications can be quite complex. For this reason, I wrote the comp library as a framework to more easily specify completion functions. The basic Elvish mechanisms and functions are still used in the backend, so you can rest assured about their compatibility with the basic mechanisms.

As a first step, if you haven’t done so already, you should install the elvish-completions package using epm:

use epm
epm:install github.com/zzamboni/elvish-completions

From the file where you will define your completions (or from your interactive session if you just want to play with it), load the comp module:

use github.com/zzamboni/elvish-completions/comp

The main entry points for this module are comp:item, comp:sequence and comp:subcommands. Each one receives a single argument containing a “completion definition”, which indicates how the completions will be produced. Each one receives a different kind of completion structure, and returns a ready-to-use completion function, which can be assigned directly to an element of $edit:completion:arg-completer. A simple example:

edit:completion:arg-completer[foo] = (comp:item [ bar baz ])

If you type this in your terminal, and then type foo<space> and press Tab, you will see the appropriate completions:

> foo <Tab>
 COMPLETING argument _
 bar  baz

To create completions for new commands, your main task is to define the corresponding completion definition. The different types of definitions and functions are explained below, with examples of the different available structures and features.

Note: the main entry points return a ready-to-use argument handler function. If you ever need to expand a completion definition directly (maybe for some advanced usage), you can call comp:-expand-item, comp:-expand-sequence and comp:-expand-subcommands, respectively. These functions all take the definition structure and the current command line, and return the appropriate completions at that point.

We now look at the different types of completion definitions understood by comp.

Items

The base building block is the “item”, can be one of the following:

  • An array containing all the potential completions (it can be empty, in which case no completions are provided). This is useful for providing a static list of completions.
  • A function which returns the potential completions (it can return nothing, in which case no completions are provided). The function should have one of the following arities, which affect which arguments will be passed to it (other arities are not valid, and in that case the item will not be executed):
    • If it takes no arguments, no arguments are passed to it.
    • If it takes a single argument, it gets the current (last) component of the command line @cmd; this is just like the handler functions understood by the edit:complete-getopt command.
    • If it takes a rest argument, it gets the full current command line (the contents of @cmd); this is just like the functions assigned to $edit:completion:arg-completer.

Example #3: a simple completer for cd

In this case, we define a function which receives the current “stem” (the part of the filename the user has typed so far) and offers all the relevant files, then filters those which are directories, and returns them as completion possibilities. We pass the function directly as a completion item to comp:-expand.

fn complete-dirs [arg]{ put {$arg}* | each [x]{ if (-is-dir $x) { put $x } } }
edit:completion:arg-completer[cd] = (comp:item $complete-dirs~)

For file and directory completion, you can use the utility function comp:files instead of defining your own function (see Utility functions). comp:files uses edit:complete-filename in the backend but offers a few additional filtering options:

edit:completion:arg-completer[cd] = (comp:item [arg]{ comp:files $arg &dirs-only })
Sequences and command-line options

Completion items can be aggregated in a sequence of items and used with the comp:sequence function when you need to provide different completions for different positional arguments of a command, including support for command-line options at the beginning of the command (comp:sequence uses edit:complete-getopt in the backend, but provides a few additional convenient features). The definition structure in this case has to be an array of items, which will be applied depending on their position within the command parameter sequence. If the the last element of the list is the string ... (three periods), the next-to-last element of the list is repeated for all later arguments. If no completions should be provided past the last argument, simply omit the periods. If a sequence should produce no completions at all, you can use an empty list []. If any specific elements of the sequence should have no completions, you can specify { comp:empty } or [] as its value.

If the &opts option is passed to the comp:sequence function, it must contain a single definition item which produces a list of command-line options that are allowed at the beginning of the command, when no other arguments have been provided. Options can be specified in either of the following formats:

  • As a string which gets converted to a long-style option; e.g. all to specify the --all option. The string must not contain the dashes at the beginning.
  • As a map in the style of complete-getopt, which may contain the following keys:
    • short for the short one-letter option;
    • long for the long-option string;
    • desc for a descriptive string which gets shown in the completion menu;
    • arg-mandatory or arg-optional: either one but not both can be set to $true to indicate whether the option takes a mandatory or optional argument;
    • arg-completer can be specified to produce completions for the option argument. If specified, it must contain completion item as described in Items, and which will be expanded to provide completions for that argument’s values.

Simple example of a completion data structure for option -t (long form --type), which has a mandatory argument which can be elv, org or txt:

[ &short=t
  &long=type
  &desc="Type of file to show"
  &arg-mandatory=$true
  &arg-completer= [ elv org txt ]
]

Note: options are only offered as completions when the use has typed a dash as the first character. Otherwise the argument completers are used.

Example #4: we can improve on the previous completer for cd by preventing more than one argument from being completed (only the first argument will be completed using complete-dirs, since the list does not end with ...):

edit:completion:arg-completer[cd] = (comp:sequence [ [arg]{ comp:files $arg &dirs-only }])

Example #5: a simple completer for ls with a subset of its options. Note that -l and -R are only provided as completions when you have not typed any filenames yet. Also note that we are using comp:files to provide the file completions, and the ... at the end of the sequence to use the same completer for all further elements.

ls-opts = [
  [ &short=l                 &desc='use a long listing format' ]
  [ &short=R &long=recursive &desc='list subdirectories recursively' ]
]
edit:completion:arg-completer[ls] = (comp:sequence &opts=$ls-opts [ $comp:files~ ... ])

Example #6: See the ssh completer for a real-world example of using sequences.

Subcommands

Finally, completion sequences can be aggregated into subcommand structures using the comp:subcommands function, to provide completion for commands such as git, which accept multiple subcommands, each with their own options and completions. In this case, the definition is a map indexed by subcommand names. The value of each element can be a comp:item, a comp:sequence or another comp:subcommands (to provide completion for sub-sub-commands, see the example below for vagrant). The comp:subcommands function can also receive the &opts option to generate any available top-level options.

Example #7: let us reimplement our completer for the brew package manager, but now with support for the install, uninstall and cat commands. install and cat gets as completions all available packages (the output of the brew search command), while uninstall only completes installed packages (the output of brew list). Note that for install and uninstall we automatically extract command-line options from their help messages using the comp:extract-opts function (wrapped into the -brew-opts function), and pass them as the &opts option in the corresponding sequence functions. Also note that all &opts elements get initialized at definition time (they are arrays), whereas the sequence completions get evaluated at runtime (they are lambdas), to automatically update according to the current packages. The cat command sequence allows only one option. The load-time initialization of the options incurs a small delay, and you could replace these with lambdas as well so that the options are computed at runtime. Note also the usage of the comp:decorate function to colorize the package names in different colors for each command.

fn -brew-opts [cmd]{
  brew $cmd -h | take 1 | \
  comp:extract-opts &regex='--(\w[\w-]*)' &regex-map=[&long= 1]
}
brew-completions = [
  &install= (comp:sequence &opts= [ (-brew-opts install) ] \
    [ { brew search | comp:decorate &style=green } ... ]
  )
  &uninstall= (comp:sequence &opts= [ (-brew-opts uninstall) ] \
    [ { brew list | comp:decorate &style=red }   ... ]
  )
  &cat= (comp:sequence [{ brew search | comp:decorate &style=blue }])
]
edit:completion:arg-completer[brew] = (comp:subcommands \
  &opts= [ version verbose ] $brew-completions
)

Note that in contrast to our previous brew completer, this definition is much more expressive, accurate, and much easier to extend.

Example #8: a simple completer for a subset of vagrant, which receives commands which may have subcommands and options of their own. Note that the value of &up is a comp:sequence, but the value of &box is another comp:subcommands which includes the completions for box add and box remove. Also note the use of the comp:extract-opts function to extract the command-line arguments automatically from the help messages. The output of the vagrant help messages matches the default format expected by comp:extract-opts, so we don’t even have to specify a regular expression like for brew.

Tip: note that the values of &opts are functions (e.g. { vagrant-up -h | comp:extract-opts }) instead of arrays (e.g. ( vagrant up -h | comp:extract-opts )). As mentioned in the previous example, both are valid, but in the latter case they are all initialized at load time (when the data structure is defined), which might introduce a delay, particularly with more command definitions. By using functions the options are only extracted at runtime when the completion is requested. For further optimization, vagrant-opts could be made to memoize the values so that the delay only occurs the first time.

vagrant-completions = [
  &up= (comp:sequence [] \
    &opts= { vagrant up -h | comp:extract-opts }
  )
  &box= (comp:subcommands [
      &add= (comp:sequence [] \
        &opts= { vagrant box add -h | comp:extract-opts }
      )
      &remove= (comp:sequence [ { \
            vagrant box list | eawk [_ @f]{ put $f[0] } \
        } ... ] \
        &opts= { vagrant box remove -h | comp:extract-opts }
      )
])]

edit:completion:arg-completer[vagrant] = (comp:subcommands \
  &opts= [ version help ] $vagrant-completions
)

Example #9: See the git completer for a real-world subcommand completion example, which also shows how extensively auto-population of subcommands and options can be done by extracting information from help messages.

Utility functions

The comp module includes a few utility functions, some of which you have seen already in the examples.

comp:decorate maps its input through edit:complex-candidate with the given options. Can be passed the same options as edit:complex-candidate. In addition, if &suffix is specified, it is used to set both &display-suffix and &code-suffix. Input can be given either as arguments or through the pipeline:

> comp:decorate &suffix=":" foo bar
▶ (edit:complex-candidate foo &code-suffix=: &display-suffix=: &style='')
▶ (edit:complex-candidate bar &code-suffix=: &display-suffix=: &style='')
> put foo bar | comp:decorate &style="red"
▶ (edit:complex-candidate foo &code-suffix='' &display-suffix='' &style=31)
▶ (edit:complex-candidate bar &code-suffix='' &display-suffix='' &style=31)

comp:extract-opts takes input from the pipeline and extracts command-line option data structures from its output. By default it understand the following common formats:

-o, --option                Option description
-p, --print[=<what>]        Option with an optional argument
    --select <type>         Option with a mandatory argument

Typical use would be to populate an &opts element with something like this:

comp:sequence &opts= { vagrant -h | comp:extract-opts } [ ... ]

The regular expression used to extract the options can be specified with the &regex option. Its default value (which parses the common formats shown above) is:

&regex='^\s*(?:-(\w),?\s*)?(?:--?([\w-]+))?(?:\[=(\S+)\]|[ =](\S+))?\s*?\s\s(\w.*)$'

The mapping of capture groups from the regex to option components is defined by the &regex-map option. Its default value (which also shows the available fields) is:

&regex-map=[&short=1 &long=2 &arg-optional=3 &arg-mandatory=4 &desc=5]

At least one of short or long must be present in regex-map. The arg-optional and arg-mandatory groups, if present, are handled specially: if any of them is not empty, then its contents is stored as arg-desc in the output, and the corresponding arg-mandatory / arg-optional is set to $true.

If &fold is $true, then the input is preprocessed to join option descriptions which span more than one line (the heuristic is not perfect and may not work in all cases, also for now it only joins one line after the option).

Contributing your completions

So you have created a brand-new completion function and would like to share it with the Elvish community. Nothing could be easier! You have two main options:

  • Publish them on your own. For example, if you put your .elv files into their own repository in GitHub or Gitlab, they are ready to be installed and used using epm.
  • Contribute it to an existing repository (for example elvish-completions). Just add your files, submit a pull request, and you are done.

I hope you have found this tutorial useful. Please let me know in the comments if you have any questions, feedback or if you find something that is incorrect.

Now, go have fun with Elvish!

Elvish, an awesome Unix shell

2019/09/12: Updated with some new links and information based on my later usage of Elvish. See https://zzamboni.org/tags/elvish for other things I have written about it.

I’m always on the lookout for new toys, particularly if they make my work more productive or enjoyable. For a couple of months now I have been using a new Unix shell called Elvish as my default login shell on macOS, and I love it. It’s a young project but very usable and with some very nice features.

Here are a few of the things I like about it:

  • The Elvish language is clean and powerful, with clear syntax. It supports name spaces, exception handling and proper data structures, including lists and maps.
  • It has a rich built-in library of functions, including string manipulation, regex matching, JSON parsing, etc.
  • Commands and functions can output two types of streams: bytes (what we usually see as standard output/error) and data (data values, potentially containing structured data), which makes it very flexible.
  • It has very nice interactive features, including as-you-type syntax checking and highlighting, asynchronous prompt themes, support for custom completions, a built-in command-line file browser, prompt hooks, configurable keybindings, and many others.
  • Exceptions! In Elvish, if a command exits with a non-zero code, it generates an exception that stops the execution of the current script or function. This breaks completely with the Unix-shell tradition of silently setting return codes, and it takes a while to get used to it, but once you do it’s a very powerful idea, as it forces you to think about failures and to plan for them in your code.
  • It is extensively documented. Take a look at https://elv.sh and you will see.

Elvish is very powerful, but it’s different enough from other shells that it’s worth your time to read through some of the documentation when you start. I would recommend the Some Unique Semantics page, which assumes you know other shells already. From there you can move to some of the other tutorials and reference documentation.

Of course, Elvish is not without its quirks and drawbacks. None of these has been a deal-breaker for me, but just for completeness:

  • It’s very young. Occasionally I still encounter crashes, but they are few and far between, and the developer is always very responsive. Also, the language is still subject to change and there are still backwards-incompatible changes with relative frequency. If you are looking for absolute stability, it’s not yet ready. (2019/06/12 update: it has been months (probably at least a year) without encountering a single crash)
  • Its language syntax is still a bit quirky. Spacing is sometimes important. For example, in the if=/=else construct, the ”} else {” has to be just like that—with spaces, and in a single line, for it to be recognized by the parser. (2019/06/12 update: the reason for this is explained in Effective Elvish)
  • The data/byte separation is not fully clear (at least to me) yet. Sometimes the data stream can be interpreted as bytes as well, sometimes it does not. The language is still evolving, so I am sure this will become clearer in the future. (2019/06/12 update: this is just a matter of getting used to it)
  • There is not yet a large body of code/scripts you can use. This is very noticeable in completions—while Elvish supports completions, there are very few implementations. Coming from Fish, which has an impressive library of custom completions out-of-the-box, this lack is very noticeable, particularly with complex commands like git. (2019/06/12 update: there is still not as much as for Fish or other shells, but steadily increasing. See Awesome Elvish for a list)

I am very happy with Elvish, and if you are interested in this sort of thing, I encourage you to take a look. If you need a starting point, you can use my configuration files at as an example of the kind of things you can do with it.

I will post more Elvish tips and tricks over time.

Other

New book: Literate Config

{{< leanpubbook book=”lit-config” style=”float:right” >}}

I am happy to announce the new release of my new book “Literate Config”, devoted to the use of Literate Programming for writing and documenting configuration files with Emacs and org-mode.

This is the first in my planned “Geek Booklets” series, which will include short texts about specific topics. You can get at LeanPub, like all my other books. You can get it for free or for a price of your choosing. You can also read it online.

I hope you enjoy it! Your feedback, as usual, is welcome.

Nuevo libro “Utilerías de Unix”

(I usually write in English, but this new book is in Spanish, so the announcement is also in Spanish. An English version of this book is in the works)

Me complace anunciar la primera edición de mi nuevo libro, y mi primer libro en Español, “Utilerías de Unix”, en el que podrás aprender a utilizar algunas de las herramientas más útiles que se encuentran en un sistema Unix/Linux para automatizar y hacer más eficientemente todo tipo de tareas.

Puedes obtener más información, leer una muestra gratuita y comprarlo en LeanPub.

{{< utileriasbookimglink >}}

¡Espero lo disfrutes! Favor de enviarme tus comentarios a través de la página /Escribe al Autor/ en LeanPub.

Automating Leanpub book publishing with Hammerspoon and CircleCI

I am the author of two books: /Learning CFEngine/ and /Learning Hammerspoon/, both self-published using Leanpub. The source of my books is kept in GitHub repositories. In this post I will show you how I use the Leanpub API together with Hammerspoon and CircleCI as part of my workflow, to automate and monitor the building, previewing and publishing of my books.

file:images/hammerspoon-github-circleci-leanpub-transp.jpg

{{% tip %}} The Hammerspoon section of this post is Mac-specific (since Hammerspoon is a Mac-only application), but the integration between GitHub, CircleCI and Leanpub can be applied regardless of the OS you use. {{% /tip %}}

Leanpub basics

First, some basic concepts about the Leanpub API. See the documentation for the full details, I’m only mentioning clear some things you need to know to understand the rest of this post.

  • Book slug: Each Leanpub book is identified by a slug, which is basically an unique author-chosen identifier for the book. The book slug is included in the book’s Leanpub URL. For example, the URL for Learning Hammerspoon is https://leanpub.com/learning-hammerspoon, therefore its slug is learning-hammerspoon. The slug can be changed by the author as part of the book configuration, and is used in all the API calls to identify the book for which an operation should be performed. {{% note %}} The tools I describe below assume by default that your book’s git repository name is the same as its Leanpub slug. You can specify it if this is not the case by providing some additional configuration parameters. {{% /note %}}
  • API key: Every Leanpub author gets an /API key/, which is a randomly-generated string of characters which is used as an authentication token. The API key needs to be provided on most Leanpub API calls (some query operations are allowed without a key).
  • Build types: The Leanpub API allows you to trigger several types of build operations on a book:
    • Preview builds all the formats supported by Leanpub (PDF, ePub, Mobi), using the whole book as defined in the Book.txt file.
    • Subset preview builds only the PDF version of a book, from a subset of files defined in the Subset.txt file in your repository.
    • File preview builds only a segment of text you need to provide as part of the API call. I do not use this operation in my workflows.
    • Publish builds and publishes a new version of the book. Publishing means that it becomes the version available for purchase. Optionally, when publishing a new release you can send out an email with release notes to people who have already purchased the book (in any case, the new version of the book also becomes available for them to download).
  • Book writing mode refers to the source from which Leanpub gets the text for your book. I use “Git and Github”, but these techniques should work equally well with BitBucket or any other platforms that can trigger a webhook when your text is updated.

The beginning: triggering and watching builds by hand

{{% warning %}} This is not part of my final workflow and you can safely skip it. It is only a bit of historic perspective for how my workflow evolved. {{% /warning %}}

As an initial step, I wrote some shell scripts to trigger and watch the progress of Leanpub builds by hand. I use the Elvish shell, and my scripts are published as the leanpub Elvish module. These allow you to trigger book builds (only preview and subset builds, no publishing) by hand, and also to watch the progress of any operation. If you use Elvish, you can install the module like this:

use epm
epm:install github.com/zzamboni/elvish-modules
use github.com/zzamboni/elvish-modules/leanpub

Then, whenever you commit changes to your text, you can trigger a build and watch its progress like this:

leanpub:preview # or leanpub:subset
leanpub:watch

You can combine build-and-watch in a single command:

leanpub:preview-and-watch
leanpub:subset-and-watch

{{% note %}}If your current directory name is not the same as your book slug, you can pass the book slug as an argument. For example:

leanpub:preview-and-watch my_book

{{% /note %}}

After a while using these scripts, I thought I would put some work on improving both the aesthetics and the functionality of my automation. The next sections are what I came up with.

Watching build activity with Hammerspoon

The first step is to get rid of the need to run those “watch” scripts, which produce raw JSON output from the Leanpub API, and use nice macOS notifications to track the activity, like these:

images/leanpub-notifs/build-step11.png images/leanpub-notifs/build-step15.png images/leanpub-notifs/build-step30.png

These are produced by a Spoon I wrote called Leanpub. You can install, load and configure it using the SpoonInstall spoon.

{{% tip %}} If you want to learn how this Spoon is implemented, please check out the “Writing your own extensions and Spoons” chapter in /Learning Hammerspoon/. {{% /tip %}}

For example, in my configuration I have the following code to configure the spoon to watch for both of my books:

Install:andUse("Leanpub",
               {
                 config = {
                   -- api_key = "my-api-key",
                   watch_books = {
                     { slug = "learning-hammerspoon" },
                     { slug = "learning-cfengine" }
                   }
                 },
                 start = true
})

Note that you also need to specify your Leanpub API key, which you can get and manage in the Author / Your API Key in Leanpub:

file:images/leanpub-api-key.png

{{% warning %}} You can specify the api_key value in the main declaration as shown (commented) above. However, be careful if you keep your configuration file in GitHub or some other publicly-accessible place. What I do is keep a separate file called init-local.lua which I do not commit to my git repository, and where I set my API key as follows:

-- Leanpub API key
spoon.Leanpub.api_key = "my-api-key"

This file in turn gets loaded into my main config file as follows:

local localfile = hs.configdir .. "/init-local.lua"
if hs.fs.attributes(localfile) then
  dofile(localfile)
end

{{% /warning %}}

Reload your Hammerspoon configuration. Now when you trigger a preview or publish (for example, using the scripts above), you will after a few seconds start seeing the corresponding notifications.

{{% tip %}} Note that the actual behavior and appearance of the notifications produced by Hammerspoon (like those of any other application) depend partly on the settings you have in the macOS Notifications control panel. For example, if you don’t see any notifications, make sure the Hammerspoon notifications are not blocked and that you have not enabled “Do not disturb” mode. If you don’t see any buttons in the notifications, it may indicate that you have the Hammerspoon alert style set to “Banners” instead of “Alerts”. {{% /tip %}}

Triggering builds with a static webhook

The most basic way of automatically triggering builds is by using a webhook to trigger the Leanpub API directly. This is described in your book’s “Getting Started” page, which you can access at https://leanpub.com/YOUR_BOOK/getting_started (replacing YOUR_BOOK with your book’s slug). This works well, but the downside is that the webhook is “hardcoded” so you can only trigger a fixed type of build per webhook (e.g. subset or regular preview). This means that if you want to trigger a different type of build, you need to keep multiple webhooks defined, and activate the one you want by hand:

file:images/github-webhooks.png

{{% tip %}} If you only want to automatically trigger one type of build then you don’t need to do anything else. Continue reading if you are interested in a flexible workflow which allows you to trigger different types of builds based on tags you define on your book’s repository. {{% /tip %}}

Triggering builds via CircleCI

Looking for ways to further automate the preview and publish workflow of my books, I came across CircleCI, a popular CI/CD platform which is easy to use and allows creation of libraries called “orbs” to encapsulate more complex behaviors. I wrote an orb called zzamboni/leanpub for automating interactions with the Leanpub API. With it, you can set up a build/preview/publish workflow which you can trigger directly from git.

Here’s my preferred workflow (you can build others as well using the leanpub orb, see below for ideas):

  • A subset preview is triggered for every commit to the book’s repository. I keep Subset.txt with the same content as Book.txt, so a subset preview gives me a PDF-only build of my whole book (you could also modify Subset.txt before each commit depending on the part of the book you want to preview, but this is outside the scope of this article). This allows me to have a continuous PDF preview of any changes I make to my book.
  • A regular preview is triggered for commits that are tagged with a tag starting with preview. This builds the book in all the formats supported by Leanpub (PDF, epub, mobi). This allows me to check the output in all formats when I’m doing finishing touches before publishing a new version of the book, or when I make major changes.
  • A silent publish is triggered for commits tagged with a tag starting with silent-publish. This builds and publishes the book, but without sending out release notes. I use this for “minor” updates to the published book which I don’t think need to be widely announced (e.g. fixing typos and formatting, etc.)
  • Finally, a publish is triggered for commits tagged with a tag starting with publish. This builds and publishes the book, but also sends out release notes to its readers. The release notes are taken from the description of the tag (if it is an annotated tag) or from the commit message of the tagged commit (if it’s a regular tag).

To implement this workflow, all you have to do is add a file .circleci/config.yml to your repository, containing the following:

version: 2.1

orbs:
  leanpub: zzamboni/leanpub@0.1.1

# This tag-based book building workflow dispatches to the correct job
# depending on tagging
workflows:
  version: 2
  build-book:
    jobs:
      - leanpub/subset-preview:
          filters:
            tags:
              ignore:
                - /^preview.*/
                - /^publish.*/
                - /^silent-publish.*/
      - leanpub/full-preview:
          filters:
            tags:
              only: /^preview.*/
            branches:
              ignore: /.*/
      - leanpub/auto-publish:
          name: leanpub/silent-publish
          auto-release-notes: false
          filters:
            tags:
              only: /^silent-publish.*/
            branches:
              ignore: /.*/
      - leanpub/auto-publish:
          auto-release-notes: true
          filters:
            tags:
              only: /^publish.*/
            branches:
              ignore: /.*/

{{% note %}} The leanpub orb defines the “jobs” to trigger the different types of builds, but the logic regarding tags is defined in the worflows: section of the config file, by selecting or ignoring them according to our previous description. You can choose your own tags, or do the workflow based on branch names, or any other condition you want. {{% /note %}}

{{% warning %}} The default config assumes that your git repository name corresponds to the book slug. If this is not the case, you have to add a book-slug parameter to all the job calls in the config file. See the orb documentation for details. {{% /warning %}}

Once you have committed this file, you can enable CircleCI on it as follows:

  • If you have defined static webhooks in your repository as described before, make sure to disable them.
  • Login at https://circleci.com/ using your GitHub account.
  • In the “Add projects” screen, choose your repository and click “Set Up Project”. Since you have already added the config.yml file, you can skip that part and click on “Start building”.
  • The first build will fail because you have not provided your Leanpub API key yet: file:images/circleci-first-job-fail.png
  • To fix this, you need to define an environment variable called LEANPUB_API_KEY within your CircleCI project. Click on the project name, and then on the settings button at the top-left of the screen. Once there, select the “Environment Variables” section and enter the environment variable: file:images/circleci-add-leanpub-api-key.png
  • Now you can go to the “Workflows” screen and click on “Rerun” for your book’s workflow (alternatively, make a new commit on your git repository). Assuming you have the Leanpub Spoon installed as described before, you should see the notifications for your book’s build within a few seconds. file:images/circleci-successful-job.png

Conclusion

Using the techniques described above has made my book building and publishing much easier. I have been using them for a few weeks, and the latest release of /Learning Hammerspoon/ was published using this workflow already. I hope you find it useful as well! Please let me know in the comments if you have any questions or feedback.

Hosting a Ghost Blog in GitHub - the easier way

When I was planning the reboot of my website, I seriously considered using Ghost. It has a very nice UI, beautiful and usable theme out of the box, and a very active community. Eventually I decided to use Hugo, but in the process discovered that it is possible to host a statically-generated Ghost website using GitHub Pages.

Overview

The general approach, described in multiple articles I found, is the following:

  1. Install and run Ghost locally
  2. Edit/create your content on your local install
  3. Create a static copy of your Ghost site by scraping it off the local install.
  4. Push the static website to GitHub Pages

So far, so good. It makes sense. But all those articles share one thing: they suggest using a tool called buster which, as far as I can tell, it’s a web-scraping tool, specialized for Ghost. However, it has a number limitations–for example, it does not slurp Ghost static pages, and it hasn’t been updated in a very long time (there’s a fork with somewhat more recent activity).

I found the use of buster puzzling, since there is a perfectly mature, functional and complete tool for scraping off a copy of a website: good old trusty wget. It is included (or easily available) in most Unix/Linux distributions, it is extremely powerful, and has features that make it really easy to create a local, working copy of a website (including proper translation of URLs). I used it to create the static archive of my old blog, BrT, when I decided to retire its WordPress backend years ago.

Another thing I found is that most instructions suggest storing only the generated website in your GitHub repository. I prefer keeping the source files and the generated website together. GitHub pages allows serving the website from different sources, including the repo’s gh-pages branch, its master branch, or the /docs directory in the master branch. Personally, I prefer using the /docs directory since it allows me to keep both the source and the generated website in the same place, without any branch fiddling.

So, without further ado, here are the detailed instructions. I ran these on my Mac, but most of them should work equally well on Linux or any other Unix-like system.

Install Ghost

  1. Download Ghost (version 1.7.1 as of this writing):
    cd ~/tmp # or some other suitable place
    wget https://github.com/TryGhost/Ghost/releases/download/1.7.1/Ghost-1.7.1.zip
        
  2. Unpack it in a suitable directory, initialize it as a GitHub repository and commit the Ghost plain install (to have a baseline with the fresh install):
    mkdir test-ghost-blog
    cd test-ghost-blog
    unzip ../Ghost-1.7.1.zip
    git init .
    git add .
    git commit -m 'Initial commit'
        
  3. Install the necessary Node modules, update the git repository:
    npm install
    git add .
    git commit -m 'Installed Node dependencies'
        
  4. Install knex-migrator, needed for the DB initialization:
    npm install -g knex-migrator
        
  5. Initialize the database and start Ghost (knex-migrator may give a “Module version mismatch” message, but it seems to work OK anyway):
    knex-migrator
    npm start
        
  6. Your blog is running! You can visit it at http://localhost:2368/: file:images/ghost-initial-screen.png
  7. Go to http://localhost:2368/ghost, create your user and set up your blog info: file:images/ghost-setup-blog.png

    {{% note %}}You may want to use an email address you don’t mind being public. See ”Security Considerations” below.{{% /note %}}

  8. You can now start creating content and configuring the local Ghost instance. file:images/ghost-admin-screen.png
  9. When you have things the way you like them, you can commit the changes to the git repository:
    git add .
    git commit -m 'Finished local Ghost setup'
        

Export website

Now that you have your blog set up locally, we need to generate a static copy that can be published to GitHub. For this we will use the wget command. I gathered the correct options from this blog post by Ilya a few years ago, although it’s not too hard to deduct them from the wget man page.

  1. We will publish the blog from the docs directory of our repository, so that’s where we need to store the static copy:
    wget -r -nH -P docs -E -T 2 -np -k http://localhost:2368/
        

    This command will crawl the entire site and create a static copy of it under the docs directory. You can open the file docs/index.html in your web browser to verify.

  2. Add the generated pages to the git repository:
    git add docs
    git commit -m 'Initial commit of static web site'
        

Push to GitHub

We can finally create our GitHub repo and push the contents to it.

  1. Create the repository. I’m using here the =hub= command, but of course you can also do it by hand in the GitHub website (in this case you need to add the git remote by hand as well):
    hub create
        
  2. Push the local repository to GitHub (this includes both the Ghost source and the generated website under docs):
    git push -u origin master
        

Publish!

Now all we need to do is enable GitHub Pages on our repository, so that the contents under docs gets published.

  1. Go to your repository’s “Settings” screen: file:images/ghost-repo-settings-screen.png
  2. Scroll down to the “GitHub Pages” section, choose the “master branch /docs folder” option and click the “Save” button: file:images/ghost-repo-github-pages-setting.png

We are done! After a few minutes (usually takes 2-5 minutes for the contents to be published the first time, afterwards updates are nearly instantaneous), you will find your new website’s content under http://<github-username>.github.io/<github-repo-name>. In our example, the URL is https://zzamboni.github.io/test-ghost-blog/:

file:images/ghost-published-blog.png

The update workflow

After the initial setup, you need to follow these steps when you want to update your website:

  1. Start Ghost inside your GitHub repository:
    npm start
        
  2. Connect to http://localhost:2368/ and update your contents. You can also change the blog settings, themes, etc.
  3. Re-crawl the site to generate the local copy:
    wget -r -nH -P docs -E -T 2 -np -k http://localhost:2368/
        
  4. Update and push the whole git repository:
    git add .
    git commit -m 'Website update'
        

Steps 3 and 4 can be easily automated. I keep the following =update_website.sh= script in the repository:

#!/bin/bash
OUTDIR=docs
LOCAL_GHOST="http://localhost:2368/"
wget -r -nH -P $OUTDIR -E -T 2 -np -k $LOCAL_GHOST && \
git add . && \
git ci -m 'Update website' &&  \
git push

Then you can just run this script from within your repository after making any changes:

./update_website.sh

Variations

The method described above is my favorite because it allows me to keep the source data and generated pages in the same repository. However, there are other variations that you might want to use:

  • If your repository is named <username>.github.io, you cannot configure GitHub Pages to serve content from the /docs directory, it is automatically served from the root directory of the master branch. In this case you need to store only the generated pages in the repository (you could also reverse the setup: have the generated website in the root directory, and the local Ghost install under the /source directory).
  • You can choose to serve contents from the gh-pages branch instead of the /docs directory. This allows you to keep the source and output still in the same repository. You will need to switch from one branch to the other between updating the contents and generating the static web site (you may need to keep both branches in different directories, so that your local Ghost install can still access its database in the master branch while you fetch it to generate the static website).

You can read more about the different ways to serve GitHub Pages content in the GitHub documentation.

You can use the same method to host your static content somewhere other than GitHub pages.

Security considerations

One of the most-touted benefits of a static website (also known, I discovered recently, as the JAMstack) is security – without server-side active components, it’s much harder for a website to be compromised.

Sharp-eyed readers may have noticed that with the setup described above, Ghost’s entire database file gets checked into your repository. This file contains not only your published blog posts, but also your user definitions (including hashed versions of the user passwords) and draft posts.

The local Ghost install uses a SQLite database, stored at content/data/ghost-dev.db, which you can query using the sqlite3 command. For example, to see the user definitions:

sqlite3 content/data/ghost-dev.db 'select * from users'

While this seems scandalous, keep in mind that the active Ghost installation is only running locally on your machine, and is not accessible to anyone from the outside, even when it is running (the server is only bound to localhost). Still, you may want to keep in mind:

  • Your name and email address are accessible. Your name is visible in any posts you write, but you may want to set your email address to one you don’t mind being public.
  • Don’t use a password that you also use in any other place (this is a good security recommendation in general). The password is hashed, but this prevents the password from being useful even if someone manages to figure it out.
  • If you share your machine with others, keep in mind that any other local users will be able to access your local Ghost install as long as it’s running. If you don’t trust those users, make sure you set a good password and shut it down when you are not using it.
  • To prevent the problem altogether, add content/data/ghost-dev.db to your .gitignore file so it does not get checked into the repository. Make sure you make a backup of it separately so you can recover your blog in case of local data loss.

Conclusion

I followed the steps described above to create a test Ghost install, which you can access at https://zzamboni.github.io/test-ghost-blog/, and its corresponding GitHub repository at https://github.com/zzamboni/test-ghost-blog/. You can also find this article published there.

I hope you’ve found it useful. I’d love to hear your comments!

The Big Website Reboot

Welcome to the new zzamboni.org.

Over the years, my website has seen its fair share of transformations, change and breakage. For a few years now, it had stagnated - me with not having much time to update, and its accumulated technological cruft piling high enough to keep me from even touching it, lest I break something.

So, the time has come to do a reboot. With this post, I’m launching the new and reinvented zzamboni.org. Through previous incarnations I had always tried to preserve backwards compatibility in my URLs so people could still find my old stuff, but this has become too big a burden. I may add some things under their old location, and I may add over time some tools to make it easier to find the old stuff, but in general this won’t be a concern and I will not let it stop me from adding new content.

Over the years, the underlying technologies have changed (Wordpress, Posterous, Jekyll, Octopress, Enwrite). As a tech guy, I always enjoy playing with new toys. For now I have settled on using Hugo, a great static-website generator, using its Ananke theme. Hugo and Ananke are powerful and flexible to satisfy my needs, both current and future, as far as I can foresee them. The website continues to be hosted through the fantastic Github Pages. My website was hosted on Github Pages for many years, but it is now served by Netlify.

So this is it. For now the site is mostly empty, but new content will be appearing shortly, both new and ported from my old website. I have started adding posts from my previous blogs. Thanks to Hugo’s aliases feature, most of them should be accessible still through their old URLs.

Please take a look around, and let me know if you find anything broken.

Footnotes