Fig was born out of frustration with heavyweight frameworks like Ansible that involve a huge dependency footprint and present a mere illusion of "declarative" configuration around something that is an inherently imperative/procedural process.
At the time of writing, a git clone --recursive https://github.com/ansible/ansible.git creates a 224 megabyte directory containing 4,697 files. This is in addition to the Python language runtime itself and the additional modules and supporting infrastructure that may be involved, such as pip and virtualenv.
In comparison, Fig's source is roughly 468 KB on disk, and consists of about 6,600 lines of code across 89 files.
Ansible configuration exists in YAML files which look static but are deceptively dynamic: values in these files get parsed as static values, Python expressions, or Jinja2 templates that may themselves contain Jinja2 filters and Python snippets. This mish-mash of conceptual models leads to awkward syntax that requires careful quoting and a constant mental effort to separate each of the three layers (static YAML, Jinja2 interpolation/filtering, and Python evaluation).
For example, consider this configuration that moves items into a ~/.backups directory:
- name: dotfiles | backup originals
command: mv ~/{{ item.0 }} ~/.backups/
creates=~/.backups/{{ item.0 }}
removes=~/{{ item.0 }}
loop: '{{ (dotfile_files + dotfile_templates) | zip(original_check.results) | list }}'
when: item.1.stat.exists and not item.1.stat.islnk
loop_control:
label: '{{item.0}}'Note:
- The context-dependent need to quote Jinja2 syntax interpolation (
{{ ... }}) in some places but not others, due to conflict with YAML syntax rules. - Undifferentiated mixing of Python evaluation (eg. list concatenation with
+) and Jinja2 filtering/transformtion (eg.| zipand| list) in a single value. - Awkward encoding of imperative programming patterns using YAML keys (eg.
loopandloop_controlto describe a loop;whento describe a conditional). - Context-specific embedding of Python expressions (eg. raw Python code being passed as a string in the
whenproperty, but elsewhere being interpolated in Jinja2 interpolation). - Implicit/magical variable naming conventions (eg. use of
loopimplies the existence of anitemvariable). - No obvious scoping rules eg. variables like
dotfile_filesanddotfile_templatesare magically available with no obvious source (they are defined in another file); others likeoriginal_checkare also magically available, but defined in a prior task).
Given all this convolution, Fig proposes that it is simpler to just embody this imperative, procedural work in an actual programming language. By using TypeScript, we can obtain a comparable (or superior) level of static verification to what we would get with Ansible's YAML, as well as enjoying the benefits that come with using a "real" programming language in terms of tooling (eg. editor autocompletion, code formatting etc). By providing a DSL (Domain-specific language) (eg. implemented in Fig's DSL), we can conserve and arguably improve on the ergonomic properties of writing Ansible's YAML. For example, the common task of moving items into a backup directory before installing their replacements can be extracted into an "operation", and the call-site becomes:
task('move originals to ~/.backups', async () => {
const files = [...variable.paths('files'), ...variable.paths('templates')];
for (const file of files) {
const src = file.strip('.erb');
await backup({src});
}
});An even simpler example can be found in the "vim" aspect (simpler, because it doesn't deal with any templates that have an ".erb" extension):
task('move originals to ~/.backups', async () => {
const files = variable.paths('files');
for (const src of files) {
await backup({src});
}
});To understand this, you need only understand TypeScript (or really, just JavaScript) syntax, unlike the Ansible example, which requires you to know YAML, Jinja2, Python, and the internal logic of how they are all layered together.
All of the above is not to say that Ansible is a bad tool — I use it in other contexts quite productively, to manage remote hosts, for instance — but that it might not be the best fit for dotfile management and configuration of a single local machine.
Overall structure remains similar to Ansible, but I made some changes to better reflect the use case here:
- Configuration is divided into "aspects" that contain:
- A TypeScript
index.tsthat defines tasks to be executed. - An
aspect.jsonoraspect.tsfile that contains metadata, such as adescriptionand (optional)variables. - An (optional)
filesdirectory containing resources to be copied or otherwise manipulated. - An (optional)
templatesdirectory containing templates to be dynamically generated (and then copied, installed etc). - An (optional)
supportdirectory to contain any other useful resources (eg. helper scripts etc).
- A TypeScript
- A top-level
project.ts(orproject.json) declares:- Supported platforms (eg. "darwin", "linux") plus their related aspects and variables.
- Profiles (eg. "personal" and "work") along with their associated variables, and patterns for determining which profile should apply on a given machine.
- Default variables that apply in the absence of more specific settings (see "Variables" for more details).
- The Fig source itself lives in the
figdirectory. - All interaction occurs via the top-level
installscript, which invokes Fig via a set of helper scripts in thebindirectory.
Again, things are broadly modeled on Ansible, but simplified to reflect that fact that while Ansible is made to orchestrate multiple (likely remote) hosts, Fig is for configuring one local machine at a time:
| Ansible | Fig |
|---|---|
| Hosts: Machines to be configured (possibly remote) | n/a (always the current, local machine) |
| Groups: Collections of hosts, so you can conveniently target multiple hosts without having to address each one individually | Profiles: An abstract category indicating the kind of a host (eg. "work" or "personal") |
| Inventory: A list of hosts (or groups of hosts) to be managed | n/a ("project.ts" or "project.json" file contains map from hostname to profile to be applied) |
| Roles: Capabilities that a host can have (eg. webserver, file-server etc) | Aspects: Logical groups of functionality to be configured (eg. dotfiles, terminfo etc) |
| Tasks: Operations to perform (eg. installing a package, writing a file) | Tasks: Same as Ansible. |
| Handlers: Side effects that run (eg. restarting a service) if a task made a change to the system (eg. writing a config file) | Handlers: Same as Ansible. |
| Plays: A mapping between hosts (or groups) and the tasks to be performed on them | n/a (it's just a file containing tasks) |
| Playbooks: Lists of plays | n/a ("project.ts" or "project.json" file contains a map from platform to the aspects that should be set up on a given platform) |
| Tags: Keywords that can be applied to tasks and roles, useful for selecting them to be run | n/a (not needed) |
| Facts: (Inferred) attributes of hosts | Attributes: Same as Ansible, but with a better name |
| Vars: (Declared) values that can be assigned to groups, hosts or roles | Variables: Same as Ansible, but belong to profiles and aspects |
| Modules: Units of code that implement operations (ie. these are what tasks use to actually do the work) | Operations: Code for performing operations |
| Templates: Jinja templates with embedded Python and "filters" | Templates: ERB templates with embedded JavaScript |
| Files: Raw files that can be copied using modules | Files: Raw files that can be copied using operations |
| Syntax: YAML with interpolated Jinja syntax containing Python and variables | Syntax: TypeScript and (plain) JSON |
Fig implements a simplified, tiny subset of Ansible's nearly 3,400 "modules" (in Fig, called "operations") necessary for configuring a single system in a basic way. At the time of writing, that means:
| Fig operation | Ansible module |
|---|---|
| backup | n/a |
| command | command |
| cron | cron |
| defaults | osx_defaults |
| fetch | get_url |
| file | file |
| line | lineinfile |
| template | template |
Because Fig tasks are defined using TypeScript, you can define and use variables just like you would in any TypeScript program. As built-in language features, these follow the lexical scoping rules that you would expect to apply to const and let.
Additionally, there is a higher-level representation of variables that can be accessed by the DSL. This enables variables to be defined at various levels of abstraction (for example, as a project-wide default, or an aspect-specific one), with a well-defined precedence that ensures that the "most local" definition wins. Variable access is always explict, either by a call to to the variable() API (eg. example from the "dotfiles" aspect), or by accessing the variables object from inside templates (eg. example from the "launchd" aspect).
The levels are, from lowest to highest precedence:
| Kind | Description |
|---|---|
| Base | Currently just one of these, profile, which is set to either 'personal', 'work' or null |
| Attributes | Derived from system using the Attributes class (eg. home, platform, username) |
| Defaults | Defined in the top-level project.ts here |
| Profile | Defined in the top-level project.ts for each profile (eg. personal, work) |
| Platform | Defined in the top-level project.ts for each platform (eg. darwin, linux) |
| Dynamic | Calculated in variables.ts at the top-level (eg. identity) |
| Aspect (static) | Defined in aspects/*/aspect.json or aspects/*/aspect.ts files (eg. aspects/dotfiles/aspect.json) |
| Aspect (derived) | Derived using the variables() API (eg. dotfiles variables() example) |
Most of these are static, arising from JSON files, but two of the later levels ("Dynamic" and "Aspect (derived)") provide the means to dynamically set or derive the value of a variable at runtime.
Because Fig is written in TypeScript, and tasks are defined in TypeScript files, extending the Fig DSL is a simple matter of writing and using new functions.
For example, my dotfiles repo defines a helpers.ts file containing functions like is() and when() for expressing common conditional logic. These can be used like this to set up a cron job only for me ('wincent') and only on my personal machines ('personal'):
task('create ~/Library/Cron', when('wincent', 'personal'), async () => {
await file({
path: '~/Library/Cron',
state: 'directory',
});
});or like this, to check if we're running on Linux:
if (is('linux')) {
await file({path: '~/share', state: 'directory'});
await file({path: '~/share/terminfo', state: 'directory'});
} else {
await file({path: '~/.terminfo', state: 'directory'});
}- 2009: Originally, the repo was just a collection of files with no installation script.
- 2011-2015: I created a
bootstrap.rbscript (final version here) for performing set-up. - 2015: I briefly experimented with using a
Makefile(final version here). - 2015-2020: I switched to Ansible (completing the transition in cd98e9aaab).
- 2020-present: I started feeling misgivings about the size of the dependency graph and in truth I was probably using less than 1% of Ansible's functionality, so moved to the current set-up, which is described in this file. A representative "final state" of the Ansible-powered implementation can be seen here.
The goal was to replace Ansible with some handmade scripts using the smallest dependency graph possible. I originally tried out Deno because that would enable me to use TypeScript with no dependencies outside of Deno itself, however I gave up on that when I saw that editor integration was still very nascent. So I went with the following:
- n (as a submodule) and some hand-rolled Bash scripts to replace virtualenv and friends (Python, pip).
- Yarn (vendored) to install TypeScript.
Beyond that, there are no runtime dependencies outside of the Node.js standard library, although I do use dprint to format code. Ansible itself is replaced by a set of self-contained TypeScript scripts. Instead of YAML configuration files containing "declarative" configuration peppered with Jinja template snippets containing Python and filters, we just use TypeScript for everything. Instead of Jinja template files, we use ERB/JSP-like templates that use embedded JavaScript when necessary.
Because I needed a name to refer to this "set of scripts", it's called Fig (a play on "Config").
The "support/" directory contains some plain JavaScript helpers that generate TypeScript type definitions and "assert" functions based on some small JSONSchema schemas, to facilitate working with JSON files (such as the "project.json" and various "aspect.json" files) in a type-safe manner.