article

article is a free command-line tool for building and maintaining a self-hosted publishing archive.

It started with a practical problem: take an existing Substack publication, preserve the articles and images locally, and build a normal static website that can be served from your own machine. It also supports articles written directly in Markdown, so Substack can remain one publishing destination without being the canonical copy of the work.

The archive is the important part. Importers feed it. The static site is generated from it.

Substack export ----\
                     >---- canonical Markdown archive ---- static HTML ---- nginx
Markdown article ---/

The canonical archive uses plain Markdown, human-readable front matter, and local images. There is no database and no proprietary storage format.

Commands

article init
article import markdown article.md
article import substack substack-export.zip
article sync substack-export.zip
article build --clean
article status
article verify
article help

sync is a convenience alias for import substack.

Build requirements

article is a multi-file C++23 application built with a conventional Makefile.

The Makefile deliberately defaults to clang++.

Debian / Ubuntu

Install the required development packages:

sudo apt update
sudo apt install clang make pkg-config libarchive-dev libcurl4-openssl-dev libxml2-dev libheif-dev libjpeg-dev libxxhash-dev

The dependencies have narrow jobs:

• libarchive reads Substack export ZIP files.
• libxml2 parses exported HTML.
• libcurl downloads article images and YouTube metadata/thumbnails.
• libheif + libjpeg convert HEIC/HEIF images to browser-safe JPEG.
• xxHash supplies XXH3_64 fingerprints for incremental Substack syncs.

Build

Clone the repository and enter the source directory:

git clone https://github.com/vinthewrench/article.git
cd article

Build the development version:

make

The default compile configuration uses:

clang++ -std=c++23 -Wall -Wextra -Wpedantic -g -O0

For an optimized release build:

make release

Run the smoke test:

make check

Install the command system-wide:

sudo make install

After installation:

article help

instead of:

./article help

Publication projects

The article executable is separate from the publication it manages.

A publication directory contains its configuration, canonical archive, templates, CSS, and optionally the Docker/nginx files used to serve the generated site.

For example:

my-publication/
├── article.conf
├── archive/
├── templates/
├── static/
├── docker-compose.yml
└── nginx/

The installed article executable can manage multiple independent publication directories.

Initialize a publication

Create an empty directory for the publication:

mkdir my-publication
cd my-publication

Then initialize it:

article init

This creates the archive directories and starter site files without overwriting files that already exist.

It also creates article.conf if one does not already exist.

Example:

site_name=My Articles
index_title=My Articles
author=
base_url=

archive_dir=archive
template_dir=templates
static_dir=static

publication_domains=

article.conf is deliberately simple key=value text.

For a publication that has used multiple domains, they can be listed as a comma-separated value:

publication_domains=example.substack.com,www.example.com,example.com

These domains help the importer recognize and rewrite links between articles in the publication.

Nothing about the executable is tied to one publication name or domain.

Project root discovery

An article.conf file identifies the root of an Article publication.

Commands can therefore be run from the project directory itself:

cd my-publication
article status

or from a directory underneath it:

cd my-publication/archive/content/articles
article status

article searches upward for article.conf and uses that directory as the publication root.

If no Article project can be found, the command stops rather than creating an archive in the wrong directory.

Get your Substack export

From your Substack Dashboard, go to Settings, then scroll all the way down to Import / Export.

Under Export your data, select New export. Substack will prepare the export and email you when it is ready. Go back to Import / Export and download it.

The downloaded ZIP normally has a generated name. You may want to rename it to something easier to recognize, for example:

substack-export-2026-08-10.zip

Do not unzip it. article reads the Substack export ZIP directly.

Import an existing Substack publication

Place the Substack export ZIP in the publication directory.

For the first import, run:

article import substack substack-export.zip
article verify
article build --clean

For later exports, use the incremental sync form:

article sync substack-export.zip
article verify
article build --clean

For a quick test with only the newest published articles:

article sync substack-export.zip --limit 10

The importer reads posts.csv, finds the corresponding exported HTML, converts each article body to canonical Markdown, downloads images locally, preserves code blocks, callouts, pull quotes, and YouTube references, and stores the result in the archive.

Published articles are stored under:

archive/content/articles/

Drafts are stored under:

archive/content/drafts/

Images are stored under:

archive/assets/images/

The importer accepts Substack exports where posts.csv and posts/ are located directly at the ZIP root or beneath a wrapper directory.

Incremental Substack sync

Substack imports store an XXH3_64 fingerprint and importer version in the article front matter.

On later syncs, unchanged Substack articles are skipped.

For example:

unchanged: some-article
unchanged: another-article

Articles that have changed are imported again.

When the Substack importer itself changes in a way that requires articles to be reprocessed, its importer version can be incremented in SubstackImporter.cpp.

Markdown remains authoritative

An important design rule is that locally authored Markdown wins.

If an article was imported from a local Markdown file and later appears in a Substack export with the same slug, a Substack sync does not overwrite the local article.

That supports this workflow:

write Markdown first
       |
       +--> article import markdown
       |
       +--> copy/paste to Substack

Substack can remain a publishing destination without becoming the master copy of the article.

A later Substack sync will not stomp on the local Markdown.

Import a Markdown article

The normal command is:

article import markdown my-article.md

If the file has no front matter, the importer creates sensible defaults:

• the first # H1 becomes the article title
• the H1 is removed from the body so the generated page does not repeat it
• the slug is generated from the title
• the date defaults to today
• the source is marked as markdown

For example:

# Building My Weather Station

This is the first paragraph of the article.

## Hardware

The hardware consists of...

Import it with:

article import markdown weather-station.md

The resulting article is stored as canonical Markdown in the publication archive.

Replace an existing Markdown article

To overwrite an existing local article with the same slug:

article import markdown my-article.md --replace

Without --replace, the importer protects an existing article from accidental replacement.

Import a draft

To import an article as a draft:

article import markdown my-article.md --draft

Drafts remain in the canonical archive but are not included with the published articles in the generated site.

Local images in Markdown

A Markdown article can use ordinary relative image paths:

![Valve controller](images/valve-controller.jpg)

When the article is imported, article copies the image into the canonical archive and rewrites the Markdown to use the archived image.

For example:

![Valve controller](/assets/images/my-article-001.jpg)

This means the article does not depend on the original working directory after it has been imported.

Remote HTTP/HTTPS images are downloaded as well.

HEIC and HEIF files are converted to JPEG so they can be served by an ordinary web server and displayed by browsers.

YouTube

Substack exports YouTube embeds as special HTML blocks.

article preserves those as a small readable directive in canonical Markdown:

:::
id: "gbhPg_5XXlw"
url: "https://www.youtube.com/watch?v=gbhPg_5XXlw"
title: "Example video"
author: "example-channel"
image: "/assets/images/example-youtube-001.jpg"
:::

The generated static site does not embed or copy the YouTube video.

Instead, it renders a static preview card containing:

• the local thumbnail
• YouTube title
• author/channel
• a play button
• a link to YouTube

Title and author information come from YouTube's oEmbed endpoint, which does not require an API key.

Missing or dead YouTube thumbnails

Old, private, or deleted videos may no longer have a thumbnail available.

When this happens, the importer prints an actionable warning such as:

warning: YouTube preview image unavailable
  post: Example Article
  video: https://www.youtube.com/watch?v=xxxxxxxxxxx
  drop image here: archive/assets/images/example-article-youtube-001.jpg
  filename: example-article-youtube-001.jpg

A replacement image can be placed at that exact location.

A manually supplied image takes precedence over a YouTube thumbnail.

Status

To see a quick summary of the publication:

article status

Example:

Articles:  132
Published: 128
Drafts:    4
Words:     256562

Verify the archive

Run:

article verify

A clean archive reports:

Verified 132 articles, 0 errors

Verification is useful after importing or syncing a publication and before generating the public site.

Build the website

Generate the static website with:

article build --clean

The site is written under:

archive/site/

--clean removes the previous generated site before rebuilding it.

The builder deliberately prints each article as it is generated so progress is visible during a full rebuild.

Do not hand-edit files under:

archive/site/

They are generated output.

The source files intended for customization are:

templates/article.html
templates/index.html
static/article.css
article.conf

Serve with nginx and Docker

The repository includes:

docker-compose.yml
nginx/default.conf

Bring the web server up with:

docker compose up -d

The default mapping is:

http://localhost:9010/

The generated site is mounted read-only into nginx:

./archive/site -> /usr/share/nginx/html

After rebuilding the site:

article build --clean

nginx immediately sees the new files. The container does not need to be rebuilt.

To stop the server:

docker compose down

Archive layout

A publication archive looks like this:

archive/
├── content/
│   ├── articles/
│   └── drafts/
├── assets/
│   └── images/
├── knowledge/
│   ├── articles/
│   ├── chunks/
│   ├── articles.json
│   └── index.json
└── site/

The canonical data is:

archive/content/
archive/assets/

These contain the material that should be preserved.

The following are derivative and can be rebuilt:

archive/knowledge/
archive/site/

Canonical article format

Articles are stored as Markdown rather than in a database.

Each article contains human-readable front matter followed by the Markdown body.

The archive therefore remains useful even without the article executable. It can be inspected, searched, backed up, copied, or processed with ordinary text tools.

That is intentional.

Code layout

The application is deliberately divided into small modules rather than being implemented as one large source file.

src/
├── main.cpp
├── Application.cpp/.hpp
├── CommandLine.cpp/.hpp
├── Config.cpp/.hpp
├── Article.hpp
├── Archive.cpp/.hpp
├── FrontMatter.cpp/.hpp
├── MarkdownImporter.cpp/.hpp
├── SubstackImporter.cpp/.hpp
├── HtmlToMarkdown.cpp/.hpp
├── MarkdownRenderer.cpp/.hpp
├── SiteBuilder.cpp/.hpp
├── ImageManager.cpp/.hpp
├── YouTube.cpp/.hpp
├── HttpClient.cpp/.hpp
├── ZipArchive.cpp/.hpp
├── Csv.cpp/.hpp
├── Defaults.cpp/.hpp
└── Util.cpp/.hpp

The basic layering is:

CommandLine
    |
Application
    |
    +--> MarkdownImporter --\
    |                       |
    +--> SubstackImporter ---+--> Archive --> SiteBuilder
                             |
                             +--> ImageManager / YouTube

Each module has one clear job.

Comments in the source explain non-obvious behavior and, where useful, the reason for the implementation choice.

Adding another importer

The canonical Article structure is independent of Substack.

An importer only needs to produce an Article and hand it to the archive.

That makes other importers natural future extensions:

article import wordpress export.xml
article import ghost export.json

Adding another source does not require changing the canonical archive format or static site generator.

Backups

Because the canonical publication consists primarily of Markdown and local image files, normal filesystem backup tools work well.

The important directories are:

archive/content/
archive/assets/

The configuration and customized presentation files should also be backed up:

article.conf
templates/
static/

The generated archive/site/ directory does not need to be backed up because it can be rebuilt.