Modernising UNIX manpages

[marcastel]

Objectives

Primary objectives:

• Migrate the UNIX man(7) page native format to Markdown while adopting the philosophy of mdoc(7).
  • Possibly constrain the Markdown layout through a (YAML) schema to enforce structure.
  • By Markdown, we mean Commonmark (and access to the AST)
• Extend the command line interface to:
  • Allow opening local manpages directly in a browser
  • Opening of manpages from an online repository
  • Provide tagging, bookmarking, and reading lists (by analogy to bookmarks in an Internet browser)
  • Automatically produce cheatsheets (i.e. a summarised manpage: SYNOPSIS + OPTIONS).
• Provide backward compatibility to support or automatically upgrade all existing manpages
• Provide the necessary tools to automatically index man pages
• Provide an alternative for UNIX man(1) utility with extended search and navigation capabilities
  • Rather than creating a new utility, we would like to see this as a mandoc(1) enhancement.
  • Implications on apropos(1), makewhatis(8), whatis(1), man.conf(5), and related, must be addressed.

Other objectives:

• Allow tabular data in manpages (Glow does a pretty good job already at displaying Markdown tables)
• Allow beautified hyperlinks in terminals (i.e. show clickable label on screen -- achieved through ANSI escape codes)
• Allow image in manpages (Some terminal emulators provide ANSI sequences for this, e.g. iTerm2)
• Allow responsive manpage layouts whether on terminal, in browser, or in print, from a single source.

Long term objectives:

• Provide cross-language utilities to create same-quality manpages
  -- one can take inspiration on how documentation has become central to modern RESTful API design tools.
• Provide both system and online access to the documentation
  -- this could be through a unique version-managed repository.
• Allow user feedback, both from the command line or online
  -- provide somethings in the likes of PHP's online documentation.
• And provide all modern tools for quality assurance and lifecycle management... online.

Background

Manpages are a fundamental component of UNIX. They would not have existed without the Troff typesetting tool -- mostly known as Groff (or GNU Troff) in most UNIX and Linux distributions.

Manpages are, IMHO, a fundamental reason for the success of UNIX as an operating system... hands on access to information when and where you need it. They are simple, structured, cross-referenced, and up to date. Further they address the requirements of all users, be they system administrators, software developers or simply end users.

However the UNIX manpage ecosystem could do with some modernisation -- read this article by the author of mdoc(7).

The need for Unix manuals to render cleanly to multiple output media favours structural rather than presentational markup, however the classical man(7) macros remain almost exclusively presentational. mdoc(7) provides a semantically superior alternative, but the use of man(7) is deeply rooted in Unix.

Future work on manpages will entail improving the semantic clarity of the man(7) macros, decoupling them as much as possible from low-level presentational requests. The aim will be to ease conversion of manpages to markup languages that do not rely on groff for display and printing, e.g. XML, while preserving the full presentational richness of manpages processed with groff.

Concurrent with work on man(7), mdoc(7) will be actively supported and its use promoted.

To summarize, the goal of this two-pronged strategy is to foster manpage markup that:

• renders cleanly to the terminal
• respects presentational markup when output to PostScript
• allows semantic analysis
• is portable

Source: GNU Groff mission statement

Roadmap

A proof of concept (POC) can be built using Pandoc (Haskell) for the conversions, and Glow (Go) as the terminal-capable Markdown renderer. Plain KornShell programming is sufficient here to demonstrate the expected output and to model the command line interface of the target utility.

Ultimately this should be a C language utility (Go optionally) based on ncurses(3), CommonMark, and libYAML.

Ideally this shall not be yet another UNIX tool but will be blended into an existing POSIX-compliant tool.

Open source projects of interest

• Glow to render markup on the terminal
• man-to-md is a Perl converted from nroff to markdown which is interesting to rapidly draft a POC.