Skip to content
CI for documentation. Testing if documentation is runable.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples adding `user: <user>` for steps (#5) Jun 12, 2019
img Use -q example May 27, 2019
lib if already root, don't need to sudo Jun 14, 2019
.gitignore adding `report` command May 24, 2019
README.md Grammar Jun 2, 2019
index.js support custom css .passing .failing for custom renderer Jun 14, 2019
package-lock.json if already root, don't need to sudo Jun 14, 2019
package.json Add git stanza in setup. Jun 9, 2019

README.md

docable

CI for documentation: Testing if documentation is runable.

Suppose you wanted to test the following tutorial (inspired from benmoral.)

Example Tutorial

Let’s create a small server using PHP. I can see your eyebrows rising, but it works surprisingly well. We’ll listen to UDP port 10000, and return any message received with a ROT13 transformation:

<?php
$sock = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_bind($sock, '0.0.0.0', 10000);
for (;;) {
    socket_recvfrom($sock, $message, 1024, 0, $ip, $port);
    $reply = str_rot13($message);
    socket_sendto($sock, $reply, strlen($reply), 0, $ip, $port);
}

Let’s start it:

$ php server.php

And test it in another terminal:

client$ echo 'Hello, world!' | nc -q 1 -u 127.0.0.1 10000
Uryyb, jbeyq!

Cool, it works. Now we want this script to run at all times, be restarted in case of a failure (unexpected exit), and even survive server restarts. That’s where systemd comes into play.

...


How would we test these instructions actually work? After you updated them? On different platforms and versions of php (5 vs 7) or tools (netcat openbsd vs traditional)? After a year has passed? That's what docable helps do for you.

Testing with docable

To test this example tutorial, we can run the following:

docable report examples/unix-service/steps.yml

docable uses a stepfile, such as the following.

setup:
#  local: {}  
  slim:
    image: ubuntu16.04-php

unix-service.md: 
  steps:
    - selectAsFile: ROT13 transformation => server.php
    - selectAndServe: "start it:"
    - selectAndExpect: "another terminal:"

The stepfile using a "select-assert" pattern for testing documentation. That is, we first find an instruction step in the tutorial using a lightweight selector, then, we transform the step into an action we can perform within a "headless infrastructure" (such as a vm), and finally we assert it's behavior is correct.

The result looks something like this:

example report

Turns out that our tutorial breaks if using a different version of netcat, which does not support the -q 1 option. Oops!

After fixing the issue by changing to using "-w 1" (terminated after timeout instead of time since no STDIN input). Running docable test, we verify have fixed the tutorial:

example execution

Installing docable

$ git clone https://github.com/ottomatica/docable
$ npm install
$ npm link

Using slim provider requires installing slim. You can also switch the stepfile to use a local provider under setup:.

You can’t perform that action at this time.