Update in documentation (#254)

* Added section about stub anatomy

* Added first version of MkDocs documentation for use in frontend site

* Updated sync doc script to strip table of contents

* Also copy changelog

* Added readthedocs theme

* Added favicon to docs

* Removed old docs

* Added build step for MkDocs

* Updated .gitignore

* Test

* Fix in doc build script

* Move docs to correct place in UI project

* Fixed bug in doc sync script

* Fixes in build scripts

* Re-enable tests

* Added api docs to menu

* Small doc fixes

* Added logo in docs

* Added meta description

* Updated CHANGELOG

* Pack docs as artifact

* Small change in .gitlab-ci.yml

* Fixed build scripts

Co-authored-by: Duco <git@ducode.org>

.gitlab-ci.yml

@@ -4,13 +4,15 @@ variables:
   DOTNET_IMAGE: mcr.microsoft.com/dotnet/sdk:6.0
   NODE_IMAGE: node:lts-stretch-slim
   DOCKER_IMAGE: docker:latest
+  PYTHON_IMAGE: python:slim-bullseye
 
 stages:
   - prepare
   - test
+  - build-docs
   - build-ui
   - build-server
-  - build-docs
+  - build-package-docs
   - post-build
   - pre-publish
   - publish
@@ -45,6 +47,23 @@ test:
     -   ./scripts/build/set-build-status.sh failure
     - fi
 
+build-docs:
+  image: $PYTHON_IMAGE
+  stage: build-docs
+  script:
+    - chmod +x ./scripts/build/build-docs.sh
+    - ./scripts/build/build-docs.sh
+  artifacts:
+    paths:
+      - docs/httpl-docs/site
+      - dist
+    expire_in: 1 week
+  after_script:
+    - if [ $CI_JOB_STATUS == 'failed' ]; then
+    -   chmod +x ./scripts/build/set-build-status.sh
+    -   ./scripts/build/set-build-status.sh failure
+    - fi
+
 build-ui:
   image: $NODE_IMAGE
   stage: build-ui
@@ -165,7 +184,7 @@ create-open-api-file:
 
 create-nuget-client-docs:
   image: $DEBIAN_IMAGE
-  stage: build-docs
+  stage: build-package-docs
   script:
     - chmod +x ./scripts/build/build-nuget-client-docs.sh
     - ./scripts/build/build-nuget-client-docs.sh

CHANGELOG

@@ -1,4 +1,4 @@
-[vnext]
+[2022.8.7]
 - Order stub execution results in UI alphabetically (https://github.com/dukeofharen/httplaceholder/pull/250).
 - Sort and group configuration items in help page in terminal (https://github.com/dukeofharen/httplaceholder/pull/250).
 - Added option to response body helper to format JSON and XML strings (https://github.com/dukeofharen/httplaceholder/pull/250).
@@ -9,6 +9,7 @@
 - Updated "update stub" endpoint so an HTTP 404 is returned when the stub ID in the URL does not exist (https://github.com/dukeofharen/httplaceholder/pull/250).
 - Fixed logging for condition checkers (https://github.com/dukeofharen/httplaceholder/pull/250).
 - Added response body parsers for scenario state and hit count (https://github.com/dukeofharen/httplaceholder/issues/182).
+- Added MkDocs project for the documentation. Release documentation as separate artifact and also package the documentation with the HttPlaceholder user interface (https://github.com/dukeofharen/httplaceholder/pull/254).
 
 # BREAKING CHANGES
 - Update stub call now returns HTTP 404 if the stub ID in the URL does not exist. This was HTTP 204 before.

docs/docs.md

@@ -8,6 +8,7 @@
   - [Docker](#docker)
   - [Hosting](#hosting)
 - **[Getting started](#getting-started)**
+- **[Stub anatomy](#stub-anatomy)**
 - **[Request conditions](#request-conditions)**
   - [General](#general-stub-info)
   - [Description](#description)
@@ -101,8 +102,6 @@ Make sure you have installed the correct .NET SDK (at least .NET 6) for your OS
 
 ## Windows
 
-### Local development setup
-
 The only thing needed to use HttPlaceholder on your local development machine, is extracting the archive with the HttPlaceholder binaries (which can be found [here](https://github.com/dukeofharen/httplaceholder/releases/latest)).
 
 For installing HttPlaceholder through your PowerShell terminal, execute the following command:
@@ -113,7 +112,79 @@ Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.We
 
 This is a self contained version of HttPlaceholder: no SDK has to be installed to run it.
 
-### Hosting under IIS
+## Linux
+
+The only thing needed to use HttPlaceholder on your local development machine, is extracting the archive with the HttPlaceholder binaries (which can be found [here](https://github.com/dukeofharen/httplaceholder/releases/latest)). You need to put HttPlaceholder on your path variable yourself.
+
+Alternatively, to install HttPlaceholder on Linux, run the following command in your terminal (make sure you're running as administrator):
+
+```bash
+sudo curl -o- https://raw.githubusercontent.com/dukeofharen/httplaceholder/master/scripts/install-linux.sh | sudo bash
+```
+
+This is a self contained version of HttPlaceholder: no SDK has to be installed to run it.
+
+If you would like to expose HttPlaceholder to the outside world, I would recommend to use Nginx or Apache as reverse proxy. To keep the service running even if you're not logged in through an SSH session, you can use something like systemd.
+
+There are two Vagrant boxes (Windows and Ubuntu) that you can use to view how installation of HttPlaceholder is done. You can find them in this repository under the folder "vagrant". You need to have [Vagrant](https://www.vagrantup.com/) installed. After that, it's just a matter of going to the correct folder in your terminal and typing `vagrant up`. HttPlaceholder will then be installed under Windows or Ubuntu and can be reached by going to `http://localhost:8080` or `https://localhost:4430`.
+
+## Mac
+
+The only thing needed to use HttPlaceholder on your local development machine, is extracting the archive with the HttPlaceholder binaries (which can be found [here](https://github.com/dukeofharen/httplaceholder/releases/latest)). You need to put HttPlaceholder on your path variable yourself.
+
+To install HttPlaceholder on Mac OS X, run the following command in your terminal (make sure you're running as administrator):
+
+```bash
+sudo curl -o- https://raw.githubusercontent.com/dukeofharen/httplaceholder/master/scripts/install-mac.sh | sudo bash
+```
+
+This is a self contained version of HttPlaceholder: no SDK has to be installed to run it.
+
+## Docker
+
+HttPlaceholder has a Docker image; it can be found [here](https://hub.docker.com/r/dukeofharen/httplaceholder). This page explains some basics about the Docker image and some examples you can use.
+
+### Basic example
+
+This is a very basic example for running HttPlaceholder locally from the command line.
+
+`docker run -p 5000:5000 dukeofharen/httplaceholder:latest`
+
+HttPlaceholder can now be reached on `http://localhost:5000` (or `http://localhost:5000/ph-ui` to get to the management interface).
+
+### Docker configuration
+
+The Docker container uses the configuration values as specified [here](#configuration). Here is an example of starting the HttPlaceholder container with different ports for HTTP and HTTPS:
+
+`docker run -p 8080:8080 -p 4430:4430 --env port=8080 --env httpsPort=4430 dukeofharen/httplaceholder:latest`
+
+### Docker Compose examples
+
+[Here](https://github.com/dukeofharen/httplaceholder/tree/master/docker) you can find a few Docker Compose examples how you can run HttPlaceholder using Docker.
+
+## Hosting
+
+### Running behind reverse proxy
+
+IIS, Nginx and Apache (and a lot of other web servers) have the option to run an application behind a reverse proxy. For HttPlaceholder to function correctly behind a reverse proxy, the server has to send a few "proxy" headers to HttPlaceholders. The following headers should be sent:
+
+- `X-Forwarded-For`: contains all IP addresses of the calling client and all proxy servers in between the client and HttPlaceholder. Used to determine the IP of the calling client.
+- `X-Forwarded-Proto`: contains the protocol of the original call to the proxying web server (`http` or `https`).
+- `X-Forwarded-Host`: contains the hostname of the original call to the proxying web server(e.g. `httplaceholder.com`).
+
+These headers are, right now, only used instead of the "real" values if the actual IP address of the proxy server is the loopback IP (e.g. `127.0.0.1`, `::1` etc.).
+
+Read more about this subject for the specific web servers:
+
+- [Nginx](https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/)
+- [Apache](https://httpd.apache.org/docs/2.4/mod/mod_proxy.html)
+- [IIS](https://techcommunity.microsoft.com/t5/iis-support-blog/setup-iis-with-url-rewrite-as-a-reverse-proxy-for-real-world/ba-p/846222)
+
+### Using SSL
+
+HttPlaceholder supports HTTPS. See [configuration](#configuration) for more information on this. By default, it uses the private key that is installed with HttPlaceholder. This file is named `key.pfx` and the password is `1234`. Before using HttPlaceholder and calling the HTTPS URL, you'll need to make sure to import and trust the .pfx file on your OS. For your convenience, three scripts (for Windows, Linux and Mac) are added for installing and trusting the .pfx file of HttPlaceholder. You can find the script in the installation folder: `install-private-key.sh` for Mac and Linux and `Install-Private-Key.ps1` for Windows.
+
+### Microsoft IIS
 
 It is also possible to host HttPlaceholder under IIS. You can just install HttPlaceholder using the installer mentioned above. You need to install the .NET Hosting Bundle in order for you to host .NET applications under IIS. You can find the installer at <https://dotnet.microsoft.com/download/dotnet-core/current/runtime>, and download the Hosting Bundle.
 
@@ -185,81 +256,10 @@ There are two Vagrant boxes (Windows and Ubuntu) that you can use to view how in
 
 - If you get an error something like `An unhandled exception was thrown by the application. code = ReadOnly (8), message = System.Data.SQLite.SQLiteException (0x800017FF): attempt to write a readonly database`, it means your SQLite database is not writable. Make sure the IIS user can write to this file.
 
-### Hosting as a Windows Service
+### Windows Service
 
 Hosting the application as a Windows Service (and subsequently using a reverse proxy in IIS to host the application) is officially not supported (maybe in the future). You can, however, use tools like [NSSM](http://nssm.cc/) (Non-Sucking Service Manager, brilliant name by the way) to host a console application as a Windows service. For configuration, you can either use the `config.json` file or the command line arguments.
 
-## Linux
-
-The only thing needed to use HttPlaceholder on your local development machine, is extracting the archive with the HttPlaceholder binaries (which can be found [here](https://github.com/dukeofharen/httplaceholder/releases/latest)). You need to put HttPlaceholder on your path variable yourself.
-
-Alternatively, to install HttPlaceholder on Linux, run the following command in your terminal (make sure you're running as administrator):
-
-```bash
-curl -o- https://raw.githubusercontent.com/dukeofharen/httplaceholder/master/scripts/install-linux.sh | sudo bash
-```
-
-This is a self contained version of HttPlaceholder: no SDK has to be installed to run it.
-
-If you would like to expose HttPlaceholder to the outside world, I would recommend to use Nginx or Apache as reverse proxy. To keep the service running even if you're not logged in through an SSH session, you can use something like systemd.
-
-There are two Vagrant boxes (Windows and Ubuntu) that you can use to view how installation of HttPlaceholder is done. You can find them in this repository under the folder "vagrant". You need to have [Vagrant](https://www.vagrantup.com/) installed. After that, it's just a matter of going to the correct folder in your terminal and typing `vagrant up`. HttPlaceholder will then be installed under Windows or Ubuntu and can be reached by going to `http://localhost:8080` or `https://localhost:4430`.
-
-## Mac
-
-The only thing needed to use HttPlaceholder on your local development machine, is extracting the archive with the HttPlaceholder binaries (which can be found [here](https://github.com/dukeofharen/httplaceholder/releases/latest)). You need to put HttPlaceholder on your path variable yourself.
-
-To install HttPlaceholder on Mac OS X, run the following command in your terminal (make sure you're running as administrator):
-
-```bash
-curl -o- https://raw.githubusercontent.com/dukeofharen/httplaceholder/master/scripts/install-mac.sh | sudo bash
-```
-
-This is a self contained version of HttPlaceholder: no SDK has to be installed to run it.
-
-## Docker
-
-HttPlaceholder has a Docker image; it can be found [here](https://hub.docker.com/r/dukeofharen/httplaceholder). This page explains some basics about the Docker image and some examples you can use.
-
-### Basic example
-
-This is a very basic example for running HttPlaceholder locally from the command line.
-
-`docker run -p 5000:5000 dukeofharen/httplaceholder:latest`
-
-HttPlaceholder can now be reached on `http://localhost:5000` (or `http://localhost:5000/ph-ui` to get to the management interface).
-
-### Docker configuration
-
-The Docker container uses the configuration values as specified [here](#configuration). Here is an example of starting the HttPlaceholder container with different ports for HTTP and HTTPS:
-
-`docker run -p 8080:8080 -p 4430:4430 --env port=8080 --env httpsPort=4430 dukeofharen/httplaceholder:latest`
-
-### Docker Compose examples
-
-[Here](https://github.com/dukeofharen/httplaceholder/tree/master/docker) you can find a few Docker Compose examples how you can run HttPlaceholder using Docker.
-
-## Hosting
-
-### Running behind reverse proxy
-
-IIS, Nginx and Apache (and a lot of other web servers) have the option to run an application behind a reverse proxy. For HttPlaceholder to function correctly behind a reverse proxy, the server has to send a few "proxy" headers to HttPlaceholders. The following headers should be sent:
-
-- `X-Forwarded-For`: contains all IP addresses of the calling client and all proxy servers in between the client and HttPlaceholder. Used to determine the IP of the calling client.
-- `X-Forwarded-Proto`: contains the protocol of the original call to the proxying web server (`http` or `https`).
-- `X-Forwarded-Host`: contains the hostname of the original call to the proxying web server(e.g. `httplaceholder.com`).
-
-These headers are, right now, only used instead of the "real" values if the actual IP address of the proxy server is the loopback IP (e.g. `127.0.0.1`, `::1` etc.).
-
-Read more about this subject for the specific web servers:
-- [Nginx](https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/)
-- [Apache](https://httpd.apache.org/docs/2.4/mod/mod_proxy.html)
-- [IIS](https://blogs.msdn.microsoft.com/webapps/2018/09/05/how-to-log-client-ip-when-iis-is-load-balanced-the-x-forwarded-for-header-xff/)
-
-### Using SSL
-
-HttPlaceholder supports HTTPS. See [configuration](#configuration) for more information on this. By default, it uses the private key that is installed with HttPlaceholder. This file is named `key.pfx` and the password is `1234`. Before using HttPlaceholder and calling the HTTPS URL, you'll need to make sure to import and trust the .pfx file on your OS. For your convenience, three scripts (for Windows, Linux and Mac) are added for installing and trusting the .pfx file of HttPlaceholder. You can find the script in the installation folder: `install-private-key.sh` for Mac and Linux and `Install-Private-Key.ps1` for Windows.
-
 # Getting started
 
 - Install HttPlaceholder (see [Installation](#installation)).
@@ -307,6 +307,44 @@ HttPlaceholder supports HTTPS. See [configuration](#configuration) for more info
 
 For more sophisticated examples, go to the paragraph [samples](#samples) to view samples for all supported HTTP condition checkers and response writers.
 
+# Stub anatomy
+
+A stub is simply a piece of YAML that contains instructions for HttPlaceholder for how the request should look like and what the response should look like when a request was matched against this stub. Besides this, some extra metadata can be provided to the stub. For reference, take a look at the example below.
+
+```yml
+- id: situation-03
+  tenant: users-api
+  scenario: users-api-scenario
+  priority: 1
+  description: Description of the stub.
+  enabled: true
+  conditions:
+    method: GET
+    url:
+      path:
+        equals: /users
+      query:
+        id:
+          equals: 15
+        filter:
+          equals: last_name
+  response:
+    statusCode: 200
+    json: |
+      {
+        "last_name": "Jackson"
+      }
+```
+
+- `id`: the unique ID of the stub. This should ALWAYS be filled in.
+- `conditions`: all the conditions a request should match before the stub is picked. Read more about it [here](#request-conditions).
+- `response`: the response that should be returned if the request matches according to the conditions. Read more about it [here](#response-writers).
+- `tenant`: a way to group several stubs together. Read more about it [here](#tenants).
+- `scenario`: a way to put a stub under a scenario. If the stub is part of a scenario, it will be possible for the stub to check for the scenario hit count or the scenario state. It is also possible to set the scenario state or clear the scenario state. Multiple stubs can be part of the same scenario. To read more about it, go [here](#scenarios) or [here](#request-scenario).
+- `priority`: a way to give a priority to a stub. If multiple stubs are found for a request, the stub with the highest priority will take precedence. Read more about it [here](#priority).
+- `description`: just a simple way for you to specify what the stub is for.
+- `enabled`: simple way of specifying if a stub is enabled or disabled. Read more about it [here](#enabled).
+
 # Request conditions
 
 Whenever HttPlaceholder receives a request, all the conditions of all stubs are checked to see which stub corresponds to the sent request. There are condition checkers for example the URL, posted data etc. This paragraph explains more.

docs/httpl-docs/.gitignore

@@ -0,0 +1,2 @@
+docs/
+site/

docs/httpl-docs/mkdocs.yml

@@ -0,0 +1,3 @@
+site_name: HttPlaceholder documentation
+theme: readthedocs
+site_description: HttPlaceholder documentation - Quickly stub away any HTTP service

docs/httpl-docs/sync.py

@@ -0,0 +1,62 @@
+import os
+import shutil
+import re
+
+dir_path = os.path.dirname(os.path.realpath(__file__))
+mkdocs_docs_path = os.path.join(dir_path, 'docs')
+
+# Create mkdocs docs folder if it doesn't exist yet.
+if not os.path.isdir(mkdocs_docs_path):
+    os.mkdir(mkdocs_docs_path)
+
+# Copy index.md to the correct location.
+docs_md_path = os.path.join(dir_path, '..', 'docs.md')
+docs_md_copy_path = os.path.join(mkdocs_docs_path, 'index.md')
+shutil.copyfile(docs_md_path, docs_md_copy_path)
+
+# Copy CHANGELOG file
+changelog_path = os.path.join(dir_path, '..', '..', 'CHANGELOG')
+changelog_path_copy_path = os.path.join(mkdocs_docs_path, 'CHANGELOG')
+shutil.copyfile(changelog_path, changelog_path_copy_path)
+
+# Remove table of contents from copied file.
+docs_md_file_read = open(docs_md_copy_path)
+lines = docs_md_file_read.readlines()
+docs_md_file_read.close()
+lines_result = []
+add_line = False
+for line in lines:
+    if line.startswith('# Installation'):
+        add_line = True
+    if add_line:
+        lines_result.append(line.rstrip('\n'))
+
+docs_md = '\n'.join(lines_result)
+
+
+def re_replace(find, replace, input):
+    return re.sub(find, replace, input, flags=re.M)
+
+
+docs_md = re_replace('^#### ', '##### ', docs_md)
+docs_md = re_replace('^### ', '#### ', docs_md)
+docs_md = re_replace('^## ', '### ', docs_md)
+docs_md = re_replace('^# ', '## ', docs_md)
+docs_md = '![](img/logo.png)\n\n' + docs_md
+docs_md_file_write = open(docs_md_copy_path, 'w')
+docs_md_file_write.write(docs_md)
+docs_md_file_write.close()
+
+# Copy img folder
+img_path = os.path.join(dir_path, '..', 'img')
+img_copy_path = os.path.join(mkdocs_docs_path, 'img')
+if os.path.isdir(img_copy_path):
+    shutil.rmtree(img_copy_path)
+shutil.copytree(img_path, img_copy_path)
+
+# Copy samples folder
+samples_path = os.path.join(dir_path, '..', 'samples')
+samples_copy_path = os.path.join(mkdocs_docs_path, 'samples')
+if os.path.isdir(samples_copy_path):
+    shutil.rmtree(samples_copy_path)
+shutil.copytree(samples_path, samples_copy_path)