docs: Migrate wiki docs to main repo

[polarathene]

Description

Migrating Github Wiki docs into the main repo with the following commands:

# Clone the two repositories to combine:
git clone https://github.com/docker-mailserver/docker-mailserver
git clone https://github.com/docker-mailserver/docker-mailserver.wiki

# Navigate to repository we want to migrate files from.
# `git-filter-repo` relocates the wiki docs from project root to sub-directory, while retaining author commit history:
cd docker-mailserver.wiki
git-filter-repo --path-rename ':website/docs/'

# Switch to main repo, create a branch for PR, add the other local repo as a remote:
cd ../docker-mailserver
git checkout -b docs/migrate-docs
git remote add wiki-local ../docker-mailserver.wiki

# Transfer the remotes files with commit history in tact to main repo:
# `git pull` to perform `git fetch` and `git merge` together.
git pull wiki-local master --allow-unrelated-histories

--allow-unrelated-histories is the usual advice for this approach. However when files are relocated, they are considered "renamed" and can lose their commit history. I've come across a solution with git-filter-repo that allowed me to migrate the wiki docs files into a sub-directory.

This location is websites/docs instead of docs which better isolates and identifies it's purpose (to be used with a static doc generator). No changes have been made, the large commit history is just for the wiki files. As a wiki contributor I'd like to retain my history of contributions to it :)

This was something I wanted to look into for a while, as did other contributors. There was a recent off-topic discussion in a PR, and this is the PR I said I would sort out from that discussion after getting my earlier ones merged.

Type of change

• Improvement (non-breaking change that does improve existing functionality)
• This change requires a documentation update (wiki pages should be removed in future to avoid falling out of sync, with the main wiki page remaining to direct users to new docs site and/or new location of docs in repo)

Checklist:

• I have made corresponding changes to the documentation (README.md or ENVIRONMENT.md or the Wiki)

---

I have made a basic Docusaurus MkDocs Material demo hosted on Github Pages (gh-pages branch) that is automated test/deploy via Github Action I put together. You can view it here: https://polarathene.github.io/docker-mailserver/config/security/understanding-the-ports/

[wernerfred]

FYI: We have an ongoing discussion in #1826 which requires an update to this PR

[polarathene]

Docs should be good to merge now to the agreed docs/content path. I've excluded the Github Wiki specific _footer.md and _sidebar.md files from this PR. Their present content is:

_Footer.md

© [_Docker Mailserver Organization_](https://github.com/docker-mailserver)

This project is licensed under the MIT license.

_Sidebar.md

- [**Home**](./)
- [**An Introduction to Mail Servers**](./Introduction)

---

#### Configuration

- Your best friend [setup.sh](./setup.sh)
- User management
  - [Accounts](./Configure-Accounts)
  - [Aliases](./Configure-aliases)
- Best practices
  - [DKIM](./Configure-DKIM)
  - [DMARC](./Configure-DMARC)
  - [SPF](./Configure-SPF)
  - [Autodiscovery](./Configure-autodiscover)
- Security
  - [Understanding ports](./Understanding-the-ports)
  - [SSL/TLS](./Configure-SSL)
  - [Fail2ban](./Configure-Fail2ban)
- When something went wrong
  - [Debugging](./Debugging)
  - [FAQ](./FAQ-and-Tips)
- [Mail delivery with POP3](./Configure-POP3)

---

#### Advanced

- [Optional configuration](./List-of-optional-config-files-&-directories)
- Maintenance
  - [Update & cleanup](./Update-and-cleanup)
- Override the default config of
  - [Dovecot](./Override-Default-Dovecot-Configuration)
  - [Postfix](./Override-Default-Postfix-Configuration)
- [LDAP authentication](./Configure-LDAP)
- [Email filtering with Sieve](https://github.com/tomav/docker-mailserver/wiki/Configure-Sieve-filters)
- [Email gathering with fetchmail](./Retrieve-emails-from-a-remote-mail-server-(using-builtin-fetchmail))
- Email forwarding with 
  - [Relay Hosts](./Configure-Relay-Hosts)
  - [AWS SES](./Configure-AWS-SES)
- [Full-text search](./Full-text-search)
- [Kubernetes](./Using-in-Kubernetes)
- [IPv6](./IPv6)

---

#### Tutorials

- [FAQ](./FAQ-and-Tips)
- [Installation examples](./Installation-examples)

---

#### Use Cases

- [Forward-Only mailserver with LDAP authentication](./Forward-Only-mailserver-with-LDAP-authentication)

AFAIK, git log --follow allows for properly tracking commit history when files are moved / renamed. Git GUI clients also usually support this, but Github has been known for years to not support this and actively ignores discussions about it for some reason.

git-filter-repo rewrites the history of the wiki commits to present them as if they had this they always had the new file location and name. This should at least retain history via Github UI unless we move/rename files again in future, anyone that wants to view history prior to that can rest assured that it's retained, it's just Githubs UI being misleading.

Here's the reproduction script for this PR:

transfer-wiki-commits.sh:

#! /usr/bin/env bash

#################################################################################
#
# Extract Github Wiki commits to transfer to another repo and keep commit history
#
#################################################################################

# Clone the two repositories to combine:
git clone https://github.com/docker-mailserver/docker-mailserver --single-branch
git clone https://github.com/docker-mailserver/docker-mailserver.wiki



#########################################################################################################################################
#
# `git-filter-repo` rewrites history to accomodate for any changes (file rename and location): https://github.com/newren/git-filter-repo
#
# May not be needed as commits of these changes without internal file content changes can usually be tracked by Git (`git log --follow`).
# Git continues to improve it's tracking over the years, eg: In 2021 with Git 2.31 (https://stackoverflow.com/a/36615639/11681500)
# Only Github's history view doesn't present history related to such changes. This is mostly an attempt to improve UX for contributors.
#
#########################################################################################################################################

# `git-filter-repo` commands will be run from this repo:
cd docker-mailserver.wiki

# No value in having mixed casing, rename all filenames to be lowercase:
git filter-repo --filename-callback 'return filename.lower()'

# These are Github Wiki specific, not required for porting:
# Similar to `rm _footer.md _sidebar.md` but doesn't require a commit, removes files from history entirely
git filter-repo --invert-paths --path '_footer.md' --path '_sidebar.md'

# Files are matched and relocated to directory hierarchy matching `_sidebar.md`:
# Many filenames are also modified for convenience
git filter-repo --paths-from-file ../rename-hierarchy.txt

# Migrate the content from top-level to a subdirectory:
# Equivalent to `--path-rename :docs/content/`
git filter-repo --to-subdirectory-filter docs/content/



#####################################################
#
# Copy over the commits while retaining their history
#
#####################################################

# Switch to the primary repo, create a branch for PR, add the local wiki repo as a remote:
cd ../docker-mailserver
git checkout -b docs/migrate-docs
git remote add wiki-local ../docker-mailserver.wiki

# Transfer the files (with commit history intact) from the `wiki-local` remote to the primary repo:
# `--allow-unrelated-histories` carries over the commit history
git fetch wiki-local
git merge --allow-unrelated-histories wiki-local/master

rename-hierarchy.txt:

# Github Wiki uses flat hierarchy structure for files.
# This uses `--paths-from-file` to rename/relocate files
# Note `git-filter-repo` docs state you should filter paths before renaming them (didn't seem required here): https://github.com/newren/git-filter-repo/issues/96#issuecomment-786320550

home.md==>index.md

# Configuration
setup.sh.md==>config/setup.sh.md

regex:configure-(accounts|aliases)\.md$==>config/user-management/\1.md

regex:configure-(dkim|dmarc|spf|autodiscover)\.md$==>config/best-practices/\1.md

regex:configure-(ssl|fail2ban)\.md$==>config/security/\1.md
understanding-the-ports.md==>config/security/understanding-the-ports.md

faq-and-tips.md==>faq.md
regex:(debugging|faq)\.md$==>config/troubleshooting/\1.md

configure-pop3.md==>config/pop3.md


# Advanced
list-of-optional-config-files-&-directories.md==>advanced/optional-config.md
update-and-cleanup.md==>advanced/maintenance/update-and-cleanup.md

regex:override-default-(.*)-configuration\.md==>advanced/override-defaults/\1.md

configure-ldap.md==>advanced/auth-ldap.md

regex:.*(sieve|fetchmail).*\.md==>advanced/mail-\1.md
regex:configure-(aws-ses|relay-hosts)\.md==>advanced/mail-forwarding/\1.md

regex:.*(full-text-search|kubernetes|ipv6)\.md==>advanced/\1.md


# Tutorials

installation-examples.md==>tutorials/installation-examples.md


# Use Cases

forward-only-mailserver-with-ldap-authentication.md==>uses-cases/forward-only-mailserver-with-ldap-authentication.md

[wernerfred]

LGTM 👍🏻
I will squash-merge with your approval.

[polarathene]

I will squash-merge with your approval.

Won't that defeat the purpose and remove history? git blame would just have all lines point to this commit/PR, there wouldn't be past revisions. I'll verify this on my end by performing the actions on my clone later today, but I'd like to retain the history properly.

A rebase merge would dirty the master branch history by dumping almost 500 commits, but a merge commit (without the squash) should retain it and not dirty the branch as visibly (the commits would appear interleaved relevant to their commit date AFAIK, but not rewrite history). Normally I'd advocate for squash and merge for PRs, but not this time if I understand the different kinds correctly.

[wernerfred]

You are absolutely right. My Bad. I will of course merge (without Squash).
I will nevertheless wait again for your approval/response and (as it is already midnight) merge tomorrow.

[wernerfred]

Ready to Go?

[polarathene]

Yep, merge commit away and I'll cherry pick my additions over to your branch 👍