diff --git a/CONTRIBUTING.MD b/CONTRIBUTING.MD new file mode 100644 index 000000000..f07195b31 --- /dev/null +++ b/CONTRIBUTING.MD @@ -0,0 +1,74 @@ +# Contributing + +## Prerequisites + +- [.NET 9.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) +- [Node.js 22.13.1 (LTS)](https://nodejs.org/en/blog/release/v22.13.1) + - [Aspire 9.4.1](https://learn.microsoft.com/en-us/dotnet/aspire/) + ```bash + dotnet workload install aspire + ``` + +## Validate the fully assembled documentation + +```bash +dotnet run --project aspire +``` + +Will spin up all our services and clone and build all the documentation sets. + +```markdown +dotnet run --project aspire -- --assume-cloned --skip-private-repositories +``` + +`--assume-cloned` will assume a documentation set set is already cloned if available locally. + +`--skip-private-repositories` will skip cloning private repositories. It will also inject our `docs-builder docs into the +navigation. This allows us to validate new features' effect on the assembly process. + +Our [Integration Tests](./tests-integration/Elastic.Assembler.IntegrationTests) use this exact command to validate the +assembler builds. + +## Continuously build all assets during development. + +```shell +./build.sh watch +``` + +This will monitor code, cshtml template files & static files and reload the application +if any changes. + +Web assets are reloaded through `parcel watch` and don't require a recompilation. + +Markdown files are refreshed automatically through livereload + +Code or layout changes will relaunch the server automatically + +# Release Process + +This section outlines the process for releasing a new version of this project. + +## Versioning + +This project uses [Semantic Versioning](https://semver.org/) and its version is +automatically determined by [release-drafter](https://github.com/release-drafter/release-drafter) +based on the labels of the pull requests merged into the `main` branch. + +See the [release-drafter configuration](./.github/release-drafter.yml) for more details. + +## Creating a New Release + +To create a new release trigger the [release](https://github.com/elastic/docs-builder/actions/workflows/release.yml) workflow on the `main` branch. + +Every time a pull request is merged into the `main` branch, release-drafter will +create a draft release or update the existing draft release in the [Releases](https://github.com/elastic/docs-builder/releases) page. + +To create a new release you need to publish the existing draft release created by release-drafter. + +> [!IMPORTANT] +> Make sure the [release-drafter workflow](https://github.com/elastic/docs-builder/actions/workflows/release-drafter.yml) is finished before publishing the release. + +> [!NOTE] +> When a release is published, the [create-major-tag workflow](./.github/workflows/create-major-tag.yml) +> will force push a new major tag in the format `vX` where `X` is the major version of the release. +> For example, if the release is `1.2.3` was published, the workflow will force push a new tag `v1` on the same commit. diff --git a/README.md b/README.md index 76d189a1e..4abb8c8ef 100644 --- a/README.md +++ b/README.md @@ -2,84 +2,90 @@ [![ci](https://github.com/elastic/docs-builder/actions/workflows/ci.yml/badge.svg?branch=main&event=push)](https://github.com/elastic/docs-builder/actions/workflows/ci.yml) -You've reached the home of the latest incarnation of the documentation tooling. +Distributed documentation tooling for a distributed company. This repository is host to: -* *`docs-builder`* command line tool to generate single doc-sets (13mb native code, no dependencies) -* *`docs-assembler`* command line tool to assemble all the doc sets. (IN PROGRESS) -* `elastic/docs-builder@main` Github Action to build and validate a repositories documentation +* *`docs-builder`* command line tool to generate single doc-sets +* *`docs-assembler`* command line tool to assemble all the doc sets. -## Command line interface +Both get distributed [as native OSX, Linux and Windows binaries for several CPU architectures.](#installation) -```bash -$ docs-builder --help -Usage: [command] [options...] [-h|--help] [--version] - -Converts a source markdown folder or file to an output folder - -Options: - -p|--path Defaults to the`{pwd}/docs` folder (Default: null) - -o|--output Defaults to `.artifacts/html` (Default: null) - --path-prefix Specifies the path prefix for urls (Default: null) - --force Force a full rebuild of the destination folder (Default: null) - -Commands: - generate Converts a source markdown folder or file to an output folder - serve Continuously serve a documentation folder at http://localhost:3000. - diff validate Validates redirect rules have been applied to the current branch. - File systems changes will be reflected without having to restart the server. -``` +The documentation files: +* are written in common Markdown with [Additional syntax extensions](https://docs-v3-preview.elastic.dev/elastic/docs-builder/tree/main/syntax/) to produce a richer writing and reading experience. +* By conventions the documentation lives in `docs` folder but the tooling will look for any folder holding the `docset.yml` configuration file given the current working directory. -In the near term native code will be published by CI for the following platforms +## Distributed documentation -| OS | Architectures | -|----------|---------------| -| Windows | x64, Arm64 | -| Linux | x64, Arm64 | -| macOS | x64, Arm64 | +The main driver for folks writing documentation is `docs-builder`. +This tool builds each repository in isolation and in addition produces a full mapping of all the linkable resources it contains in a `links.json` file. -And we'll invest time in making sure these are easily obtainable (`brew`, `winget`, `apt`) +Each time a repository successfully builds on its respective main integration branch, our automation will publish its links.json file. +For example, [Elasticsearch's links.json](https://elastic-docs-link-index.s3.us-east-2.amazonaws.com/elastic/elasticsearch/main/links.json) representing all linkable resources in the Elasticsearch repository. -For now you can run the tool locally through `docker run` +The `docs-assembler` tool then assembles all the repositories in the [link-registry](https://elastic-docs-link-index.s3.us-east-2.amazonaws.com/link-index.json) using their last known good commit. -```bash -docker run -v "./.git:/app/.git" -v "./docs:/app/docs" -v "./.artifacts:/app/.artifacts" \ - ghcr.io/elastic/docs-builder:edge -``` +This allows us to: -This ensures `.git`/`docs` and `.artifacts` (the default output directory) are mounted. +* Validate outbound and inbound links ahead of time, even during local `docs-builder` builds. +* Snapshot builds: only building commits that produced a `links.json` + * Documentation errors in one repository won't affect all the others. + * Resilient to repositories having build failures on their integration branches, we fall back to the last known good commit. -The tool will default to incremental compilation. -Only the changed files on subsequent runs will be compiled unless you pass `--force` -to force a new compilation. +## Installation +### Linux / OSX ```bash -docker run -v "./.git:/app/.git" -v "./docs:/app/docs" -v "./.artifacts:/app/.artifacts" \ - ghcr.io/elastic/docs-builder:edge --force +curl -sL https://ela.st/docs-builder-install | sh ``` -#### Live mode +### Windows +```ps +iex (New-Object System.Net.WebClient).DownloadString('https://ela.st/docs-builder-install-win') +``` + +Installing through the script will download the latest version of the tool and make it available in your `PATH`. + +If you want to manually install the tool you can download the latest release from the [Releases Page](https://github.com/elastic/docs-builder/releases) -Through the `serve` command you can continuously and partially compile your documentation. +### Building locally + +Install [.NET 9.0](https://dotnet.microsoft.com/en-us/download/dotnet/9.0), then run: ```bash -docker run -v "./.git:/app/.git" -v "./docs:/app/docs" -v "./.artifacts:/app/.artifacts" \ - -p 3000:3000 ghcr.io/elastic/docs-builder:edge serve +./build.sh publishbinaries ``` +After which the locally built binaries will be available at: -Each page is compiled on demand as you browse http://localhost:3000 and is never cached so changes to files and -navigation will always be reflected upon refresh. +* **docs-builder**: `./.artifacts/publish/docs-builder/release/docs-builder` +* **docs-assembler**: `./.artifacts/publish/docs-assembler/release/docs-assembler` + +## Getting Started -Note the docker image is `linux-x86` and will be somewhat slower to invoke on OSX due to virtualization. +Our [Documentation](https://docs-v3-preview.elastic.dev/elastic/docs-builder/tree/main/contribute/locally) is the best place to learn how to start using the tool locally. +The TLDR, however, is + +* Running `docs-builder` from the root of any checkout with a `docs` folder will build the documentation. + * Running `docs-builder` consecutively will only rebuild the documentation that has changed. + * Running `docs-builder` with the `--force` flag will force a full rebuild of the documentation. + * You can target different folders by passing a path to `docs-builder --path ` +* Running `docs-builder serve` will provide a local server with live reloading. + * You can leave this command running while you add/remove/rename files in your `docs` folder. + + +### Other commands to know: + +* `docs-builder mv` [Move files and folders](https://docs-v3-preview.elastic.dev/elastic/docs-builder/tree/main/contribute/move) +* `docs-builder diff validate` [Manage redirects across doc sets](https://docs-v3-preview.elastic.dev/elastic/docs-builder/tree/main/contribute/redirects#validation) +* `docs-builder inbound-links validate-link-reference` can be used after a build to validate the local `links.json` against all published documentation. -## Github Action -The `docs-builder` tool is available as github action. +## Github Action -Since it runs from a precompiled distroless image `~25mb` it's able to execute snappy. (no need to wait for building the tool itself) +The `docs-builder` tool is available as a GitHub action. +It runs as native code on a distroless image `~25mb` so there is little overhead invoking the tooling. ```yaml jobs: @@ -91,12 +97,9 @@ jobs: uses: elastic/docs-builder@main ``` - - ### GitHub Pages -To set up the tool to publish to GitHub pages use the following configuration. -**NOTE**: In the near feature we'll make this a dedicated single step GitHub action +To set up the tool to publish to GitHub Pages, use the following configuration. ```yaml environment: @@ -110,6 +113,8 @@ steps: id: deployment ``` +This single action will build and validate the documentation before publishing. + Make sure your repository settings are set up to deploy from GitHub actions see: https://github.com/elastic/{your-repository}/settings/pages @@ -119,84 +124,6 @@ https://github.com/elastic/{your-repository}/settings/pages --- -## Validating redirection rules - -If documentation is moved, renamed or deleted, `docs-builder` can verify if changes in the working branch in relation to the default branch are reflected in the repository's `redirects.yml`. Verification in the local machine is currently supported. - -`docs-builder diff validate ` - -`` is an optional parameter to customize the documentation folder path. It defaults to `docs`. - ---- - -## Run without docker - -You can use the .NET CLI to publish a self-contained `docs-builder` native code -binary. (On my M2 Pro mac the binary is currently 16mb) - -Install [.NET 9.0](https://dotnet.microsoft.com/en-us/download/dotnet/9.0), then run: - -```bash -dotnet publish "src/docs-builder/docs-builder.csproj" -``` - -The resulting binary `./.artifacts/publish/docs-builder/release/docs-builder` will run on machines without .NET installed. - -# Performance - -To test performance it's best to build the binary and run outside of docker: - -For reference here's the `markitpy-doc` docset (50k markdown files) currently takes `14s` vs `several minutes` compared to -existing surveyed tools - # Local Development -## Prerequisites - -- [.NET 9.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) -- [Node.js 22.13.1 (LTS)](https://nodejs.org/en/blog/release/v22.13.1) - - -## Continuously build all assets during development. - -```shell -./build.sh watch -``` - -This will monitor code, cshtml template files & static files and reload the application -if any changes. - -Web assets are reloaded through `parcel watch` and don't require a recompilation. - -Markdown files are refreshed automatically through livereload - -Code or layout changes will relaunch the server automatically - -# Release Process - -This section outlines the process for releasing a new version of this project. - -## Versioning - -This project uses [Semantic Versioning](https://semver.org/) and its version is -automatically determined by [release-drafter](https://github.com/release-drafter/release-drafter) -based on the labels of the pull requests merged into the `main` branch. - -See the [release-drafter configuration](./.github/release-drafter.yml) for more details. - -## Creating a New Release - -To create a new release trigger the [release](https://github.com/elastic/docs-builder/actions/workflows/release.yml) workflow on the `main` branch. - -Every time a pull request is merged into the `main` branch, release-drafter will -create a draft release or update the existing draft release in the [Releases](https://github.com/elastic/docs-builder/releases) page. - -To create a new release you need to publish the existing draft release created by release-drafter. - -> [!IMPORTANT] -> Make sure the [release-drafter workflow](https://github.com/elastic/docs-builder/actions/workflows/release-drafter.yml) is finished before publishing the release. - -> [!NOTE] -> When a release is published, the [create-major-tag workflow](./.github/workflows/create-major-tag.yml) -> will force push a new major tag in the format `vX` where `X` is the major version of the release. -> For example, if the release is `1.2.3` was published, the workflow will force push a new tag `v1` on the same commit. +See [CONTRIBUTING.md](CONTRIBUTING.md) for more information on how to develop locally and contribute to the project. diff --git a/docs-builder.sln b/docs-builder.sln index dd6035845..7af8a0bd8 100644 --- a/docs-builder.sln +++ b/docs-builder.sln @@ -13,6 +13,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "root", "root", "{94AC66E5-3 dotnet-tools.json = dotnet-tools.json global.json = global.json Directory.Packages.props = Directory.Packages.props + CONTRIBUTING.MD = CONTRIBUTING.MD EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{BE6011CC-1200-4957-B01F-FCCA10C5CF5A}"