Merge pull request #2252 from layer5io/leecalcote/docs/build-release

[Docs] Build and Release (CI) page

docs/_data/toc.yml

@@ -10,6 +10,8 @@
           url: project/contributing
         - title: Releases
           url: project/releases
+        - title: Build & Release (CI)
+          url: project/build-and-release
         - title: Vulnerabilities
           url: project/security-vulnerabilities
         - title: FAQ

docs/pages/project/ci-build-and-release.md

@@ -0,0 +1,201 @@
+---
+layout: default
+title: Build & Release (CI)
+permalink: project/build-and-release
+---
+
+Meshery’s build and release system incorporates many tools, organized into different workflows each triggered by different events. Meshery’s build and release system does not run on a schedule, but is event-driven. GitHub Actions are used to define Meshery’s CI workflows. New builds of Meshery and its various components are automatically generated upon push, release, and other similar events, typically in relation to their respective master branches.
+
+## Artifacts
+
+Today, Meshery and Meshery adapters are released as Docker container images, available on Docker Hub. Meshery adapters are out-of-process adapters (meaning not compiled into the main Meshery binary), and as such, are independent build artifacts.The process of creating Docker images, tagging with the git commit SHA and pushing to Docker Hub is being done automatically using GitHub Actions.
+
+### Artifact Repositories
+
+Artifacts produced in the build processes are published and persisted in different public repositories and in different formats.
+
+| Location      | Project       | Repository    |
+| ------------- | ------------- | ------------- |
+| Docker Hub    | Meshery       | [https://hub.docker.com/r/layer5/meshery](https://hub.docker.com/r/layer5/meshery) |
+| GitHub        | mesheryctl    | [https://github.com/layer5io/meshery/releases](https://github.com/layer5io/meshery/releases) |
+| Docker Hub    | Meshery Adapter for \<service-mesh\> | https://hub.docker.com/r/layer5/meshery-\<service-mesh\> |
+| Docs          | Meshery Documentation | [https://docs.meshery.io](https://docs.meshery.io) |
+| GitHub        | [Service Mesh Performance](https://smp-spec.io) | [https://github.com/layer5io/service-mesh-performance](https://github.com/layer5io/service-mesh-performance) |
+
+## Secrets
+
+Some portions of the workflow require secrets to accomplish their tasks. These secrets are defined within the respective repositories and accessible to workflows during runtime. Currently defined secrets include:
+
+- `DOCKER_USERNAME`: Username of the Docker Hub user with the right privileges to push images
+- `DOCKER_PASSWORD`: Password for the Docker Hub user
+- `GO_VERSION`: As of December 9th 2020 it is 1.15
+- `IMAGE_NAME`: appropriate image name for each of the Docker container images. All are under the `layer5io` org.
+- `SLACK_BOT_TOKEN`: Used for notification of new GitHub stars given to the Meshery repo.
+- CYPRESS_RECORD_KEY`: Used for integration with the Layer5 account on Cypress.
+- `GLOBAL_TOKEN`: Used for securely transmitting performance test results for the None Provider.
+
+The Docker Hub user, `mesheryci`, belongs to the "ciusers" team in Docker Hub and acts as the service account under which these automated builds are being pushed. Every time a new Docker Hub repository is created we have to grant “Admin” (in order to update the README in the Docker Hub repository) permissions to the ciusers team.
+
+## Checks and Tests
+Meshery’s CI workflow incorporates several checks (partial list below) during merges and/or commits to any branches and pull requests to master branch to prevent broken code from being merged into master. 
+
+Collectively, Meshery repositories will generally have CI workflow for commits and pull requests that consist of the following actions:
+
+- Lint check (golint)
+- Static analysis check (staticcheck)
+- Vet (govet)
+- Security checks (gosec)
+- Unit tests (go tests)
+- Build (go build)
+- Release binaries through GoReleaser (only for mesheryctl in the Meshery repository)
+- Docker build, tag and push
+
+## Automated Builds
+
+All Meshery GitHub repositories are configured with GitHub Actions. Everytime a pull request is submitted against the master branch of any repository, that repository’s GitHub Actions will be invoked (whether the PR is merged or not). Workflows defined in Meshery repository will generally (but not always) perform the following actions:
+
+1. trigger a Docker build to build a Docker container image
+1. generate two Docker tags: 
+   1. a tag containing the git merge SHA
+   1. a tag containing that particular release’s git tag (if one is present)
+1. assign each of these two tags to the new container image as well as the latest tag.
+1. push the new Docker tags and image to Docker Hub.
+
+### Building `mesheryctl`
+
+As a special case, the meshery repository contains an additional artifact produced during each build. This artifact is mesheryctl which is built as an executable binary. In order to make the job of building mesheryctl easier for a combination of different platform architectures and operating systems, we are using [GoReleaser](https://goreleaser.com). Irrespective of branch, for every git commit and git push to the meshery repository, GoReleaser will execute and generate the OS and arch-specific binaries ( but will NOT publish them to GitHub). Even though mesheryctl binaries are built each time a pull request is merged to master, only stable channel artifacts are published (persisted).
+
+### Releasing `mesheryctl` to GitHub
+
+Only when a git tag containing  a semantic version number is present (is a commit in the master branch) will GoReleaser execute, generate the archives, and also publish the archives to [Meshery’s GitHub releases](https://github.com/layer5io/meshery/releases) automatically. GoReleaser is configured to generate artifacts for the following OS, ARCH combination:
+
+- Darwin - i386, x86_64
+- Linux - i386, x86_64
+- Windows - i386, x86_64
+- FreeBSD - i386, x86_64
+
+The artifacts will be made available as a tar.gz archive for all the operating systems. mesheryctl is bundled into packages for commonly used package managers: homebrew and scoop.
+
+#### Homebrew
+
+GoReleaser facilitates the creation of a brew formula for mesheryctl. The [homebrew-tap](https://github.com/layer5io/homebrew-tap) repository is the location for Layer5’s brew formulas.
+
+#### Scoop
+
+GoReleaser facilitates the creation of a Scoop app for mesheryctl. The [scoop-bucket](https://github.com/layer5io/scoop-bucket) repository is the location of Layer5’s Scoop bucket.
+
+## Release Versioning
+
+We follow the commonly used semantic versioning for Meshery, Meshery Adapter and Performance Benchmark Specification releases. Given a version number MAJOR.MINOR.PATCH.BUILD, increment the:
+
+- MAJOR version - major changes with rare potential for incompatible API changes.
+- MINOR version - add functionality in a backwards-compatible manner.
+- PATCH version - mostly for bug and security fixes.
+- AlPHA/BETA/RC - used to facilitate early testing of an upcoming release.
+
+### Component Versioning
+
+Meshery comprises a number of components including a server, adapters, UI, and CLI. As an application, Meshery is a composition of these different functional components. While all of Meshery’s components generally deploy as a collective unit (together), each component is versioned independently, so as to allow them to be loosely coupled and iterate on functionality independently.  Some of the components must be upgraded simultaneously, while others may be upgraded independently. See [Upgrading Meshery](/guide/upgrade) for more information.
+
+GitHub release tags will contain a semantic version number. Semantic version numbers will have to be managed manually by tagging a relevant commit in the master branch with a semantic version number (example: v1.2.3). 
+
+## Release Process
+
+Documentation of Meshery releases contains a table of releases and release notes and should be updated with each release.
+
+### Automated Releases
+
+Releases are manually triggered by a member of the release team publishing a release. Release names and release tags need to be assigned by the publishing team member. GitHub Action workflows will trigger and take care of running the required steps and publishing all artifacts (e.g., binary and docker images).
+
+### Workflow Triggers
+
+The following events will trigger one or more workflows:
+
+1. Tagged Release
+1. Commit pushed to the master branch
+1. PR opened or commit pushed to PR branch
+1. PR merged to the master branch
+
+### Release Notes
+
+While use of GitHub Actions facilitates automated builds, ReleaseDrafter is helping with facilitating automated release notes and versioning. 
+### Generating Release Notes
+
+ReleaseDrafter generates a GitHub tag and release draft. ReleaseDrafter action will trigger and will automatically draft release notes according to the configuration set-up. ReleaseDrafter drafts releases as soon as a commit is made into master after the previous release. The GitHub Action, ReleaseDrafter, is compatible with semantic releases and is used to auto-increment the semantic version number by looking at the previous release version. 
+
+#### Automated Release Notes Publishing
+
+The publishing of release notes to Meshery Docs is automated. Triggered by a release event, a workflow will checkout the Meshery repo, copy the auto-drafted release notes into a Jekyll collection in Meshery Docs, and generate a pull request.
+
+#### Automated Pull Request Labeler
+
+A GitHub Issue labeler bot is configured to automatically assign labels to issues based on which files have changed in which directories. For example, a pull request with changes to files in the “/docs/**” folder will receive the “area/docs” label. Presence of the “area/docs” label is used to trigger documentation builds and Netlify builds of the Meshery Docs. Similar labels are assigned and used to trigger workflows or used as conditional flags in workflows to determine which workflows or which steps in a workflows to run. 
+
+## Release Channels
+
+Artifacts of the builds for Meshery and its components are published under two different release channels, so that improved controls may be provided to both Meshery users and Meshery developers. The two release channels are *edge* and *stable* release channels.
+
+Relative to stable releases, edge releases occur much more frequently. Edge releases are made with each merge to master, unless that merge to master is for a stable release. Stable releases are made with each merge to master when a GitHub release tag is also present in the workflow.
+
+### Stable Channel
+
+The following is an example of the release channels and the docker tags used to differentiate them. The latest tag will be applied only to images in the stable release channel. Here are two releases with two different images.
+
+**Latest Stable Image**
+
+- layer5/meshery:stable-latest
+- layer5/meshery:stable-v0.4.1
+- layer5/meshery:stable-324vdgb (sha)
+
+**Older Stable Image**
+
+- layer5/meshery:stable-v0.4.0
+- layer5/meshery:stable-289d02 (sha)
+
+Every docker image built receives either the edge tags or the stable tabs. Which set of image tags assigned is determined by whether a release tag is present or not. In other words, stable channel docker images get the “stable” tags only in the presence of a release tag (e.g. v0.4.1).
+
+### Edge Channel
+
+The edge release channel generally contains code less tested, less “baked”. The primary reason for "edge" is to allow contributors and advanced users to get at features sooner than later. Some features need testing that is best facilitated by letting users with tolerance and patience try them out.
+
+Stable and edge releases are both published to the same Docker Hub repository. Docker Hub repositories differentiate release channels by image tag. The following Docker images tagging convention is followed:
+
+**Latest Edge Image**
+
+- layer5/meshery:edge-latest
+- layer5/meshery:edge-289d02 (sha)
+
+**Older Edge Image**
+
+- layer5/meshery:edge-324vdgb (sha)
+
+
+### Switching Between Meshery Release Channels
+
+Users are empowered to switch between release channels at their leisure.
+
+#### Switching Release Channels Using mesheryctl
+
+Users can use mesheryctl to switch between release channels, e.g. `mesheryctl system channel [stable|edge]`.  Alternatively, users can manually switch between channels by updating the docker image tags in their meshery.yaml / Kubernetes manifest files. This command generates a meshery.yml (a docker-compose file) with release channel-appropriate tags for the different Docker container images.
+
+#### Viewing Release Channel and Version Information in Meshery UI
+
+Users are shown their Meshery deployment’s release channel subscription enient new setting in the Preferences area of the Meshery UI, so that people can alternatively use the UI to switch between channels if they like. Version numbers for Meshery adapters are also shown in the UI.
+
+## Release Cadence
+
+Minor releases of the Meshery project are release frequently (on a monthly basis on average) with patch releases made on-demand in-between those times. The project does not have long term releases that are sustained with bug fixes, yet. Bug fixes and patches will be released as needed on the latest release version.
+
+### Release Support
+
+General community support and commercial support from Layer5 is available. Separately, third parties and partners may offer longer-term support solutions.
+
+#### Pre v1.0
+
+Project focuses on functionality, quality and adoption, while retaining the flexibility for shifts in architecture. 
+
+#### Post v1.0
+
+Once a 1.0 release has been made, Around once a month or so, the project maintainers will take one of these daily builds and run it through a number of additional qualification tests and tag the build as a Stable release. Around once a quarter or so, the project maintainers take one of these Stable releases, run through a bunch more tests and tag the build as a Long Term Support (LTS) release. Finally, if we find something wrong with an LTS release, we issue patches.
+
+The different types (Daily, Stable, LTS) represent different product quality levels and different levels of support from the Meshery team. In this context, support means that we will produce patch releases for critical issues and offer technical assistance.