Skip to content

Commit

Permalink
Various tweaks to prep for docs transfer (#120)
Browse files Browse the repository at this point in the history
  • Loading branch information
ndintenfass committed Oct 23, 2018
1 parent d27aaa8 commit 73f11fe
Show file tree
Hide file tree
Showing 8 changed files with 70 additions and 75 deletions.
44 changes: 9 additions & 35 deletions README.md
@@ -1,39 +1,16 @@
# CircleCI Build Processing & 2.1 Configuration (Preview)
# CircleCI Orbs (Preview)

## Getting Started
You can turn on build processing at the bottom of the "Advanced" section of your project's settings. If you don't see this setting, it should appear in the next few days for you as we roll it out to more and more organizations.
## NOTE OCT 2018
We have moved the documentation for the new 2.1 features that are NOT about orbs specifically to our main docs site:
[https://circleci.com/docs/2.0/reusing-config/](https://circleci.com/docs/2.0/reusing-config/)

Note that if you are turning on build processing to use the new 2.1 configuration features you will want to read [the getting started section](https://github.com/CircleCI-Public/config-preview-sdk/blob/master/docs/getting-started.md) of the [docs on the new configuration features](https://github.com/CircleCI-Public/config-preview-sdk/blob/master/docs/README.md).
## Getting Started with Orbs Preview

You should also consider upgrading your `circleci` CLI to the new version. To do so, follow the instructions on the [circleci-cli repository](https://github.com/CircleCI-Public/circleci-cli/blob/master/README.md) for upgrading if you already have `circleci` installed, or installing from scratch if you haven't yet installed it.
Be sure to start with the [docs](/docs/README.md).

## Why turn on build processing?
We will soon turn on build processing for all CircleCI 2.0 builds, but we are providing this opt-in period to provide earlier access to some new features that build processing will enable and to help us make transitions to the new system more gradual and less risky.
Also, you'll need to be sure to use the new [Build Processing setting](https://circleci.com/docs/2.0/build-processing/) on your projects.

The new build processing feature enables the following:

1. Use of the [new configuration version 2.1 features](docs/whats-new.md)
2. Use of the new [API endpoint to trigger builds with workflows](https://circleci.com/docs/api/v1-reference/#new-project-build)
3. Auto-cancelation of redundant builds containing workflows.

A few highlight links to get going:

* [What's new in version 2.1 of configuration](docs/whats-new.md).
* [Docs on new version 2.1 configuration features](docs/README.md)
* [Design of the new build processing system](docs/design-approach.md)

## IMPORTANT: Preview Caveats
We are committed to achieving backwards compatibility in almost all cases, and we expect for most projects turning on build processing will have no effect on existing builds. Please let us know if you experience breaking builds that worked before you turned on build processing but broke once you turned it on.

We are considering the new build processing system in preview until we have solved (crossed off) all of the following use cases (all of which are under development or slated soon as of August 2018):

* ~Builds can use the 2.1 config features~
* ~Works with GitHub webhooks~
* ~New API endpoint to trigger builds, including running all workflows in the build~
* ~All jobs run inside a Workflow part 1: workflow auto-wrapping for jobs called `build`~
* ~Solve for full BitBucket support - API and webhooks~
* ~Auto-cancel redundant builds (including workflows)~
* ~Solve UI for Rerun of a job (rerun the workflow of the job)~
Finally, be sure to get the new [circleci CLI](https://github.com/CircleCI-Public/circleci-cli/) or upgrade if you already have `circleci` installed.

## IMPORTANT: 2.1 Configuration Caveats
With the introduction of build processing we are making available version 2.1 of build configuration.
Expand All @@ -51,10 +28,7 @@ There are two ways to stay up-to-date with changes we make to our new configurat
2. Get email updates from us about the configuration preview: [https://circle.ci/2HbCmKq](https://circle.ci/2HbCmKq).

## Giving Feedback
1. Come to [CircleCI Discuss](https://discuss.circleci.com/t/2-1-config-and-build-processing/24102) to post feedback.
2. Tweet @circleci with thoughts
3. Add [issues](https://github.com/CircleCI-Public/config-preview-sdk/issues) to this repo.
4. Vote or add to our [Ideas board](https://circleci.com/ideas/)
Add [issues](https://github.com/CircleCI-Public/config-preview-sdk/issues) to this repo.

## Documentation
The [docs](/docs/) in this repository cover the details of the new configuration features.
30 changes: 16 additions & 14 deletions docs/README.md
@@ -1,24 +1,26 @@
# CircleCI new configuration documentation

Documentation for authoring and using the new CircleCI Configuration and CircleCI Orbs. These docs are part of the [CircleCI config preview SDK](https://github.com/CircleCI-Public/config-preview-sdk).

Orbs are packages of reusable CircleCI configuration. They allow you to create encapsulated, parameterized commands and jobs that can be used across projects.
# CircleCI Orbs
Orbs are reusable packages of CircleCI configuration. They allow you to create encapsulated, parameterized commands, jobs, and executors that can be used across projects.

## Table of contents
1. [What's new in the config preview?](whats-new.md)
1. [Getting started with new config features](getting-started.md)
1. [Getting started with orbs](getting-started.md)
1. [Using existing orbs](using-orbs.md)
1. [Orb structure](structure.md)
1. [Using orbs](using-orbs.md)
1. [Authoring and Publishing Orbs](orbs-authoring.md)
1. [Testing orbs](orbs-testing.md)
1. [Authoring and publishing orbs](orbs-authoring.md)
1. [Inline orbs](inline-orbs.md)
1. [Testing orbs](orbs-testing.md)
1. [Orb deployment techniques](orbs-cd.md)
1. [Understanding the design of Orbs](design-approach.md)
1. [Design considerations for your orbs](design-considerations.md)
1. [Understanding the build configuration lifecycle](config-lifecycle.md)

## 2.1 configuration features available in orbs
_(TODO: vet that all of the content below, especially parameter types, is ported over to the 2.1 in main docs)_
1. [Commands](commands.md)
1. [Parameters](parameters.md)
1. [Jobs](jobs.md)
1. [Executors](executors.md)
1. [Conditional Steps](conditional-steps.md)
1. [Pre and Post Steps in Jobs](pre-and-post-steps.md)
1. [Understanding the design of Orbs](design-approach.md)
1. [Design considerations for your orbs](design-considerations.md)
1. [Configuration lifecycle](config-lifecycle.md)
1. [CD Techniques](orbs-cd.md)

## Potentially obsolete pages, left here for review
1. [What's new in the config preview?](whats-new.md)
14 changes: 7 additions & 7 deletions docs/design-approach.md
@@ -1,10 +1,10 @@
# Explaining the design approach to CircleCI configuration

## What problems are we trying to solve with the new build processing system?
1. **Better [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) support**
1. **Better [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) support**
The configuration in CircleCI 2.0 is designed to be highly deterministic. As a result, the syntax proved in many cases to be much more verbose and prone to too much boiler plate and repeated code blocks. CircleCI aims to provide better ways avoid repetitive build configuration.

2. **Code reuse across projects**
2. **Code reuse across projects**
Many customers state that they want better ways to share configuration across projects. This repo introduces Orbs, which are reusable packages of parameterizable CircleCI configuration elements that can be added to the Orb registry for use across projects.

3. **Easier path to common configuration**
Expand All @@ -25,8 +25,8 @@ Developers and customers also state that they want more off-the-shelf options to
> The core structure is driven by workflows that invoke and coordinate jobs that express a set of steps to run and the execution environment in which to run them. Our configuration code is intrinsically meta to your build process. We want to let you focus on what your code is going to do inside the runtime and get out of your way as much possible. Using a declarative syntax allows to give you an expressive set of primitives without making you learn much, if anything, about how our internal systems work or learn some new programming language.
## Configuration lifecycle
1. **Retrieval:** The build configuration you have written is retrieved from your git repository when CircleCI receives webhooks from your version control provider (eg: GitHub). This configuration can be passed in as part of an API call triggering a build.
2. **Processing:** The build configuration goes through a processing step prior to hitting the workflows and job execution layers of CircleCI. All invoked commands, jobs, and other structural elements, along with any parameters passed in this invocations, are resolved during this configuration processing step.
1. **Retrieval:** The build configuration you have written is retrieved from your git repository when CircleCI receives webhooks from your version control provider (eg: GitHub). This configuration can be passed in as part of an API call triggering a build.
2. **Processing:** The build configuration goes through a processing step prior to hitting the workflows and job execution layers of CircleCI. All invoked commands, jobs, and other structural elements, along with any parameters passed in this invocations, are resolved during this configuration processing step.

> One major implication is that configuration processing does not have access to your runtime environment or your environment variables and other sensitive information.
Expand All @@ -41,18 +41,18 @@ CircleCI chose to first build a YAML-based packaging mechanism because:
* Our configuration can be more easily self-documenting and statically processed by using a data format instead of a programming language.
1. **Transparency:** If you can execute an orb you can see the source of that orb. Orbs are less like your core software, and more like a piece of configuration you would otherwise copy and paste. Orbs are part of your build process, so you should be able to see what they do if you can execute them.
1. **First-class Metadata:** CircleCI allows a `description` key as part of the structure of orbs, so it's easier to generate documentation or read through the code without having extra-structural comments.
1. **Semantic Versioning and Locking:** CircleCI wants to allow orb authors to dynamically add features and fixes to orbs. At the same time, we want to prevent things from changing out from under orb users: fixing the orbs that users leverage so that users' configurations remain static unless the user specifies otherwise.
1. **Semantic Versioning and Locking:** CircleCI wants to allow orb authors to dynamically add features and fixes to orbs. At the same time, we want to prevent things from changing out from under orb users: fixing the orbs that users leverage so that users' configurations remain static unless the user specifies otherwise.
CircleCI choose to enforce a strict format for semantic versioning [semver](https://semver.org/) on all published revisions. When importing we allow an orb user to lock a specific revision or to assume the risk of breakage and to use the latest version (`volatile`).
1. **Register-time Dependency Resolution:** If an orb (`my-orb`) imports other orbs, we will resolve and lock those dependencies at the time that `my-orb` is added to the registry.
For instance, let's say that you publish version 1.2.0 of `my-orb`, and it contains:

```yaml
orbs:
foo-orb: somenamespace/some-orb:volatile`
foo-orb: somenamespace/some-orb:volatile`
```

At the time you register 1.2.0 of `my-orb` the latest version of `foo-orb` will be flattened into `my-orb` version 1.2.0. If a new version of `foo-orb` is published it won't get incorporated into `my-orb` until you publish a new version. **NOTE:** _We recommend selecting the fully qualified version of the orbs that you import to ensure deterministic behavior._

1. **One registry per CircleCI installation:** Orbs are specific to running CircleCI builds, so we decided to avoid the complexities and security surface area of having arbitrary registries. All orbs used in a CircleCI build must be in the registry of the installation on which your build runs.
1. **One registry per CircleCI installation:** Orbs are specific to running CircleCI builds, so we decided to avoid the complexities and security surface area of having arbitrary registries. All orbs used in a CircleCI build must be in the registry of the installation on which your build runs.
1. **All orbs live in a namespace:** All orbs live in exactly one namespace. There is no "empty" namespace, nor are there reserved special defaults like "_" for CircleCI or "official" orbs. We are likely to introduce a Certified Orbs program or similar in the future, but that will offer first-class metadata on the orb, explicitly defining inclusion rather than relying on implied in a special namespace.
1. **Revisions are immutable:** To prevent mysterious changes cropping up in the builds of those who use orbs, we do not allow changes to an orb revision once it has gone live.
4 changes: 2 additions & 2 deletions docs/design-considerations.md
Expand Up @@ -14,11 +14,11 @@ If you are providing commands, try to provide one or more executors in which the
### Name things concisely
Remember that use of your commands and jobs is always contextual to your orb, so you can use general names like "run-tests" in most cases.

### Required vs. Optional parameters
### Required vs. optional parameters
Try to provide sound default values of parameters whenever possible.

### Job-only orbs are inflexible
While sometimes appropriate, it can be frustrating for users to not be able to use the commands in their own jobs. Pre and post steps when invoking jobs are a work-around for users on this front.

### Parameter `steps` are powerful
### Parameter `steps` are powerful for orb users
Wrapping steps provided by the user allows you to encapsulate and sugar things like caching strategies and other more complex tasks, providing a lot of value to users. Read more about this [here](parameters.md#steps).
2 changes: 1 addition & 1 deletion docs/getting-started.md
@@ -1,4 +1,4 @@
# Getting started
# Getting started with orbs

## 1. Ensure that build processing is enabled
As of September 2018 any new projects already have this enabled.
Expand Down
20 changes: 20 additions & 0 deletions docs/orbs-authoring.md
Expand Up @@ -7,6 +7,26 @@ Orbs can be authored [inline](inline-orbs.md) in your `config.yml` file or autho
All published orbs can be read and used by anyone. They are _not_ limited to just the members of your organization.
In general, we strongly recommend that you do not put secrets or other sensitive variables into your configuration. Instead, use contexts or project environment variables and reference the names of those environment variables in your orbs.

## Quick start
The high-level steps one will take to publish their first orb:

1. Claim a namespace (assuming you don't yet have one), eg:
> `circleci namespace create sandbox github CircleCI-Public`
2. Create the orb inside your namespace, eg:
> `circleci orb create sandbox/hello-world`
3. Create the content of your orb in a file. You will generally do this in your code editor in a git repo made for your orb, but for the sake of example was assume a file in `/temp/orb.yml` could be made with a bare-bones orb like:
> `echo '{version: "2.1", description: "a sample orb"}' > /usr/tmp/orb.yml`
4. Validate that your code is a valid orb using the CLI. For example, using the path above you could use:
> `circleci orb validate /tmp/orb.yml`
5. Publish a dev version of your orb. Assuming the above orb that would look like:
> `circleci orb publish /tmp/orb.yml sandbox/hello-world@dev:first`
6. Once you are ready to push your orb to production, you can publish it manually using `circleci orb publish` or promote it directly from the dev version. In the case of the above, assuming you wanted to increment the new dev version to become 0.0.1 you can use:
> `circleci orb publish promote sandbox/hello-world@dev:first patch`
7. Your orb is now published in an immutable form as a production version and can be used safely in builds. You can pull the source of your orb using:
> `circleci orb source sandbox/hello-world@0.0.1`
Below are more details on orb publishing.

## Prerequisites
To start orb publishing, you will need to opt-in to the new 3rd Party Software terms and turn on orb publishing for your organization. Only an organization admin can do this from the [organization Settings page](https://circleci.com/docs/2.0/settings/#organization-settings-page). On the "Security" tab you will find the form for opting in.

Expand Down
8 changes: 4 additions & 4 deletions docs/structure.md
Expand Up @@ -5,13 +5,13 @@ Orbs are authored with similar YAML syntax to the base CircleCI build configurat
## Anatomy of an orb
Orbs are composed of one or more of the following elements, each of which represents a type of invocable element in CircleCI project configuration:

* `description` (a string)
* `version: 2.1`
* `description: (string value)`
* [commands](commands.md)
* [jobs](jobs.md)
* [executors](executors.md)
* [orbs](inline-orbs.md)

Additionally, [usage examples](usage-examples.md) may also be provided for documentation purposes.
* [examples](usage-examples.md)

An example orb:
```yaml
Expand All @@ -28,7 +28,7 @@ jobs:
- echo
```

Note that the example Orb shown above has a single command name `echo` and a single job named `hello`.
Note that the example orb shown above has a single command name `echo` and a single job named `hello`.

### Naming

Expand Down

0 comments on commit 73f11fe

Please sign in to comment.