Skip to content

Commit

Permalink
fix images
Browse files Browse the repository at this point in the history
  • Loading branch information
@TracyRagan
TracyRagan committed Oct 25, 2022
1 parent 9ce34dc commit b104efe
Show file tree
Hide file tree
Showing 6 changed files with 207 additions and 375 deletions.
154 changes: 1 addition & 153 deletions content/en/guides/userguide/First Steps/2 Intro to Set Up.md
View file
Expand Up @@ -5,157 +5,5 @@ weight: 1
description: >
Get Started by Adding Ortelius to Your Pipeline
---

## Ortelius' Pipeline Automation

In order to continuously gather pipeline intelligence, Ortelius must become part of your pipeline. Ortelius integrates into your CI/CD process using the Ortelius Command Line (CLI). The Ortelius CLI gathers supply chain data based on a single pipeline workflow at the build and deploy steps. The build step gathers Swagger, SBOM, Readme, licenses, Git data, Docker image, and other build output. The deploy step records when a release occurs, what was sent and where the objects were sent to.

### Ortelius Data Gathering using .toml

The Ortelius CLI reads from a .toml file. The .toml file contains non-derived information for each artifact that you create at your build step. In Ortelius, an artifact is referred to as a _Component_. A _Component_ is a Container, DB Object, or file object (.jar, Lamda Function, Apex file, etc.). The .toml file will provide the 'non-derived' data for the _Component_ your are tracking in Ortelius which includes the _Component_ name, owner, _Component type_, and owner contact details. The Ortelius CLI will read the .toml file from the Git Repository associated to your pipeline. If you are using a Mono Repository for your entire codebase, you will need a separate Component.toml file for each _Component_, managed in sub-directories.

In a cloud-native, microservice architecture there are many, if not hundreds, of _Components_. Organizing your _Components_ within Ortelius is done in two ways. They are grouped based on a subject _Domain_ and assigned to a logical _Application_. Not all _Components_ need to be assigned to an _Application_, but they should be stored in a subject matter _Domain_ so they can be easily found and reused.

A logical _Application_ is a collection of _Components_ that make up a complete software systems consumed by an end user. Applications are composed of shared _Components_ and _Application_ specific _Components_, and are a logical representation of what _Components_ need to be deployed in order for the software system to run.

>Note: Once created, your .toml file does not need to be updated unless the non-derived information changes, or you want to reorganize to which Applications or _Domains_ the Component has been assigned. For example, a Component has been reassigned to a new owner and new team represented by a _Domain_ or _Application_.
Perform the following steps to add your _Components_ using the .toml file:

#### Step 1 - Define Your Ortelius Pipeline Variables

The following variables should be set at the beginning of your Pipeline.

| Variable | Value | Description |
| ------- | ----- | ----------- |
| DHURL | URL to Ortelius Login | The URL used to access Ortelius. |
| DHUSER | UserID | The ID used to log into Ortelius |
| DHPASS | password | The password used to log into Ortelius. This can encrypted based on the CI/CD solution. |
| DOCKERREPO | Name of your Docker Repository | For Components that are Docker Images. Not needed for non-docker objects. |
| IMAGE_TAG | Tag for the Docker Image if used | For Components that are Docker Images. Not needed for non-docker objects. |

Example

```bash
export DHURL=https://deployhub.example.com
export DHUSER=Stella99
export DHPASS=chasinghorses
export DOCKERREPO=quay.io/ortelius/hello-world
export IMAGE_TAG=1.0.0
```

#### Step 2 - Create your Component.toml file

Cut and paste the following into a component.toml file, update 'your' information, and commit/push it to your Git Repository.

```toml
# Application Name and Version - not required. If not used the Component will not be associated to an Application
Application = "GLOBAL."your Application Name"
Application_Version = "your Application Version"

# Define Component Name, Variant and Version - required
Name = "GLOBAL.your Component Name"
Variant = "${GIT_BRANCH}"
Version = "vyour Component Version.${BUILD_NUM}-g${SHORT_SHA}"

# Key/Values to associate to the Component Version
[Attributes]
DockerBuildDate = "${BLDDATE}"
DockerRepo = "${DOCKERREPO}"
DockerSha = "${DIGEST}"
DockerTag = "${IMAGE_TAG}"
SlackChannel = "your slack channel"
ServiceOwner = "your Component Developer Name"
ServiceOwnerEmail = "your Component Owner Email"
```

For example:

```toml
# Application Name and Version
Application = "GLOBAL.Welcome"
Application_Version = "1.0.0"

# Define Component Name, Variant and Version
Name = "GLOBAL.hello-world"
Variant = "${GIT_BRANCH}"
Version = "v1.0.0.${BUILD_NUM}-g${SHORT_SHA}"


# Key/Values to associate to the Component Version
[Attributes]
DockerBuildDate = "${BLDDATE}"
DockerRepo = "${DOCKERREPO}"
DockerSha = "${DIGEST}"
DockerTag = "${IMAGE_TAG}"
SlackChannel = "OrteliusSlack"
ServiceOwner= "Jane Coder"
ServiceOwnerEmail = "JC@ortelius.io"

```

>Note: For SaaS users, you will have a second high-level qualifier that was created as part of your sign-up. This secibd high-level qualifier must be used as the start of your Application Name and Component Name. For example: _GLOBAL.Santa Fe Software.Online Store_.
#### Step 3 - Add a step in your pipeline to run Syft if you are not generating SBOMS (Optional)
Ortelius can consume any SPDX and CycloneDX formatted SBOM. If you are already generating SBOMs, you will pass the name of the SBOM results to Ortelius is step 4 below. If you are not generating SBOMs as part of your pipeline process, you will need to add SBOM generation to collect the lower dependency data. Following is how to add Syft to your workflow to include the collection of SBOM data.

[Syft SBOM tool](https://github.com/anchore/syft) will generate Software Bill of Material Reports for popular coding languages and package managers, including Docker images.

The following code example scans a Docker Image to generate the SBOM. See [Syft Options](https://github.com/anchore/syft#supported-sources) to scan other objects and coding languages.

```bash
# install Syft
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b $PWD

# create the SBOM
./syft packages $DOCKERREPO:$IMAGE_TAG --scope all-layers -o cyclonedx-json > cyclonedx.json

# display the SBOM
cat cyclonedx.json
```

#### Step 4 - Run the Ortelius CLI to add Your Component and Create an Application

>Note: To complete the process you will need to install the Ortelius CLI where your CI/CD server is running. Refer to the [Ortelius GitHub CLI Documentation](https://github.com/ortelius/cli/blob/main/doc/dh.md) for installation instructions.
Execute the following calls to the Ortelius CLI as part of your workflow. It should be called after the build and SBOM generation:

With CycloneDX SBOM

```bash
dh updatecomp --rsp component.toml --deppkg "cyclonedx@name of your SBOM file"

Example:
dh updatecomp --rsp component.toml --deppkg "cyclonedx@cyclonedx.json"
```
With SPDX SBOM

```bash
dh updatecomp --rsp component.toml --deppkg "spdx@name of your SBOM file. "

Example:
dh updatecomp --rsp component.toml --deppkg "spdx@spdx.json"
```
Without SBOM

```bash
dh updatecomp --rsp component.toml
```

##Results

**Component Details**
![Component Details](/static/helloworld-comp-details.png)

**Component SBOM**
![Component SBOM](/static/helloworld-comp-sbom.png)

**Component License**
![Component License](/static/helloworld-comp-license.png)

**Component Key Values**
![Component Key Values](/static/helloworld-comp-kv.png)

**Component Readme**
![Component License](/static/helloworld-comp-readme.png)
{{% include "/guides/userguide/reusable/Ortelius Pipeline Automation.md/" %}}

221 changes: 1 addition & 220 deletions content/en/guides/userguide/integrations/CI-CD_Integrations.md
View file
Expand Up @@ -5,224 +5,5 @@ weight: 49
description: >
Integrating Ortelius into your CI/CD process.
---
{{% include "/guides/userguide/reusable/Ortelius Pipeline Automation.md/" %}}


## Ortelius' Pipeline Automation

Ortelius must be automated via your pipeline process to continuously gather supply chain and deployment metadata. The Ortelius' CLI gathers supply chain data based on a single pipeline workflow at the build and deploy step. The build step gathers Swagger, SBOM, Readme, licenses, Git data, Docker image, and other build output. The deploy step records when a release occurs, what was sent and where the objects were sent to.

To prime the process, Ortelius needs a .toml file that contains data that cannot be derived from the pipeline workflow. Non-derived data must be added to the .toml file to begin the process and refers to each objects Git Repository. You will need one Component .toml file for each object. An object can be a a Container, DB object or file (.jar file, Lamda Function, Apex, etc.). Objects are referred to as 'Components' in Ortelius.

creates new _Component Versions_ and _Application Versions_, and deploys independent _Components_ into specified _Environments_. "Build" data is saved as configuration information and then associated to the _Component_ and _Application_. Ortelius' uses a standard CLI to integrate into CI/CD Tools.

### Using Ortelius CLI with CI/CD Tools

Below are the steps needed to capture Service Catalog data from your CI/CD Tool. These steps can be run outside of your CI/CD Tool to manually simulate your Pipeline process.

#### TL;DR;

[Jenkinsfile Example](/Jenkinsfile)

#### Steps

##### Install Ortelius CLI

[Install Python 3.8](https://www.python.org/downloads/)

`pip install deployhub`

[CLI GitHub Repo](https://github.com/ortelius/compupdate) with [additional documentation](https://github.com/ortelius/compupdate/blob/main/doc/dh.md).

##### 1. Setup a repo to work with

`git clone https://github.com/dstar55/docker-hello-world-spring-boot`

##### 2. Add Ortelius Config File

Create a component.toml in the root of your repo.

_**Note:** This is done once per repo._

```toml
# Define Component Name, Variant and Version
Name = "GLOBAL.${DHORG}.${DHPROJECT}.hello-world"
Variant = "${GIT_BRANCH}"
Version = "v1.0.0.${BUILD_NUM}-g${SHORT_SHA}"

# Export the derived IMAGE_TAG to the CI/CD Tool
[Export]
IMAGE_TAG = "${Variant}-${Version}"

# Key/Values to associate to the Component Version
[Attributes]
DockerBuildDate = "${BLDDATE}"
DockerRepo = "${DOCKERREPO}"
DockerSha = "${DIGEST}"
DockerTag = "${IMAGE_TAG}"
```

##### 3. Setup Environment Variables

These variable normally come from you CI tools such as Jenkins, but we need to set them up manually to simulate a CI Tool.

_**Note:**_ These variables can be set at the beginning of your Pipeline or at the Global Environment Variable Setting for your CI/CD tool.

```bash
export DHUSER=<myuserid>
export DHPASS=<mypassword>
export DHORG=<myorg>
export DHPROJECT=<myproject>
export DOCKERREPO=quay.io/ortelius/hello-world
export DHURL=https://console.deployhub.com
export BUILD_URL=http://jenkins.myproject.org
export BUILD_NUM=101
```

##### 4. Gather data from Git

Run the `dh envscript` which runs git commands to grab data about your repo and persist that data into a shell script as environment variables. `source` the shell script in subsequent steps to expose the git data. This process is how you pass data between steps in most CI/CD tools.

_**Note:**_ This is done after your have cloned your repo in your Pipeline.

```bash
cd docker-hello-world-spring-boot
dh envscript --envvars component.toml --envvars_sh dhenv.sh
```

##### 5. Perform a docker build and push

```bash
source dhenv.sh
docker build --tag $DOCKERREPO:$IMAGE_TAG .
docker push $DOCKERREPO:$IMAGE_TAG

# This line determines the docker digest for the image
echo export DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' $DOCKERREPO:$IMAGE_TAG | cut -d: -f2 | cut -c-12) >> dhenv.sh
```

_**Note:**_ IMAGE_TAG environment variable is created when running `dh envscript` in the previous step.

##### 6. Create an SBOM of image using Syft

```bash
source dhenv.sh
# install Syft
curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b $PWD

# create the SBOM
./syft packages $DOCKERREPO:$IMAGE_TAG --scope all-layers -o cyclonedx-json > cyclonedx.json

# display the SBOM
cat cyclonedx.json
```

##### 7. Create your component in Ortelius/DeployHub Team SaaS

```bash
source dhenv.sh
dh updatecomp --rsp component.toml --deppkg "cyclonedx@cyclonedx.json"
```

##### Verify New Component

Login to https://console/deployhub.com and navigate to the _Components_ List, then double click on the _Component_ and verify the details that were captured by the CLI.

#### Saved Build Data

Build data is saved to the _Component Version_ and displayed on the _Component Version_ Dashboard. In some cases, Ortelius may need to save custom data. This can be done using the _Component_ Attributes and saved as a key/value pair. Custom data displays in the Attributes Section as Attributes to the _Component Version_ .

The _Component_ Type determines what data is saved. See [Defining Components](/guides/userguide/publishing-components/2-define-components/) for a complete listing of the Detail data saved for each different _Component_ Type.

### Continuous Configuration Management and Versioning

The continuous configuration management and versioning process creates new _Application Versions_ based on build update detection. When an update is detected, Ortelius identifies a new _Component Version_ in the continuous delivery workflow and triggers a dependency relationship map update for all consuming _Applications_ which creates new _Application Versions_.

This continuous configuration step is particularly important in a microservice implementation. Every time a new microservice is pushed across the pipeline, it impacts the configuration of the "logical" _Application_ that consumes it. A new microservice _Component_ creates a new _Application Version_. This is how Ortelius presents the _Application Version_ Dashboard data showing the relationship and difference maps for each cluster to which the microservice _Component_ is deployed.

### _Component_ Versioning Schema

Ortelius versions _Components_ differently than it does new _Application Versions_. _Components_ are tied to Git commits, _Applications_ are not.

The following are the parts of the versioning schema:

| Schema | Description|
|---|---|
|**Base name** | is the static part of the _Component_ Name. For example: email-service.|
| **Variant** | is a high level place holder for the versions are created within. The Variant can be aligned with a feature or branch. For example: ssl-update. Variant is not required.|
|**Version**| is the schematic version number or schematic + Git commit. For example: v1.5.1.0 or v1.5.1-g3d55a2 |

By default, Ortelius uses an advanced format for the versioning schema. It will automatically increment the version number based on last version found. The advanced format is used since Ortelius has the information from Git to provide a complete version schema. The advanced formatting is:

~~~bash
<base name>;<variant>;v<version>-g<git commit>
~~~

The version segment is broken down further into:

~~~bash
v<schematic number>-<number of commits>
~~~

The number of commits provides an auto-increment of the last part of the schema number.

An example of the full name is:

~~~bash
email-service;ssl-update;v1_5_1_145-g3d55a2
~~~

Below is a breakdown of the individual parts of the versioning schema:

| Part | Explanation |
| --- | --- |
| **email-service** | Base Name that you supply to the Ortelius Plugin for the _Component_. |
| **semi colon** | Separator. |
| **ssl-update** | Variant that you supply to the Ortelius Plugin for the _Component_. |
| **semi colon** | Separator. |
| **v** | Indicates start of the version. |
| **1_5_1**| Version that you supply to the Ortelius Plugin for the _Component_. |
| **145** | Number of Git Commits for the associated repository. Used for auto incrementing. |
| **-g** | Separator indicating that the Git commit is next. |
| **3d55a2** | Git command SHA for the commit that triggered the CD process. |

Note: The Variant is optional. That part of the _Component_ name will be left out if the Variant passed to Ortelius by the CD workflow is blank.

### _Application_ Versioning Schema

The Ortelius continuous delivery integration will automatically maintain the _Application Version_ if the _Application_ Name and a starting version identifer are provided. When provided, a new _Application Version_ will be created using the previous version identifier as a starting point. An _Application Base_ version must exist in for Ortelius to automatically create a new _Application Version_ when a new _Component Version_ is created. The base version gives Ortelius the starting point to perform automatic _Application_ versioning.

Ortelius uses a similar naming convention for _Application Versions_ as it does for _Component Versions_:

| Part | Explanation |
| --- | --- |
|**Base name**| The static part of the _Application_ Name. For example: webstore.|
|**Variant** | A high level place holder for that versions are created within. The Variant can be aligned with a feature or branch. For example: 50percent-sale. Variant is not required.|
|**Version**| The schematic version number. For example, 1_2_10.|

Ortelius uses the base name; schematic version number ; version number as the default format for the versioning schema. The formatting is:

~~~bash
<base name>;<variant>;<schematic version number>;<version number> if a Variant is used.
~~~

Deploy will automatically increment the version number based on last version found. It will create a new _Application Version_ based on the last _Application Version_ and replace the old _Component Version_ with the new _Component Version_. The new _Component Version_ will be added if an old verion was not found.

You can use your CI/CD process to include variance in your versioning number (base name, variant, version). See [Component Versioning Schema](/guides/userguide/integrations/ci-cd_integrations/#_component_-versioning-schema).

### Deploying the _Application Version_

Once the versioning is completed, Ortelius is called to perform the deployment. The _Application Version_ and _Environment_ are required for the deployment. The deployment number will be returned to both Ortelius and the continuous delivery solution to be displayed in the output.

## Life Cycle Tasks and Continuous Delivery Workflows

You can incorporate Ortelius' Life Cycle _Subdomain_ Tasks as follows:
### Move
Ortelius will move the _Application_ from one Life Cycle _Subdomain_ to another using the Ortelius _Move Task_. This enables the _Application_ to move through the same pipeline steps that the CD process is using.
### Approve
If you are using DeployHub Pro, you can incorporate Approvals into your process. Ortelius will perform an Approval of an _Application_ Move. This _Approve_ feature is used to reflect an approval that was done in the CD Pipeline for an _Application's_ audit trail.

For documentation on incorporating Tasks, refer to your specific CD tool documentation as listed above.

0 comments on commit b104efe

Please sign in to comment.