Skip to content
This repository has been archived by the owner on Nov 2, 2023. It is now read-only.

Commit

Permalink
Merge branch 'prepare-release' into release
Browse files Browse the repository at this point in the history
  • Loading branch information
s4sdkJenkinsBot committed Nov 20, 2018
2 parents bfc8b6c + d03df5d commit 9ca4365
Show file tree
Hide file tree
Showing 6 changed files with 114 additions and 48 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
@@ -1,6 +1,6 @@
# SAP S/4HANA Cloud SDK Pipeline

![Picture of Pipeline](images/s4sdk-pipeline.png)
![Visualisation of SAP S/4HANA Cloud SDK Pipeline](images/s4sdk-pipeline.gif)

## Description

Expand Down
30 changes: 26 additions & 4 deletions configuration.md
View file
Expand Up @@ -56,6 +56,7 @@ To adjust the SAP S/4HANA Cloud SDK Pipeline to your project's needs, it can be
| `productiveBranch` | | `master` | The name of your default branch. This branch will be used for deploying your application. Other branches will skip deployment. |
| `projectName` | | `artifactID` from pom | Name of the project |
| `collectTelemetryData` | | `true` | No personal data is collected. For details, consult the [analytics documentation](doc/operations/analytics.md). |
| `unsafeMode` | | `false` | Enable unsafe mode to skip checking environment variables for insecure elements. Only use this for demo purposes, **never for productive usage**. |

#### automaticVersioning
The pipeline can be configured to store release candidates in a nexus repository after they passed all stages successfully. By turning on automatic versioning, one can avoid that multiple builds of a continuously delivered application lead to version collisions in nexus. When activated, the pipeline will assign unique maven versions for each release candidate. If you are not building a continuously delivered application, you will typically disable automatic versioning.
Expand Down Expand Up @@ -166,11 +167,23 @@ Running end-to-end tests can be restricted to the `productiveBranch` with the op
This might be useful when the end-to-end tests slow down development, and build failure on the `productiveBranch` is acceptable.
By default this feature is turned off.

Additional parameters can be passed for each end-to-end test deployment by specifying _optional_ `parameters` for an application URL.
These parameters are appended to the npm command during execution.
This could be used for example to split the entire end-to-end test scenario into multiple sub-scenarios and running these sub-scenarios on different deployments.
For example, when using nightwatch, these scenarios can be defined via annotations in the test descriptions and can be called with the `--tag` parameter as shown in the example below.

Example:
```yaml
endToEndTests:
enableZeroDowntimeDeployment: true
onlyRunInProductiveBranch: true
appUrls:
- url: <application url>
credentialId: e2e-test-user-cf
parameters: '--tag scenario1'
- url: <application url 2>
credentialId: e2e-test-user-cf
parameters: '--tag scenario2 --tag scenario3'
```

#### performanceTests
Expand Down Expand Up @@ -279,7 +292,7 @@ For `neoTargets` the following properties can be defined:
| `account` | X | | Identifier of the subaccount|
| `application` | X | | Name of the application in your account |
| `credentialsId` | X | | ID of the credentials stored in Jenkins and used to deploy to SAP Cloud Platform |
| `ev` | | | List of strings representing one environment variable assignment in the form of 'KEY=VALUE'|
| `environment` | | | Map of environment variables in the form of KEY: VALUE|
| `vmArguments` | | | String of VM arguments passed to the JVM|
| `runtime` | X | | Name of the runtime: neo-java-web, neо-javaee6-wp, neо-javaee7-wp. See the [runtime](https://help.sap.com/viewer/65de2977205c403bbc107264b8eccf4b/Cloud/en-US/937db4fa204c456f9b7820f83bc87118.html) for more information.|
| `runtimeVersion` | X | | Version of the runtime. See [runtime-version](https://help.sap.com/viewer/65de2977205c403bbc107264b8eccf4b/Cloud/en-US/937db4fa204c456f9b7820f83bc87118.html) for more information.|
Expand All @@ -294,8 +307,8 @@ productionDeployment:
account: '<Sub account>'
application: 'exampleapp'
credentialsId: 'NEO-DEPLOY-PROD'
ev:
- 'STAGE=Production'
environment:
STAGE: Production
vmArguments: '-Dargument1=value1 -Dargument2=value2'
runtime: 'neo-javaee6-wp'
runtimeVersion: '2'
Expand Down Expand Up @@ -342,12 +355,21 @@ nodeSecurityScan:
```

#### whitesourceScan
Configure credentials for [WhiteSource](https://www.whitesourcesoftware.com/) scans.
Configure credentials for [WhiteSource](https://www.whitesourcesoftware.com/) scans. The minimum required maven WhiteSource plugin version is `18.6.2`, ensure this in the plugins section of the project `pom.xml` file.

```xml
<plugin>
<groupId>org.whitesource</groupId>
<artifactId>whitesource-maven-plugin</artifactId>
<version>18.6.2</version>
</plugin>
```

| Property | Mandatory | Default Value | Description |
| --- | --- | --- | --- |
| `product` | X | | Name of your product in WhiteSource. |
| `credentialsId` | X | | Unique identifier of the `Secret Text` on Jenkins server that stores your organization(API Token) of WhiteSource. |
| `whitesourceUserTokenCredentialsId` | | | Unique identifier of the `Secret Text` on Jenkins server that stores WhiteSource `userKey` of a user. This is required only if the administrator of the WhiteSource service has enabled additional access level control. More details can be found [here](https://whitesource.atlassian.net/wiki/spaces/WD/pages/417529857/User+Level+Access+Control+in+Integrations+and+APIs). |

Please note that you can not have a `whitesource.config.json` in your project, since the Pipeline generates one from this configuration.

Expand Down
115 changes: 75 additions & 40 deletions doc/operations/operations-guide.md
View file
Expand Up @@ -2,14 +2,56 @@

This guide describes life-cycle management of the Cx Server for Continuous Integration and Delivery. The server is controlled with the `cx-server` script.

### Introduction

#### Introduction
The `cx-server` directory is included in projects which are created by using the SAP S/4HANA Cloud SDK Maven Archetypes.
It contains a life-cycle management utility script `cx-server` and a configuration file `server.cfg`.

For convenient usage of the script, a [completion script](https://raw.githubusercontent.com/SAP/cloud-s4-sdk-pipeline-docker/master/s4sdk-jenkins-master/cx-server/cx-server-completion.bash) for `cx-server` is provided. Source it in your shell, or refer to the documentation of your operating system for information on how to install this script system wide.
For the convenient usage of the script, a [completion script](https://raw.githubusercontent.com/SAP/cloud-s4-sdk-pipeline-docker/master/s4sdk-jenkins-master/cx-server/cx-server-completion.bash) for `cx-server` is provided.
Source it in your shell, or refer to the documentation of your operating system for information on how to install this script system wide.

#### System requirement

##### Productive usage
In order to setup the cx-server for the productive purpose, we recommend the minimum hardware and software requirements as mentioned below.
As of now, we do not support the productive setup on a Windows operating system.

| Property | Recommendation |
| --- | --- |
| Operating System | `Ubuntu 16.04.4 LTS` |
| Docker | `18.06.1-ce` |
| Memory | `4GB` reserved for docker |

#### start
##### Development usage
The `cx-server` can also run on a Windows or MacOs. But, only for the development purposes.
In order to run the `cx-server` on Windows, you need to share the `C` drive with a docker demon as explained [here](https://docs.docker.com/docker-for-windows/#shared-drives).
Set the docker memory to at least 4GB, you can [configure](https://docs.docker.com/docker-for-windows/#advanced) this under the `Advanced` settings tab.

#### Configuring the `cx-server`
The `cx-server` can be customized to fit your use case. The `server.cfg` file contains the configuration details of your `cx-server`.

| Property | Mandatory | Default Value | Description |
| --- | --- | --- | --- |
| `docker_image` | X | `s4sdk/jenkins-master:latest`| Jenkins docker image name with the version to be used|
| `docker_registry` | | Default docker registry used by the docker demon on the host machine | Docker registry to be used to pull docker images from|
|`jenkins_home`| X|`jenkins_home_volume`| The volume to be used as a `jenkins_home`. Please ensure, this volume is accessible by the user with id `1000` |
|`http_port`| X|`80`||
|`http_proxy`| | | Effective value of 'http_proxy' environment variables|
|`https_proxy`| | | Effective value of 'https_proxy' environment variables|
|`no_proxy`| | | Whitelisting of hosts from the proxy. It will be appended to any previous definition of `no_proxy`|
|`backup_directory`| | `(pwd)/backup` | Directory where the backup of the jenkins home directory contents are stored|
|`backup_file_name`| |`jenkins_home_YYYY-MM-DDThhmmUTC.tar.gz`| Name of the backup file to be created|
|`x_java_opts`| || JAVA_OPT that needs to be passed to the Jenkins container|
|`cache_enabled`| |`true` | Flag to enable or disable the caching mechanism for `npm` and `maven` dependencies|
|`mvn_repository_url`| | Maven central repository URL| It will be used if you need to configure a custom maven repository|
|`npm_registry_url`| | Central NPM registry| It will be used if you need to configure a custom npm registry|
|`x_nexus_java_opts`| | | You can configure the JAVA_OPTS of the download cache server using this option|


>**Note:** Please note that if you are using proxy settings and configured them in `server.cfg`, you also have to configure them in the Jenkins UI under `advanced` section of `Plugin Manager`(<Jenkins_URL>/pluginManager/advanced).
The variables set in `server.cfg` will be written into the environment of the corresponding containers and will be only used by the containers that are spawned by the Jenkins during pipeline execution.

#### Life-cycle of `cx-server`
##### start
You can start the Jenkins server by launching the `start` command.

```bash
Expand All @@ -19,51 +61,32 @@ You can start the Jenkins server by launching the `start` command.
When launched, it checks if the Docker container named `s4sdk-jenkins-master` already exists.
If yes, it restarts the stopped container. Otherwise, it spawns a new Docker container based on the configuration in `server.cfg`.

Example:

```bash
# Address of the used docker registry
docker_registry=registry.hub.docker.com

# Name of the used docker image
docker_image="s4sdk/jenkins-master:latest"
```

#### stop
##### stop
The Cx Server can be stopped with the `stop` command.
```bash
./cx-server stop
```
This stops the Jenkins Docker container if it is running. A subsequent `start` command restores the container.

#### remove
##### remove
This command removes the Jenkins container from the host if it is not running.

```bash
./cx-server remove
```

#### backup
The `jenkins_home` contains the state of the Jenkins which includes important details such as settings, Jenkins workspace and job details.
##### backup
The `jenkins_home` contains the state of the Jenkins which includes important details such as settings, Jenkins workspace, and job details.
Considering the importance of it, taking regular backup of the `jenkins_home` is **highly recommended**.

```bash
./cx-server backup
```
This command creates a backup file and stores it on a host machine inside a directory named `backup`. In order to store the backup on external storage, you can customize the location and name of the backup file in the `server.cfg`.

Example:
```bash
# Absolute path to a directory where the backup of the jenkins_volume stored
backup_directory="$(pwd)/backup"

# Name of the backup file
backup_file_name="jenkins_home_$(date -u +%Y-%m-%dT%H%M%Z).tar.gz"
```

> **Note:** Administrator of the Jenkins must ensure that the backup is stored in a safe storage.
#### restore
##### restore
In an event of a server crash, the state of the Jenkins can be restored to a previously saved state if there is a backup file available. You need to execute the `restore` command along with the absolute path to the backup file that you want to restore the state to.

Example:
Expand All @@ -75,13 +98,13 @@ Example:
> **Warning:** In order to restore the Jenkins home directory, this command stops the Jenkins server first and **delete the content of the Jenkins home directory**.
> After the completion of the restore operation, it starts the Jenkins server upon user confirmation.
#### update script
##### update script
The `cx-server` script can be updated via the `update script` command, if a new version is available.
```bash
./cx-server update script
```

#### update image
##### update image
By default, the Cx Server image defined by `docker_image` in `server.cfg` always points to the newest released version.
In productive environments, you will however likely want to fix the Cx Server image to a specific version.
By defining `docker_image` with a version tag (e.g. `docker_image=s4sdk/jenkins-master:v3`), you avoid unintended updates as a side-effect of restarting the Continuous Delivery server.
Expand All @@ -102,15 +125,7 @@ Note: The command only works if you use the default image from Docker Hub.
#### Caching mechanism
The `cx-server` provides the local cache for maven and node dependencies. This is enabled by default. A Docker image of [Sonatype Nexus OSS 3.x](https://www.sonatype.com/download-oss-sonatype) is used for this.

By default the caching service makes use of maven central and npm registry are used for downloading the dependencies. This can be customized in `server.cfg` as shown below.

```bash
### maven proxy target (default is maven central)
mvn_repository_url="https://your-local-maven-repo.corp/maven2/"

### npm proxy target (default is central npm registy)
npm_registry_url="https://your-local-npm-registry.corp/"
```
By default the caching service makes use of maven central and npm registries for downloading the dependencies. This can be customized in `server.cfg`.

In a distributed build environment, the Nexus server is started on each agent.
The agent initializer script `launch-s4sdk-agent.sh` takes care of the automatic start of the caching server.
Expand All @@ -132,7 +147,15 @@ If you prefer to use different caching mechanism or not using any, you can disab
cache_enabled=false
```

### Disk space cleanup
#### Plugins
All the plugins that are required to run the SAP S/4HANA Cloud SDK Continuous Delivery Pipeline
are already pre-installed. If you update or downgrade them to a specific version, it will be lost every time the `cx-server` image is updated. All the plugins are updated with the latest version.
If there is a need, the user can install additional plugins and configure them.
However, the `cx-server update` will not update the plugins that are custom installed.

#### Troubleshooting

##### Disk space cleanup
If you encounter an issue related to diskspace on a cx-server, you can free up space by launching [system prune](https://docs.docker.com/engine/reference/commandline/system_prune/) command.

***WARNING***
Expand All @@ -141,3 +164,15 @@ Do not launch this command when cx-server is not running. Because the command wi
```bash
docker system prune --all
```

##### Logs

You can find the logs of the `cx-server` and the caching server as part of the Docker logs on the host machine.

```
docker logs s4sdk-jenkins-master
docker logs s4sdk-nexus
```



Binary file added images/s4sdk-pipeline.gif
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed images/s4sdk-pipeline.png
View file
Binary file not shown.
15 changes: 12 additions & 3 deletions s4sdk-pipeline.groovy
View file
@@ -1,6 +1,6 @@
#!/usr/bin/env groovy

final def pipelineSdkVersion = 'v12'
final def pipelineSdkVersion = 'v13'

pipeline {
agent any
Expand All @@ -13,6 +13,7 @@ pipeline {
stages {
stage('Init') {
steps {
milestone 10
library "s4sdk-pipeline-library@${pipelineSdkVersion}"
stageInitS4sdkPipeline script: this
abortOldBuilds script: this
Expand All @@ -21,6 +22,7 @@ pipeline {

stage('Build') {
steps {
milestone 20
stageBuild script: this
}
}
Expand Down Expand Up @@ -56,7 +58,10 @@ pipeline {
}

stage('Quality Checks') {
steps { stageS4SdkQualityChecks script: this }
steps {
milestone 50
stageS4SdkQualityChecks script: this
}
}

stage('Third-party Checks') {
Expand Down Expand Up @@ -88,11 +93,15 @@ pipeline {

stage('Artifact Deployment') {
when { expression { commonPipelineEnvironment.configuration.runStage.ARTIFACT_DEPLOYMENT } }
steps { stageArtifactDeployment script: this }
steps {
milestone 70
stageArtifactDeployment script: this
}
}

stage('Production Deployment') {
when { expression { commonPipelineEnvironment.configuration.runStage.PRODUCTION_DEPLOYMENT } }
//milestone 80 is set in stageProductionDeployment
steps { stageProductionDeployment script: this }
}

Expand Down

0 comments on commit 9ca4365

Please sign in to comment.